# ProstDev — Full Content Export > ProstDev — where MuleSoft mastery meets fun. Easy-to-follow video tutorials, AI experiments, and deep-dive articles on MuleSoft, DataWeave, Anypoint Code Builder, and integration. --- # Blog Posts ## Recap skills: catch up on any Claude session Source: https://prostdev.com/post/recap-skills-catch-up-on-any-claude-session | Published: Sep 1, 2026 | Category: Guides You step away from a session. A meeting, lunch, the end of the day. When you come back, the context is gone. You are staring at a wall of earlier messages trying to remember what you were even doing, whether the thing you kicked off ever finished, and what the agent was waiting on you to decide. I got tired of reconstructing that by hand every time, so I wrote a skill for it: `recap`. You open the session, type the trigger, and get a 30 to 60 second read that tells you the goal, where things actually stand, and what is on you versus what the agent will pick up once you say go. It is read-only on purpose, so it never surprises you by resuming work while your back was turned. There are actually two versions of it, because a recap means something different depending on where you are working. Let me show you both and when each one fits. ## TL;DR - `recap` catches you up when you reopen a Claude session cold: it gives a short, read-only report of the goal, where things stand, and a next-steps list split into `[You]` and `[Me]`. - There are two versions because the state that goes stale differs by product. The Claude Code one (`dev` plugin) refreshes agents, background jobs, PRs, and your git tree; the other (`general` plugin) refreshes artifacts, project files, and connector actions in the Claude app, claude.ai, and Cowork. - Both are strictly read-only. A recap never resumes work, pushes a commit, or finishes an artifact while catching you up, so it stays a truthful snapshot. - Install from [ProstDev/skills](https://github.com/ProstDev/skills) with `/plugin marketplace add ProstDev/skills` in Claude Code, or `npx skills add ProstDev/skills` in any other agent. ## Why two recaps A recap is only useful if it checks the *real* current state, not what the conversation last claimed. Earlier messages go stale the moment something finishes in the background or you push a commit yourself. So the skill has to go look. The problem is that "what to go look at" is completely different across Claude products. In Claude Code, the things that drift are agents, background jobs, pull requests, and your git working tree. In the Claude app, on claude.ai, or in Cowork, none of those exist. What drifts there is an artifact you were editing, a file in your project, a long generation that was still running, or a connector action that may or may not have actually landed. One skill trying to cover both would either be vague or full of "if you are in Claude Code, do this, otherwise do that" branches. So I split it in two. Same shape, same report, different refresh step. ## What they share Both skills follow the same three steps, and the report at the end is identical: - **Refresh (read-only).** Go check whatever this session actually put in flight. Skip anything that does not apply. - **Reconstruct the goal.** State it in one line, in your terms, not in tool or implementation language. If it is genuinely unclear, say so instead of guessing and presenting the guess as fact. - **Report.** Plain language, skimmable in under a minute: the **goal**, the **status** in a few lines, anything **needed from you**, and a **next-steps** list where every line is tagged `[You]` or `[Me]`. That last part is the piece I use the most. The mine-versus-yours split means I can glance at the report and immediately see what I have to unblock versus what the agent will handle on its own the moment I give the word. > [!NOTE] > Both skills are strictly read-only. During a recap they will not fix a failing check, push a commit, finish an artifact, or resume an unfinished task, even when the fix looks small and obviously right. A recap's whole job is a truthful snapshot. Mixing in new work defeats the point, so anything actionable shows up under "needs from you" or as a `[Me]` next step, not something done on the spot. ## The Claude Code recap This one lives in the `dev` plugin. Its refresh step knows about the things a coding session leaves running: - **Background agents and subagents** it spawned. If one is still going, it gets reported as in-progress, not waited on. - **Background shell or workflow jobs** started with `run_in_background`. - **Scheduled work** like a cron routine or a loop the session set up. - **An open pull request** it opened or is waiting on. It pulls the live CI and review state with `gh pr view` and `gh pr checks` rather than trusting what the conversation last said. - **Uncommitted work.** It runs `git status` and `git diff` in every repo the session touched, so an earlier "done" claim gets checked against what is actually on disk. That git check is the one that saves me most often. It is very easy to believe a task is finished because the conversation said so, then find out the changes were never actually written or were half-applied. The recap catches that gap. ## The recap for the rest of Claude This one lives in the `general` plugin, and it is the one that runs in the Claude app, on claude.ai, and in Cowork. Its refresh step looks at a different world: - **Artifacts and documents** produced this session. Is the latest version really the one being described, and is it finished or mid-edit? - **Project or uploaded files.** It reads the current file instead of trusting an earlier paste of it. On a filesystem-backed surface like Cowork, that includes anything written to disk. - **Async or background work**, like a long generation or export the session kicked off. - **Connector state.** If the session acted through Slack, Google, GitHub, or another connector, it checks whether the action actually landed (message sent, doc saved, issue opened) instead of assuming. - **Nothing but the thread to inspect?** Then the conversation itself is the source, and the report says so plainly rather than implying it verified something it could not. ## When a recap needs a decision from you Sometimes the honest status is "this is blocked on you." When that happens, the skill does not dump three paragraphs of context and make you parse it. It hands off to another skill, [`clarify-request`](https://github.com/ProstDev/skills/blob/main/plugins/general/skills/clarify-request/SKILL.md), so the decision arrives as one clear question at a time with a recommended answer. If nothing is actually blocked, it says so. It will not invent a question just to fill the slot. That restraint is deliberate. A recap you can trust to be honest about "nothing needed from you, here is where things stand" is worth far more than one that manufactures busywork. ## How to install them Both recaps live in my public skills repo, [ProstDev/skills](https://github.com/ProstDev/skills), alongside the rest of the skills I use on every session. If you are in Claude Code, add the marketplace once and install the plugin that has the recap you want: ```bash /plugin marketplace add ProstDev/skills /plugin install dev@prostdev # the Claude Code recap /plugin install general@prostdev # the recap for the app, claude.ai, and Cowork ``` The Claude desktop app and claude.ai support only the `general` plugin, so that is the one to add there through Settings, then Plugins, then Add. And if you are in any other agent, `npx skills add ProstDev/skills` copies the files straight into your repo. Once installed, you do not have to remember a command. Just ask for a recap, or say something like "catch me up" or "where did we leave off," and the skill's description takes it from there. ## Wrapping up The recap skills exist because returning to a session with zero loaded context is a tax I pay constantly, and reconstructing it by hand is slow and error-prone. Two versions, one for Claude Code and one for everywhere else, because the thing worth checking is genuinely different in each place. Both give you the same honest, read-only snapshot: here is the goal, here is where it really stands, here is what is on you. If you want the background on what agent skills even are and how they load, I wrote a [beginner's guide to AI agent skills](/post/what-are-ai-agent-skills-and-how-to-use-them/) that covers the `SKILL.md` format and how an agent picks the right one for your task. --- ## Sound and desktop alerts for Claude Code with hooks Source: https://prostdev.com/post/sound-and-desktop-alerts-for-claude-code-with-hooks | Published: Aug 31, 2026 | Category: Tutorials If you use Claude Code for anything that runs for a while, you know the feeling. You kick off a long task, tab away to read something or answer a message, and then forget to check back. Minutes later you realize Claude finished ages ago. Or worse, it's been sitting there the whole time, blocked on a permission prompt, waiting for you to click "yes." I wanted a nudge. A sound and a little desktop banner that tells me two things: "Claude is done" and "Claude needs you." So I wired it up with the hooks system, and it took about five minutes. Let me show you how. This is especially handy for long-running work, the kind where you kick off a big task and step away instead of staring at the terminal the whole time. ## TL;DR - Add two hooks to `~/.claude/settings.json`: a `Stop` hook (fires when Claude finishes a turn) and a `Notification` hook with the `permission_prompt` matcher (fires when Claude is blocked on your approval). - Each hook plays a different macOS sound with `afplay` (so you can tell "done" from "needs you" by ear) and posts a Notification Center banner with `osascript`. - `jq -r '.cwd' | xargs basename` puts the project folder name in the banner title, so you know which session is calling when several are open. - The commands are macOS-specific, but the hook wiring in `settings.json` is identical on any OS. The full config is below. ## What are hooks? Claude Code has a hooks system in `settings.json`. Hooks are just shell commands that fire on certain lifecycle events. You tell Claude "when X happens, run this command," and it runs it. Two events matter for our alerts: - **`Stop`** fires every time Claude finishes responding. - **`Notification`** (with the matcher `permission_prompt`) fires when Claude is blocked waiting for you to approve a tool call. Each hook receives a JSON payload on standard input with fields like `cwd` (the project directory) and `session_id`. A shell command can read that payload to add some context, which we'll use to put the project name in the banner. ## The config Here's what I added to `~/.claude/settings.json`. I put it at the user level (that `~` path) so it applies to every project and every session globally, not just one repo. ```json title="~/.claude/settings.json" "hooks": { "Stop": [ { "hooks": [ { "type": "command", "command": "afplay /System/Library/Sounds/Glass.aiff" }, { "type": "command", "command": "proj=$(jq -r '.cwd' | xargs basename); osascript -e \"display notification \\\"Claude finished\\\" with title \\\"Claude Code — $proj\\\"\"" } ]} ], "Notification": [ { "matcher": "permission_prompt", "hooks": [ { "type": "command", "command": "afplay /System/Library/Sounds/Ping.aiff" }, { "type": "command", "command": "proj=$(jq -r '.cwd' | xargs basename); osascript -e \"display notification \\\"Claude needs your permission\\\" with title \\\"Claude Code — $proj\\\"\"" } ]} ] } ``` If your `settings.json` already has other keys, add the `hooks` block alongside them (don't paste a second top-level object). And since it's JSON, watch the commas. ## How it works, piece by piece There are only three moving parts here, so let's break them down. **`afplay` plays a built-in macOS sound.** I use `Glass.aiff` when Claude finishes and `Ping.aiff` when it needs permission. Two different tones on purpose, so I can tell by ear which situation it is without looking. Those files live in `/System/Library/Sounds/`, so go poke around in there and pick whatever you like. **`osascript -e 'display notification ...'` posts a native Notification Center banner.** The nice thing about going through AppleScript is that it's independent of which terminal you're using. Terminal.app, iTerm2, and the VS Code integrated terminal all behave exactly the same. **`jq -r '.cwd' | xargs basename` pulls the project name out of the hook's payload.** Remember that JSON on standard input? `jq` grabs the `cwd` field, and `basename` trims it down to just the folder name. So instead of a generic "Claude finished," the banner title reads something like `Claude Code — my-mule-app`. That is genuinely useful when you've got a few sessions open in different terminals and need to know which one is calling. ## A few gotchas I hit Nothing here was hard, but a couple of things tripped me up on the way, and they're worth knowing. If you test `afplay` or `osascript` through Claude's normal Bash tool, they'll fail with errors like `AudioQueueStart failed` or `nice() failed: operation not permitted`. That's the sandbox blocking audio and AppleEvents. To pipe-test the real commands, you have to explicitly bypass the sandbox. The good news: the actual hooks aren't sandboxed the same way at runtime. Hooks run as regular subprocesses, so they work fine once they're wired into `settings.json`. The second thing: **you can't control how long the banner stays on screen from the hook.** `display notification` has no duration parameter. Whether a banner disappears after a few seconds or sticks around is a macOS-wide choice, set per app under **System Settings** > **Notifications** (the "Banners" vs. "Alerts" style). It's not something the hook script can override. > [!TIP] > If you want the alert to stay put until you dismiss it, set your terminal app (or `osascript`) to "Alerts" instead of "Banners" in **System Settings** > **Notifications**. ## Why I kept it simple I originally wanted to put a snippet of Claude's last message into the banner for extra context. The `Stop` hook's payload has a `last_assistant_message` field available, so it's doable. But once I started wrestling with the `jq` and shell-escaping to get a clean one-liner, the complexity wasn't worth the payoff. The project name alone already answers the only question I actually have in that moment, which is "which session is this?" So I left it out. Sometimes the simple version is just the right version. ## One caveat: this is macOS-specific `afplay` and `osascript` are macOS tools, so this exact config won't work on Linux. The good news is that the hook wiring in `settings.json` is identical everywhere. You'd only swap the commands: - For sound, use something like `paplay` or `aplay`. - For the desktop banner, use `notify-send`. Everything else (the `Stop` and `Notification` events, the matcher, the `jq` payload trick) stays exactly the same. That's it. Five minutes of setup, and now I actually notice when Claude is done or waiting on me. Go tab away with confidence. :-) --- ## One idea, four products: the tech and product side of the gig economy Source: https://prostdev.com/post/gig-economy-same-idea-different-products | Published: Aug 20, 2026 | Category: Opinion If you strip the gig economy down to its core, almost every app is selling the same promise: own your own time, be your own boss, work when you want. On the other side of the marketplace, the customer is buying a slightly different version of that same idea: get what you need, from someone available, without a full-time hire. So you'd think all these apps would look and feel the same. They don't. And the reason they don't is the fun part. Take four of them: [Uber](https://www.uber.com), [TaskRabbit](https://www.taskrabbit.com), [Rover](https://www.rover.com), and [CleaningPal](https://cleaningpal.co). Same starting point, same two-sided marketplace shape, and yet four very different products, four different customer journeys, and four different piles of tech and trust behind the scenes. Let's see why. ## TL;DR - All four apps start from the same idea (a marketplace where people sell their time), but the **nature of the task** pulls each product in a different direction. - **Uber** optimizes for speed. You don't pick your driver, you just want a ride now, so confirmation happens in minutes. - **TaskRabbit, Rover, and CleaningPal** let you choose your provider, because the task is personal, slower, and often happens in your home. - The scarier the access (a stranger alone in your house, alone with your pets), the more the product invests in **trust**: profiles, chat, meet-and-greets, background checks, insurance, and emergency support. - Same path in, completely different products, journeys, tech stacks, and reassurance mechanisms out. That's the interesting part. ## The one thing they all share Underneath, these are all marketplaces. There's supply (drivers, taskers, sitters, cleaners) and there's demand (you), and the app's job is to match the two, handle payment, and take a cut. That's the shared skeleton. But a skeleton isn't a product. The moment you ask *what am I actually buying, and how do I feel about the person delivering it?*, the four apps split apart. The variable that splits them is the task itself: how fast you need it, whether you get to choose who shows up, and how much of your life that person gets access to. ## Uber: speed wins, so you don't get to choose With Uber, you want a ride and you want it now. Or you scheduled it ahead, but either way the whole point is that a car appears without you thinking about it. Because there are so many drivers, and because the service is measured in minutes, confirmation happens almost instantly. You tap, and within minutes someone is on their way. You don't browse driver profiles, you don't message three of them first, you don't schedule a coffee to see if you vibe. You just want the service, from whoever is available, as soon as possible. That "whoever is available" part is a real product decision. Choosing your driver would slow everything down, and slowing down is the opposite of what the ride-hailing product is for. So Uber quietly takes the choice away, and that's a feature, not a bug. > [!NOTE] > Notice how the product removes a decision on purpose. Fewer choices means faster confirmation, and speed is the entire value of a ride. The task shaped the product. ## TaskRabbit: now you get to choose, and time stretches out TaskRabbit flips two of those things at once. First, you *can* choose your tasker. You look at profiles, reviews, and rates, and you pick the person you want. Second, confirmation is no longer guaranteed in minutes. Depending on the tasker you contact first, you might hear back in minutes, or it might take days. The person on the other end is a human with their own schedule, not a pool of cars circling the block. Why the change? Because the task is different. Generally, a tasker comes to your house to assemble furniture, mount a TV, or help you move. That's someone in your space, so it makes sense to be more selective about who you let in. The product gives you that selectivity on purpose, and the tradeoff is that "instant" turns into "it depends." ## Rover: when trust is the whole product Rover is where this gets serious, and honestly a little scary when you think about it. Depending on the service, a pet sitter might just walk your dog while you're home. But a lot of the time, especially if you're out of town, that sitter has access to your home, to your key, and to your pets. Someone you found in an app is alone in your house with the things you love most. So for Rover, trust isn't a nice-to-have. It's the product. And you can see that in every feature: - **Chat with several sitters beforehand.** You're not forced to commit to the first person who replies. You can talk to a few and get a feel for them. - **Schedule a meet-and-greet.** You can meet the sitter in person, watch them hang out with your pets, and decide for yourself if this is someone you'd hand a key to. That's a huge deal, and it's baked right into the flow. - **A small insurance for the customer** in case of damage or stolen property, so there's a safety net if something goes wrong. - **A way for the sitter to reach emergency services** if a pet needs urgent care while you're away. Put those together and you get a product designed to reassure *both* sides. The customer feels safe letting someone into their home, and the sitter feels supported if something goes wrong on their watch. Compare that to Uber, where you never even learn the driver's last name, and you can see how far apart the same starting idea can land. > [!IMPORTANT] > The rule of thumb across all four: the more access the provider has to your home and your life, the more the product has to invest in trust. Rover has the most access, so it has the most trust machinery. ## CleaningPal: vetting as the value CleaningPal sits in a similar spot. Most of the time you'll be home when the cleaner arrives, again because of the nature of the service, but you might also want the place cleaned while you're out. Either way, you're letting someone into your home. What CleaningPal leans on is vetting. The cleaners are local, and they've been background-checked, interviewed, and vetted, so you're getting a trusted local cleaner and not just a random person. That's the whole pitch, and it's a direct answer to the alternative: finding a cleaner on Craigslist, Kijiji, or even a Facebook group, where you have no idea who's actually showing up (if they even show up at all!). So the reassurance mechanism is different from Rover's. Rover hands *you* the tools to judge a sitter (chat, meet-and-greet). CleaningPal does the judging *for* you up front (background checks, interviews). Same underlying worry, "can I trust this stranger in my home?", solved two different ways. And there's a piece of the CleaningPal journey that the other three don't offer nearly as easily: you can always reach CleaningPal directly by email or phone. If a cleaner doesn't show up, or if something goes wrong with the cleaner, you talk to an actual person and they make it right. They'll give you alternatives when something goes wrong: a refund, a different (trusted) cleaner, or a reschedule. Try getting a human on the phone the moment your Uber cancels, or when a tasker ghosts you. That direct line is a real reassurance, and it's part of the product. ## Same path, four different products Here's the comparison side by side: | App | Confirmation | Choose your provider? | Access to your home | Main trust mechanism | | --- | --- | --- | --- | --- | | **Uber** | Minutes (or prescheduled) | No, you want whoever's available | None | Ratings, tracked trip | | **TaskRabbit** | Varies (minutes to days) | Yes | Usually, they come to you | Profiles + reviews, you pick | | **Rover** | Varies | Yes | Often, key + pets while you're away | Chat, meet-and-greet, insurance, emergency line | | **CleaningPal** | Varies | Yes | Usually, sometimes while you're out | Vetted local cleaners + direct human support (refund, replacement, or reschedule) | Look at how much moves once you change the task. Confirmation speed, whether you choose, how much access the provider gets, and the entire trust layer all shift. And each of those shifts ripples into the tech behind the app: Uber lives and dies on real-time matching and location; Rover needs messaging, scheduling, insurance, and an emergency pathway; CleaningPal needs a vetting and background-check pipeline before a cleaner ever appears in the app. It's genuinely interesting to me how similar paths lead to completely different products, customer journeys, tech stacks, and reassurance strategies. They're all marketplaces of a sort, all contributing to the same gig economy, all selling "own your time." But the second you ask what's really being bought and who's showing up, they become four different things. That's the lesson I keep coming back to as a builder: the idea is almost never the product. The *context* is. The same promise, filtered through a different task, becomes a different app. I hope this was a fun look at the product side of these apps you probably use all the time. If you've noticed other little product decisions hiding in the gig apps you use, I'd love to hear them, maybe there's something clever I've missed. :-) Next time an app makes a choice for you, or hands one to you, ask yourself what task shaped that decision. Once you start seeing it, you can't unsee it. Thanks for reading! -Alex. --- ## AI Showdown Hard Mode: Claude Code vs MuleSoft Vibes Source: https://prostdev.com/post/ai-showdown-hard-mode-claude-code-vs-mulesoft-vibes | Published: Aug 18, 2026 | Category: Opinion Same ring, harder rules. In [the last round](/post/ai-showdown-3-ais-design-api-led-connectivity-in-mulesoft) I handed three AIs a MuleSoft architect brief and let them build a 3-layer API-led network for a Customers + Orders domain. The catch that time: half the good stuff was optional, so the tools that skipped specs, tests, and persistence weren't technically breaking any rule. This round, I closed that loophole. **Hard mode makes every architect-level requirement mandatory**: spec-first with APIkit, security enforced *and* documented in the spec, multi-environment secure properties, a global error handler, MUnit for the two guarded business rules, persistence that survives a restart, a parent POM, and the latest, identical versions across every app. No "the prompt didn't ask for it" escape hatch. There's nowhere to hide. Two contestants this time, and the matchup is the interesting part. > [!TIP] > Want the exact prompt both AIs received? The full [instructions I gave them](https://github.com/alexandramartinez/ai-showdown-hard/blob/main/instructions-given.md) are in the repo, and you can browse [the whole project here](https://github.com/alexandramartinez/ai-showdown-hard). ## The bet - 🟦 **Claude**: Claude Code, Opus 4.8, MAX effort - 🟩 **MuleSoft Vibes**: Sonnet underneath, but loaded with the full MuleSoft skill packs Vibes runs on **Sonnet, a smaller model than Opus 4.8**, but it ships with the whole MuleSoft skill set baked in. So *in theory* the domain scaffolding should let it punch above its model weight. Hard mode is exactly the test for that theory: if the skills work, the gap from last round should close. If they don't, the finicky mandatory details are where it'll show. > [!NOTE] > These aren't comparable tiers of compute: different models, different effort levels. Read this as "what each tool produced from the same hard-mode brief," not a controlled benchmark. What makes it fair is that both got the *identical* prompt with the *identical* mandatory list. ## TL;DR Both built genuinely API-led solutions this time, and that alone is a big deal, because last round Vibes skipped almost everything. But each side has exactly **one clear miss** against the mandatory list: - **Claude** (Opus 4.8) is the tighter engineering artifact: real encrypted secrets for every environment, a true reactor POM, 84 MUnit tests that actually **built green** under an enforced coverage gate, and zero duplication. Its miss: it folded the process layer into the experience app, so it ships only **two layers as real APIs**, a literal miss on the "one API per layer" rule. - **MuleSoft Vibes** (Sonnet + skills) delivered the more textbook topology: a genuine **separate process tier per entity** and real ObjectStore persistence. Its miss: it left the **secrets as fill-in-the-blank placeholders**, so security can't actually authenticate as shipped, and it copy-pasted the error handler across all five apps. **The skills narrowed a wide gap to a narrow one. They didn't erase the model's edge on the finicky, mandatory, easy-to-fake details.** ## The brief (hard mode) Same domain as last round (a Customers + Orders network across Experience, Process, and System) with these 11 operations: 1. List all customers 2. Get one customer's details 3. Get a customer's orders 4. Get one order's details 5. Create a customer 6. Edit a customer 7. Delete a customer **only if they have no orders** 8. Create an order attached to a customer 9. Edit an existing order 10. **Cancel** an order (a status change, *not* a delete) 11. Delete an order Operations #7 and #10 are business-rule traps on purpose. What changed this round is the requirements list, **all mandatory now:** 1. **Spec-first**: design each API as OAS or RAML, implement against it with APIkit. 2. **Security**: enforce client-id/secret on the experience layer *and* document the scheme in the spec. 3. **Multi-env properties**: externalize everything into dev/test/prod files, with secrets as *secure properties, not hardcoded*. 4. **Error handling**: one consistent global handler in every app, returning structured errors. 5. **Tests**: MUnit for the two guarded rules. 6. **Persistence**: mock and seed the data as if wired to a real DB, and it must survive a restart. 7. **Best practices**: consistent naming, no copy-pasted flows, a parent POM, and the latest identical versions everywhere. ## The scorecard | Requirement (all mandatory) | 🟦 Claude | 🟩 MuleSoft Vibes | |---|---|---| | **Apps** | ❌ 3 (exp + prc folded · 2 sys) | ✅ 5 (1 exp · 2 prc · 2 sys) | | **All 3 layers as real APIs** | ❌ No (process folded into experience) | ✅ Yes (dedicated process apps) | | **Parent POM** | ✅ True reactor (``, one build) | ⚠️ Version-mgmt only (no reactor) | | **Spec-first (OAS) + APIkit** | ⚠️ Yes, 3/3 apps | ✅ Yes, **5/5 apps** | | **11 operations** | ✅ 11/11 functional | ✅ 11/11 functional | | **Security enforced** | ✅ Yes + **real ciphertext secret** | ⚠️ Yes, but **secret is a placeholder** | | **Multi-env props (dev/test/prod)** | ✅ Yes | ✅ Yes | | **Secrets as real secure properties** | ✅ Yes (dev, test, **and** prod) | ❌ No (placeholders, dev only) | | **Global error handler** | ✅ **One shared file**, imported | ❌ Copy-pasted across 5 apps | | **Delete-only-if-no-orders (#7)** | ✅ Yes (test: DELETE never fires) | ✅ Yes (test: 409) | | **Cancel ≠ delete (#10)** | ✅ Yes (test: count unchanged) | ✅ Yes (test: status) | | **Persistence survives restart** | ✅ File-backed JSON | ✅ **ObjectStore `persistent="true"`** | | **MUnit tests** | ✅ **84**, green build, ≥80% gate | ⚠️ 27, no gate, not built here | | **Latest, identical versions** | ⚠️ Mule 4.11.5, all pinned | ⚠️ Mule 4.9.4, all pinned | > [!NOTE] > **Security enforced** and **Secrets as real secure properties** are really a matter of preference. Some people would rather the AI *not* drop real secrets into the project at all and generate them themselves; others prefer it filling in at least the local/dev values so the app can actually run and be tested out of the box. Neither is wrong. Either way, you should **re-add all the secrets yourself for prod (and any environment that isn't local/dev)**, never trust AI-committed values there. ## 🟦 Claude: the tighter engineering artifact Claude shipped **three apps** and a **true reactor POM** (`packaging=pom` with a `` list), so a single `mvn clean package` at the root compiles, tests, coverage-gates, and packages all three apps in one command. Children declare zero versions; everything is centralized in the parent. It went fully spec-first with OAS 3.0.3 on all three apps, and modeled the domain strictly: distinct request vs. response types (`CustomerInput` vs. `Customer`), a shared `ApiError` schema, and an order status enum. Because the APIkit scaffolder plugin sits behind an auth-gated EE Nexus that wasn't available, Claude **hand-authored** the router flows to match each spec's contract exactly, then wrote dedicated router-error MUnit suites to lock that fragile flow-name-to-spec mapping (a typo there is a silent 404). Two things stand out: - **Real secrets, everywhere.** The client secret and internal gateway key are genuine Blowfish/CBC ciphertext, filled in for **dev, test, and prod**. The decryption key is supplied only at launch via `-Denc.key=…`, never committed. Security runs as shipped. - **Proof it works.** The tree contains packaged jars and surefire reports: the **84 MUnit tests actually ran here and came back 0 failures, 0 errors**, under a parent-POM coverage gate (`requiredApplicationCoverage=80`, `failBuild=true`). That's an acceptance gate, not just a test folder. Both business rules are proven adversarially: the delete-guard asserts a 409 **and** `verify-call times="0"` on the downstream customer DELETE (proving no destructive call leaks out), and cancel asserts the order **count is unchanged** (record kept, not deleted). The catch: Claude folded the process concern into the experience app. It documented the call. API-led names three *concerns*, not necessarily three *deployables*, and its process section talks only to the System APIs, never a datastore. It even names the boundary "the promotion seam," so it lifts out into a standalone Process app if a second channel ever appears. A defensible engineering call, **but the prompt didn't leave it to judgment.** It required "at least three, one per layer," and a process *concern* inside the experience deployable is not a process *API*. So Claude ships only two layers as real APIs: a literal miss on a mandatory clause. ## 🟩 MuleSoft Vibes: the textbook topology This is the big turnaround. Last round, Vibes skipped specs, APIkit, tests, the error handler, and its writes didn't persist. **This time it did all of them.** Vibes chose **five apps**, the fullest, most literal reading of API-led: one experience, a **separate process app per entity** (customers-process, orders-process), and a system app per entity. The delete-guard lives in customers-process (which reaches into orders-system for the cross-entity check); cancel-vs-delete lives in orders-process. This is the more textbook "one API per concern" topology (the design most MuleSoft courseware would draw) and it's the only one of the two that literally satisfies the "one API per layer" requirement. It's genuinely spec-first now, with OAS on **all five apps** (~32 operations across the tiers), so every internal hop is contract-first, not just the edge. And the persistence turnaround is the clearest evidence the skills paid off: **ObjectStore v2 with `persistent="true"`**, seeded once at startup, with real CRUD writes (verified 8 `os:store` calls per system app) that survive a restart. It has 27 MUnit tests covering both guarded rules plus customer-id immutability and invalid-customer-on-create. But the two mandatory requirements that are *fiddly rather than conceptual* are exactly where it slipped: - **Secrets left as placeholders.** The secure-properties machinery is correct (AES/CBC, key supplied at runtime, enforced before routing), but the values are unfilled: `id: "![REPLACE_WITH_ENCRYPTED_DEV_CLIENT_ID]"`. And there's only a `config-secure-dev.yaml`, no test or prod secure file. Vibes technically avoided hardcoding, but by leaving the secret blank, so its security **cannot authenticate anyone** until a human encrypts real values first. A literal miss on a mandatory requirement. - **Copy-pasted error handler.** The global handler is duplicated across all five apps in two variants: exactly the copy-pasted-flow duplication the prompt told both AIs to avoid. Its parent POM also governs versions (`dependencyManagement` + `pluginManagement`) but has **no ``**, so you must `mvn install` the parent first, then build each of the five apps individually. And nothing was built in the repo: no `target/`, so the 27 tests are asserted but unproven here. > [!WARNING] > One more practical snag: I couldn't run the Vibes projects in Anypoint Code Builder at all. VS Code + ACB needs a VS Code workspace to open the apps as Mule projects, and Vibes never created one, so there was nothing for ACB to load. It had to be wired up by hand before it could run. ## Where the two really diverge Both are genuinely API-led now, so the story is in the details each got right, and each side owns exactly one miss: | Dimension | 🟦 Claude | 🟩 Vibes | |---|---|---| | **Topology** | 2 real layers (process folded) ❌ | 3 real layers ✅ | | **Parent POM** | True reactor, one build ✅ | Version mgmt only ⚠️ | | **Secrets** | Real ciphertext, all envs ✅ | Placeholders, dev only ❌ | | **Error handler reuse** | Authored once ✅ | Copy-pasted ×5 ⚠️ | | **Tests + proof** | 84, green build, gate ✅ | 27, no gate, unbuilt ⚠️ | | **Persistence** | File-backed JSON ✅ | ObjectStore `persistent` ✅ | | **Runtime** | Mule 4.11.5 (recent LTS) | Mule 4.9.4 (older) | **Vibes wins topology; Claude wins execution rigor.** Vibes' 5-app split is both the more by-the-book architecture and the compliant one, its strongest card. Claude's secrets, reactor build, coverage gate, and zero duplication are the un-glamorous, mandatory details that are easy to fake and hard to actually finish. ## The version reality check The brief said "latest, identical versions." Both used real, released versions and Java 17 (no hallucinated numbers) and both centralized versions in the parent so their apps are internally consistent. But **neither reached the true-latest Mule 4.12.0.** | Component | 🟦 Claude | 🟩 Vibes | Latest GA | |---|---|---|---| | Mule runtime | ⚠️ 4.11.5 | ❌ 4.9.4 | 4.12.0 | | Java | ✅ 17 | ✅ 17 | 17 | | mule-maven-plugin | ✅ 4.10.1 | ❌ 4.3.0 | 4.10.0 | | APIkit | ⚠️ 1.11.17 | ✅ **1.12.0** | 1.12.1 | | HTTP connector | ✅ 1.11.3 | ⚠️ 1.10.3 | 1.11.3 | | MUnit | ⚠️ 3.7.0 | ❌ 3.3.0 | 3.7.1 | Claude is on the more current line across the board and documents *why* 4.11.5 (it's the newest runtime whose embedded EE distribution is cached locally, so MUnit can boot offline). Vibes is a generation behind on almost everything, except APIkit, where its 1.12.0 is actually newer than Claude's 1.11.17. Same lesson as last round: AIs anchor to whatever versions dominated their training data, not to what shipped last week. **Pin your versions yourself.** ## So did the skills pay off? Mostly, and that's the real story. A Sonnet-based tool carrying the MuleSoft skill packs produced a legitimately API-led solution that, unlike its earlier-showdown self, has specs on every tier, APIkit, a wired global error handler, real ObjectStore persistence that survives a restart, and MUnit coverage of both guarded rules. On *conceptual completeness* and *textbook topology*, Vibes is right there, and on the "one API per layer" clause, it's flatly ahead. Where Opus 4.8 still wins is execution rigor: real secrets everywhere, no copy-paste, a one-command reactor that proved itself green under an enforced gate, and a more current version stack with the trade-offs written down. **Each side has exactly one clear miss:** Vibes left its secrets as placeholders (security can't run as shipped); Claude shipped only two layers as real APIs (no dedicated process tier). - **Want the cleaner textbook diagram, a genuine process tier, and the letter of the "one API per layer" rule?** 🟩 **MuleSoft Vibes**: the more canonical *and* more compliant architecture, and a big step up from last round. - **Want the thing you could promote to production tomorrow** (secrets that decrypt, one build that proves itself, zero duplication, an enforced quality gate)? 🟦 **Claude**: still the stronger engineering artifact. The skills closed a wide gap to a narrow one. They didn't erase the model's edge on the finicky, mandatory, easy-to-fake details, even as Claude gave one of those details back on topology. ## What's next? **AI Showdown: MuleSoft Edition (2026) 🥊** is an ongoing series comparing AI coding tools on the same MuleSoft and DataWeave challenges. Round 1 was DataWeave puzzles; Round 2 was API-led architecture; this was the hard-mode rematch. 🎯 What should the next round test, and which AI should join the ring? Drop a comment on YouTube or reply to this post. 🔔 Subscribe so you don't miss the next round: [youtube.com/prostdev](https://www.youtube.com/prostdev) --- ## ACB deploys your app with the local port instead of 8092? Your properties are resolved at build time Source: https://prostdev.com/post/acb-cloudhub-wrong-port-force-parse-config-xmls | Published: Aug 3, 2026 | Category: Guides Another one from the [**MuleSoft Community Slack**](https://docs.google.com/forms/d/e/1FAIpQLScfuc3_R8sEl23xLkZBPIs6n7--HFhyuZewJiJKsKzQbnY9HQ/viewform) workspace, and this is a great example of a bug that looks like an environment/config problem but is really a *timing* problem: when your properties get resolved. The setup: someone uses `${https.private.port}` for their HTTP listener port so that each environment can decide it. Locally they set `https.private.port=1234`. In PROD they deliberately leave it out of the config so CloudHub assigns the reserved internal port **8092** automatically. `mule.env=PROD` is passed on deploy. Straightforward, and it's exactly how per-environment properties are supposed to work. The problem: deploy the *same* app from **Anypoint Studio** and it comes up on **8092** as expected. Deploy it from **Anypoint Code Builder (ACB)** and it comes up on **1234**, the local value, in CloudHub. Same code, same `mule.env=PROD`, two different results. ## TL;DR ACB resolves `${https.private.port}` at **build time**, against your **local** config, and bakes `1234` into the packaged artifact. CloudHub then never re-resolves it in PROD, so it never falls through to `8092`. Anypoint Studio doesn't hit this because it sets a runtime argument that forces a fresh parse of the config XMLs at deploy time. Add the same argument to ACB: ```text -Dmule.deployment.forceParseConfigXmls=true ``` Put it in ACB **Settings** under **`Mule > Runtime: Default Arguments`**. Redeploy, and the listener comes up on **8092** in PROD. It's environment-agnostic, so you never have to touch it when switching between TEST and PROD. > [!TIP] > New to runtime arguments in ACB? I walk through exactly where that setting lives in > [How to add JVM/Command-line arguments to the Mule 4 Runtime in Anypoint Code Builder (ACB)](/post/how-to-add-jvm-command-line-arguments-to-the-mule-4-runtime-in-anypoint-code-builder-acb/). ## The symptom The tell is that the value being used is your **local** one, in a **remote** environment: - Deploy from **Studio** → the listener binds to **8092** in CloudHub. Correct. - Deploy from **ACB** → the listener binds to **1234** in CloudHub, even though PROD has no `https.private.port` defined and `mule.env=PROD` is set. - The PROD config file genuinely has no `https.private.port`, on purpose, so CloudHub can assign the reserved internal port itself. Because `1234` only exists in the *local* properties file, seeing it show up on a CloudHub worker is the whole clue. Something resolved that placeholder while your local config was still the active one. ## The red herring: "it's ignoring mule.env" The natural first theory is that ACB isn't passing `mule.env=PROD` to the runtime, so it loads the local config in CloudHub. That's a reasonable guess, and it's *almost* right, but not quite. The environment isn't being ignored at runtime. The issue is that the property was already resolved **before** the app ever reached CloudHub. By the time the runtime in PROD looks at the listener, the port isn't a `${...}` placeholder anymore. It's a literal `1234` that got frozen into the artifact during packaging. So chasing "how do I make ACB pass mule.env to CloudHub" sends you down the wrong path, because the runtime environment was never the problem. The **build** environment was. ## The actual cause: properties resolved at build time When ACB packages your app, it serializes a **parsed application model** as part of the artifact. This is an optimization, so the runtime doesn't have to re-parse every config XML on startup. The catch is *when* the placeholders in that model get resolved: during packaging, against whatever configuration is active **locally**. Locally, that's your local properties file, the one with `https.private.port=1234`. So the placeholder resolves to `1234` right there on your machine, and that literal value travels inside the artifact all the way to CloudHub. In PROD, the runtime uses the pre-parsed model, sees `1234` already sitting there, and never re-resolves anything, so the platform-assigned **8092** never gets a chance to apply. That also explains why leaving `https.private.port` undefined in PROD *looks* fine but doesn't help: the runtime isn't reading your PROD config for that value at all. It's reading a decision your local machine already made. ## Why Anypoint Studio gets it right Here's the part that makes it click. Studio sets this in its **default runtime arguments**, out of the box: ```text -Dmule.deployment.forceParseConfigXmls=true ``` That flag tells the runtime to **re-parse the configuration XMLs at deployment time** rather than trusting a pre-parsed model. With it on, `${https.private.port}` stays a real placeholder until it's resolved in the *target* environment, so in PROD it resolves against the PROD config, finds nothing, and correctly falls through to the platform's **8092**. ACB doesn't set that argument by default. Same code, same `mule.env`, but one tool defers property resolution to deploy time and the other doesn't. That single default is the entire difference. > [!TIP] > If you ever see a *local* value show up in a *remote* environment, suspect build-time resolution > before you suspect the remote config. The give-away is that the value can only have come from a file > that isn't deployed to that environment. ## The fix Add the same argument to ACB so it re-parses the config at deploy time: 1. Open ACB **Settings**. 2. Find the **`Mule > Runtime: Default Arguments`** section. 3. Add: ```text -Dmule.deployment.forceParseConfigXmls=true ``` 4. Redeploy. Now `${https.private.port}` is resolved in the target environment. In PROD it's undefined, so CloudHub assigns **8092**. Locally it's `1234`. Each environment decides its own port again, which was the whole point of using the placeholder. The nice thing about this fix is that it's **environment-agnostic**. It doesn't pin the app to PROD, so you can deploy to TEST or PROD without editing anything. ## The other fix that works (and its catch) While troubleshooting, another approach also worked: adding `mule.env=PROD` as a property in **`mule-artifact.json`** and deploying. That fixes it for a subtly different reason. It makes the PROD config active **at build time**, so when ACB resolves placeholders during packaging, `https.private.port` is already undefined and never gets frozen to `1234`. The value stays open and CloudHub assigns 8092. The catch is right there in how it works: it **hardcodes the environment** into the artifact. If you also deploy to TEST, you'd have to change `mule.env` in `mule-artifact.json` every time, which is exactly the manual juggling you were trying to avoid by using per-environment properties. > [!NOTE] > Both fixes converge on the same root cause, premature property resolution during ACB's build. One > defers resolution to deploy time (`forceParseConfigXmls`), the other makes the build resolve against > the right config (`mule.env` in `mule-artifact.json`). Prefer `forceParseConfigXmls` if you want a > single build that deploys anywhere. ## Quick recap - If a CloudHub app deployed from **ACB** uses a value that only exists in your **local** properties file, the placeholder was resolved at **build time**, not at deploy time. - ACB serializes a pre-parsed application model during packaging and resolves `${...}` placeholders against your local config, baking the local value into the artifact. - **Anypoint Studio** avoids this because it sets `-Dmule.deployment.forceParseConfigXmls=true` by default, forcing a fresh parse at deploy time. - Add that same argument to ACB under **`Mule > Runtime: Default Arguments`**. It's environment-agnostic, so one build deploys to any environment and each resolves its own properties. - Setting `mule.env` in `mule-artifact.json` also works, but it hardcodes the environment into the artifact, so you'd have to change it per deploy. I hope this was helpful. 💬 Prost! 🍻 --- ## 503s after adding the JWT Validation policy on an Omni Gateway MCP server? The "optional" JWKS fields aren't optional Source: https://prostdev.com/post/omni-gateway-jwt-validation-503-jwks-optional-fields | Published: Aug 2, 2026 | Category: Guides Another question in the [**MuleSoft Community Slack**](https://docs.google.com/forms/d/e/1FAIpQLScfuc3_R8sEl23xLkZBPIs6n7--HFhyuZewJiJKsKzQbnY9HQ/viewform) workspace, and this one got troubleshot by Slackbot for a good while before we finally landed on the real answer. The setup: someone secured an **MCP server on Omni Gateway** with the **JWT Validation** policy, and the moment the policy went on, every request came back **503**. The same policy and configuration was already working fine on their REST APIs, which made it extra confusing. Slackbot (which, fun fact, runs on Sonnet, same as MuleSoft Vibes) offered a very reasonable-looking checklist: JWKS fetch failures, Gatekeeper blocking on a bad policy, SSE handshake quirks, policy ordering for MCP. All plausible. None of them were it. The actual cause turned out to be much more boring, and it was hiding in a single log line. > [!NOTE] > This is a good reminder that an AI's tidy checklist is a starting point, not a diagnosis. The thing > that actually solved it was reading the logs. ## TL;DR If you configured the JWT Validation policy with **JWKS** as the key origin using the **new UI**, the two fields **JWKS Caching TTL (minutes)** and **JWKS Service connection timeout (milliseconds)** are marked *Optional* and left blank. The policy engine treats them as **required**, so it fails to apply and you get a 503 on every request. The fix: fill both in. Either set them explicitly: ```yaml jwksServiceTimeToLive: 60 # minutes jwksServiceConnectionTimeout: 10000 # milliseconds ``` Or configure the policy with the **traditional UI**, which populates those defaults for you automatically. ## The symptom: a 503 on every request, and no useful log The tell here is the combination: - Add the JWT Validation policy, and **every** request returns **503**, including the initial connection. - Remove the policy, and the server responds normally again, so the policy is clearly the cause. - There is **no error in the gateway logs** pointing at anything obvious. It just fails. That last part is what sends people down the wrong path. A 503 with no log feels like a networking or streaming problem, so it's tempting to start poking at SSE, the MCP handshake, or how your client sends the token. ## The red herrings Because MCP servers use **Server-Sent Events (SSE)**, and because JWT on a streaming connection genuinely does have quirks, the plausible theories pile up fast: - Maybe the Bearer token isn't present on the initial SSE handshake. - Maybe **policy ordering** is wrong and JWT is running before the MCP Support policy. - Maybe the JWKS endpoint isn't reachable from the gateway. These are all real things that *can* cause problems, which is exactly why they're such convincing dead ends. In this case they weren't the issue. The token was being sent, the ordering was fine, and the JWKS URL was reachable. The policy simply never finished configuring itself. ## The actual cause: one log line The breakthrough was finding this in the logs: ```text failed to configure extension: Invalid configuration: JWKS service URL, timeout and cache time to live must be set ``` There it is. The policy engine is saying it needs three things to stand up the JWKS extension: the service URL, a **timeout**, and a **cache time to live**. The URL was set. The other two were not, because the new UI left them blank and told everyone that was fine. Here's the trap. In the new policy-configuration UI, these two fields are labeled as optional: - **JWKS Caching TTL (minutes) (Optional)** - **JWKS Service connection timeout (milliseconds) (Optional)** So you fill in your JWKS URL, leave the "optional" fields empty, save, and everything *looks* correct. But the engine treats both as **required** whenever JWKS is the key origin. The UI happily lets you save a configuration the engine then refuses to apply, and because that failure happens while the policy is being brought up, you get a 503 before any request-level logging kicks in. > [!WARNING] > "Optional" in this UI means "the form won't stop you from saving it blank," not "the engine will > accept it blank." When JWKS is your key origin, both of these fields are effectively required. ## The fix Set both fields, even though the UI says you don't have to: ```yaml jwksServiceTimeToLive: 60 # minutes jwksServiceConnectionTimeout: 10000 # milliseconds ``` Save, and the policy configures cleanly. The 503s stop and the JWT Validation policy starts doing its actual job. If you'd rather not hand-set them, configure the policy through the **traditional UI** instead. It fills in sensible defaults for both the caching TTL and the connection timeout automatically, so the policy applies without you touching them. ## Why it worked on REST APIs but not the MCP server This is the part that makes the whole thing click. The reason the **same** policy config worked on the REST APIs and failed on the MCP server had nothing to do with MCP, SSE, or the token at all: - The REST APIs were configured **earlier**, through the **traditional UI**, so the JWKS timeout and cache-TTL defaults were populated for them. - The MCP server was a **newer** setup configured through the **new UI**, which left those fields blank. Same policy, same JWKS origin, two different UIs, and only one of them filled in the required defaults. That's the whole mystery. ## Quick recap - A JWT Validation policy that returns **503 on every request with no useful log**, right after you add it, is very likely failing to configure, not failing at request time. - Check the logs for `failed to configure extension: Invalid configuration: JWKS service URL, timeout and cache time to live must be set`. - The new UI marks **JWKS Caching TTL** and **JWKS Service connection timeout** as *Optional* but the engine requires them when JWKS is the key origin. Fill both in (for example `60` minutes and `10000` ms), or use the traditional UI, which sets the defaults for you. - If the same policy works on an older API but not a newer one, suspect **which UI each was configured in** before you suspect anything about MCP or SSE. Worth flagging to MuleSoft support too, since the new UI should really either populate these defaults or enforce the fields as required when JWKS is selected. I hope this was helpful. 💬 Prost! 🍻 --- ## No fx button on an ACB field? Set a dynamic value in the XML Source: https://prostdev.com/post/acb-field-no-fx-button-set-dynamic-value-with-xml | Published: Jul 24, 2026 | Category: Guides There was a question in the [MuleSoft Community Slack](https://docs.google.com/forms/d/e/1FAIpQLScfuc3_R8sEl23xLkZBPIs6n7--HFhyuZewJiJKsKzQbnY9HQ/viewform) workspace about a wall people hit in **Anypoint Code Builder (ACB)**: you want to pass a *dynamic* value into a component field, but that field has no **fx** button. The fx button (sometimes described as the "f" button) is what flips a field into expression mode so you can type DataWeave. No button, no obvious way to make the value dynamic from the UI. The good news: the restriction is only in the visual editor. The underlying Mule XML has no such limit, so you can set the value yourself. ## TL;DR Open the flow's XML and set the attribute inline with a DataWeave expression wrapped in `#[ ]`: ```xml ``` If the value comes from a **properties file** (`.yaml` or `.properties`), read it inside the expression with `Mule::p('...')` instead of the `${...}` placeholder syntax, and concatenate with `++`: ```xml ``` Replace `path` with the real attribute name for your connector field. That's the whole trick. The rest of this post explains why it works and walks through each piece. ## Why the fx button is sometimes missing Not every field in a connector is expression-enabled in the UI. Some are exposed only as plain text inputs, so the visual editor never renders the fx toggle next to them, even though the attribute behind that field will happily accept a runtime expression. It's a gap in the visual layer, not a limitation of the runtime. It can also come down to the **connector version**. Which fields expose the fx button is defined by the connector itself, so an **older version** may be missing it on a field that a newer release has since fixed. Before anything else, it's worth checking Exchange (or your `pom.xml`) for a newer version of the connector and updating it. That alone can bring the button back. Either way, the fix below works: stop fighting the UI and go one level down. ## Fix it in the XML with `#[ ]` Every Mule flow is just XML. In ACB you can open the flow's `.xml` file directly and edit the attribute by hand. Anywhere Mule accepts an expression, you write it between `#[` and `]`: ```xml ``` A few things to note: - `path` here is a stand-in. Use whatever the **actual attribute name** is for your connector's field (`path`, `url`, `query`, `fileName`, and so on). Hover the field in the UI or check the connector docs if you're not sure what it maps to. - Everything inside `#[ ]` is DataWeave, evaluated **at runtime** for each event, which is exactly what "dynamic" means here. You get the full language: variables (`vars.*`), the incoming message (`payload`, `attributes`), functions, and `++` for concatenation. - Setting the attribute in the XML is equivalent to what the fx button would have produced. You're not working around the runtime, you're just reaching the same setting a different way. Once you save, switch back to the visual view. The value shows up on the component, and the flow runs with your expression. > [!TIP] > If you're not sure which attribute a UI field writes to, set a placeholder value in the visual > editor first, then open the XML and look for it. Now you know the exact attribute name to make > dynamic. ## When the value lives in a properties file: use `Mule::p()` A very common case is that part of the dynamic value isn't computed from the payload but read from configuration, like a base URL that changes per environment. In the UI, connector fields can consume properties with the `${...}` placeholder syntax: ```xml ``` That works fine on its own. But the moment you need to **combine** that property with something else, `${...}` gets awkward, because it isn't a DataWeave expression, it's a static placeholder. You can't concatenate it with `++` or mix it with the payload. The clean way is to move the property *inside* the expression with the `Mule::p()` function. `Mule::p('property.name')` reads the value of a Mule application property (from your `.yaml` / `.properties` files, and also system properties or environment variables), so you can treat it like any other DataWeave value: ```xml ``` Now the base path comes from configuration and the order ID comes from the payload, joined at runtime. Same idea works with variables or literals: ```xml ``` > [!NOTE] > Rule of thumb: use `${property}` when a field takes a **plain, standalone** property value, and > switch to `#[Mule::p('property')]` the moment you need to **concatenate or transform** it. Both > read the same property; only the second one is a real DataWeave expression you can build on. ## Quick recap - No fx button on a field just means the visual editor won't let you type an expression there. The attribute still accepts one. - First check Exchange or your `pom.xml` for a newer connector version, since an older one can be the reason the button is missing. - Open the flow XML and set the attribute inline: `attr="#[ your expression ]"`. - For values from a properties file, read them with `Mule::p('name')` inside the `#[ ]` (instead of `${...}`) so you can concatenate and transform them. A small trick, but it turns "this field won't let me" into "this field does exactly what I need." I hope this was helpful. 💬 Prost! 🍻 --- ## How to Enable MuleSoft Vibes in VS Code Source: https://prostdev.com/post/enable-mulesoft-vibes-in-vs-code | Published: Jul 21, 2026 | Category: Tutorials You installed the Anypoint Extension Pack, opened MuleSoft Vibes... and it just won't turn on. The AI features are greyed out, and logging in to Anypoint Platform doesn't seem to change anything. The catch: **MuleSoft Vibes needs a Salesforce org with Agentforce enabled behind it**, and that connection isn't set up for you. This guide walks through the whole thing end to end, from installing the extension to seeing *"How can I help you today?"* in your editor. No prior Salesforce setup required; we'll spin up a free Developer Edition org along the way. ## What you'll need - **Visual Studio Code** installed ([download it here](https://code.visualstudio.com/download)). - An **Anypoint Platform account** with admin permissions ([sign up for free](https://anypoint.mulesoft.com/login/signup)). - A **Salesforce Developer org**, or a few minutes to [create a free one](https://developer.salesforce.com/signup) (we'll do that here). ## Step 1: Install the Anypoint Extension Pack Open VS Code, go to the **Extensions** view, and search for `anypoint extension pack`. Install the **Anypoint Extension Pack**, and make sure it's published by **Salesforce**. ![The Anypoint Extension Pack listing in the VS Code Extensions marketplace, published by Salesforce, mid-install.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-1.png) Once it finishes, you'll get two new icons in the VS Code activity bar. The first MuleSoft logo is **Anypoint Code Builder**. The second one is **MuleSoft Vibes**, the AI assistant we're here for. Click it, then click **Log In to Anypoint Platform** and follow the browser prompts to sign in. ![The MuleSoft Vibes welcome panel in VS Code showing a Log In to Anypoint Platform button.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-2.png) At this point Vibes still won't have its AI features. That's expected; the rest of the setup happens in Anypoint Platform and Salesforce. ## Step 2: Enable Agentforce in Anypoint Platform Log in to **Anypoint Platform** in your browser and open **Anypoint Code Builder**. From there, select **Go to Access Management**. ![The Anypoint Code Builder landing page inside Anypoint Platform.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-3.png) This will open **Access Management** in the **Salesforce** section. Find the **Accept** button to enable Agentforce for Anypoint Platform. ![The Access Management Salesforce section with the Enable Agentforce for Anypoint Platform panel and its Accept button.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-4.png) After you click **Accept**, this will open the terms and conditions. Check the **I accept** checkbox and press **Accept** again. ![The Accept Terms for Agentforce in Anypoint Platform dialog with the I accept the terms and conditions checkbox and an Accept button.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-5.png) To finish enabling everything, you'll need to connect a Salesforce org. If you don't have one, the next step creates a free one. ## Step 3: Create a free Salesforce Developer Edition org Head to [developer.salesforce.com](https://developer.salesforce.com) and open **Browse trials**. Pick **Get a Developer Edition**. It's free. Or you can go straight to the [sign-up link](https://www.salesforce.com/products/free-trial/developer/) directly. ![The Sign up for your Developer Edition form on Salesforce with name and email fields and a Sign me up button.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-6.png) Fill in your details and click **Sign me up**. You'll get an email to set your password; once that's done, log in to your new org. Salesforce will send a verification code to your email; enter it, and you're in your Developer Edition org. ## Step 4: Turn on Agentforce and Einstein in Salesforce Click the gear icon and open **Setup**. ![The Salesforce Developer Edition home with the gear-icon menu open and Setup available.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-7.png) In **Quick Find**, search for `agentforce`, open **Agentforce Agents**, and toggle **Agentforce** on. ![Salesforce Setup with agentforce in Quick Find and the Agentforce Agents page open.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-8.png) While you're here, search for `einstein`, open **Einstein Setup**, and make sure **Turn on Einstein** is switched on too, just in case. ![The Einstein Setup page in Salesforce with the Turn on Einstein toggle.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-9.png) ## Step 5: Connect Anypoint Platform to your Salesforce org Still in Salesforce Setup, search for `mulesoft` and open **MuleSoft Anypoint Platform Setup**. Click **Complete the Connection**. ![The MuleSoft Anypoint Platform Setup page in Salesforce with a Complete the Connection button.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-10.jpg) Copy your **Salesforce org tenant key** from the dialog. ![The Connection to MuleSoft Anypoint Platform dialog showing the Salesforce org tenant key to copy.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-11.jpg) Now switch back to **Anypoint Platform**, go to **Access Management** and the **Salesforce** section, and click **Add Salesforce Org**. Paste the tenant key you just copied. ![The Add Salesforce Org dialog in Anypoint Platform with a field for the Salesforce org tenant key.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-12.jpg) > [!TIP] > If you see the error `An error occurred: Organization is not yet onboarded to GDoT`, don't panic. > Just wait a couple of minutes, refresh, and try again. It usually clears on its own. Give the org a name you'll recognize (I called mine `sfdevaccount`) and click **Add**. ![Naming the newly added Salesforce org in Anypoint Platform.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-13.png) Anypoint Platform now gives you an **Anypoint Platform Org Key**. Copy it, go back to the Salesforce connection dialog, paste it into the **Anypoint Platform Organization ID** field, and click **Connect**. ![Pasting the Anypoint Platform Organization ID into the Salesforce connection dialog before clicking Connect.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-14.jpg) After that, you should see a success message saying the MuleSoft connection was created. ## Step 6: Flip the final Agentforce toggle One last switch: on the MuleSoft Anypoint Platform Setup page you land on after the connection succeeds, under **Managed in Salesforce**, turn on **Enable Agentforce in Anypoint Platform**. ![The Enable Agentforce in Anypoint Platform toggle under Managed in Salesforce in Salesforce Setup.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-15.jpg) That's the last piece. With this toggle on, Agentforce is enabled in Anypoint Platform and your Salesforce org is fully connected, so MuleSoft Vibes finally has everything it needs behind it. Time to head back to VS Code. ## Step 7: Use MuleSoft Vibes! Head back to VS Code and click **Try Again** on the MuleSoft Vibes panel (it might take a click or two while everything syncs). When it loads, you'll see **"How can I help you today?"** with suggested prompts. Vibes is live! 🎉 ![MuleSoft Vibes enabled in VS Code, showing How can I help you today with suggested prompts.](../../assets/blog/enable-mulesoft-vibes-in-vs-code-16.jpg) That's everything you need to enable MuleSoft Vibes from Anypoint Code Builder or VS Code. From here you can ask it about your APIs, list running applications, or have it help you deploy, all right inside your editor. --- ## 3 AIs Design API-Led Connectivity in MuleSoft: Claude vs CurieTech vs MuleSoft Vibes Source: https://prostdev.com/post/ai-showdown-3-ais-design-api-led-connectivity-in-mulesoft | Published: Jul 14, 2026 | Category: Opinion Three AIs. One MuleSoft architect brief. Zero human code. I handed **Claude**, **CurieTech AI**, and **MuleSoft Vibes** the exact same prompt — *design a 3-layer API-led connectivity network for a Customers + Orders domain* — and let each one build the whole thing on its own. Then I scored the results the way I'd review a real submission: contract-first design, business rules, error handling, tests, and version hygiene. This is Round 2 of **AI Showdown: MuleSoft Edition (2026) 🥊**. Round 1 pitted two AIs against DataWeave puzzles; this time the task is a full API-led topology, and there's a third contestant in the ring. Every solution is in the GitHub repo: 🔗 [github.com/alexandramartinez/ai-showdown-api-led](https://github.com/alexandramartinez/ai-showdown-api-led) ## TL;DR All three built the three layers (Experience → Process → System) and implemented **all 11 operations**. The differences were in the engineering discipline underneath: - **Claude** (Opus 4.8, MAX effort) was the **most disciplined build** — contract-first RAML, a parent reactor POM, and the *only* solution whose data survives a restart. Its weak spot: the oldest runtime of the three. - **CurieTech AI** was the **most complete and granular** — five apps split per entity, the most tests (41 MUnit cases), the richest error handling, and a dedicated status endpoint. Its weak spot: version drift across those five apps with no central POM. - **MuleSoft Vibes** (Sonnet) was a **fast, runnable demo** on the freshest connectors — but no API specs, no APIkit, no tests, no error handler, no docs, and writes don't persist. - **None** of the three added a security layer, and **none** used the true latest Mule runtime. ## The brief The prompt asked each AI to *think like a MuleSoft architect* and build an API-led connectivity example with all three layers for a Customers + Orders domain. The ground rules: - **Three layers**: Experience, Process, System. - **Mock and seed test data in the System layer**, as if it were wired to a real database (no actual DB connection yet). - **At least 3 APIs** (one per layer) — split further if it makes architectural sense, but don't build one giant do-everything API and don't repeat code. - **Use the latest** Mule, Maven, and DataWeave versions. And the 11 operations the network had to cover: 1. List all customers 2. Get one customer's details 3. Get a customer's orders 4. Get one order's details 5. Create a customer 6. Edit a customer 7. Delete a customer **only if they have no orders** 8. Create an order attached to a customer 9. Edit an existing order 10. **Cancel** an order (a status change, *not* a delete) 11. Delete an order Two of those are business-rule traps on purpose: #7 (block the delete when orders exist) and #10 (cancel ≠ delete). ## The contestants - 🟦 **Claude** — Opus 4.8, MAX effort - 🟨 **CurieTech AI** — model undisclosed - 🟩 **MuleSoft Vibes** — Sonnet (effort unknown) > [!NOTE] > These aren't necessarily comparable tiers of compute — the models and effort levels differ, and one vendor doesn't disclose its model. Read this as "what each tool produced from the same brief," not a controlled benchmark. ## The scorecard | Dimension | 🟦 Claude | 🟨 CurieTech | 🟩 MuleSoft Vibes | |---|---|---|---| | **Apps** | 4 (1 exp · 1 prc · 2 sys) | 5 (1 exp · 2 prc · 2 sys) | 4 (1 exp · 1 prc · 2 sys) | | **All 3 layers** | Yes | Yes | Yes | | **Parent/reactor POM** | Yes | No | No | | **Contract-first (RAML)** | Yes, all apps | Yes, all apps | No specs | | **APIkit** | Yes | Yes | Hand-rolled HTTP listeners | | **11 operations** | 11/11 functional | 11/11 functional | 11/11 coded, writes don't persist | | **Business rules (#7, #10)** | Both correct | Both correct | Both correct (on read-only data) | | **Security layer** | None | None | None | | **Error handling** | Global handlers, all apps | Global handlers + 502 gateway | None | | **Data persistence** | Across requests **and restarts** | In-memory (lost on restart) | Doesn't persist | | **MUnit tests** | 18 | 41 | 0 | | **Docs / rationale** | Plan + READMEs | READMEs per app | None | | **Version consistency** | Centralized | Drift across apps | Consistent | ## 🟦 Claude: the most disciplined build Claude produced four apps and, crucially, wired them together with a **parent reactor POM** so every app inherits the same dependency versions from one place. That's the single move that separates "four Maven projects in a folder" from "a governed project." It went **contract-first**: RAML 1.0 specs for every app, scaffolded with APIkit, with all 11 operations defined in the spec *and* implemented. Error handling is a global handler in all four apps covering 400/404/409, plus a 500 trait in the spec. The standout: Claude added the **File connector** to the System layer, so seeded data actually **persists across restarts**. It's the only one of the three where you can `POST` a customer, restart the app, `GET` it back, and it's still there. Both business-rule traps are handled — a 409 when you try to delete a customer with orders, and a status flip to `CANCELLED` for cancel-vs-delete. The catch: it shipped on **Mule 4.9.0**, the oldest runtime of the three, with the oldest plugin versions to match. Rock-solid, slightly behind. ## 🟨 CurieTech AI: the most complete and granular CurieTech went the furthest on topology — **five apps**, splitting both the Process and System layers per entity (customers vs. orders). That granularity paid off in coverage: **41 MUnit tests** (more than double Claude's), the richest error handling of the three (including proper **502 gateway** semantics between layers), and the cleanest take on the cancel rule — a dedicated `PATCH /orders/{id}/status` endpoint instead of overloading a `PUT`. It also nailed the seed-data detail that tests the business rules: 8 seed orders across 5 customers, with one customer (`CUST-005`) deliberately left order-free so you can actually exercise the "delete only if no orders" path. The catch: with five independent apps and **no parent POM**, versions drifted. The runtime ranged from 4.9.3 to 4.11.3 across apps, and APIkit/MUnit/HTTP-connector versions were similarly spread. Data lives in **ObjectStore**, so it persists across requests but is lost on restart. Feature-complete and well-tested — just missing the central version governance a parent POM gives you for free. ## 🟩 MuleSoft Vibes: a fast, runnable demo MuleSoft Vibes produced something that runs and uses the **freshest connectors** — it was the only contestant on the current HTTP connector (1.11.3) — with **consistent versions** across its four apps. But it skipped most of what makes API-led *API-led*: **no RAML specs, no APIkit** (hand-rolled HTTP listeners instead), **no MUnit tests, no global error handler, and no docs**. The operations are all coded and return the right status codes, but the data is read-only — a `GET` right after a `POST` returns a 404 for the record you just "created," because nothing actually persists. The business-rule logic is correct on paper; it just runs against data that never changes. Great for a quick "look, it works" demo — not something you'd hand to a team. ## The version reality check The brief said "use the latest." Nobody fully did — and it's a useful reminder that AIs anchor to whatever versions dominated their training data, not to what shipped last week. | Component | 🟦 Claude | 🟨 CurieTech | 🟩 Vibes | Latest GA | |---|---|---|---|---| | Mule runtime | 4.9.0 | 4.9.3–4.11.3 | 4.11.2 | 4.12.0 | | HTTP connector | 1.10.3 | 1.10.4–1.11.1 | 1.11.3 | 1.11.3 | | mule-maven-plugin | 4.3.0 | 4.5.3 | 4.7.0 | 4.10.0 | | APIkit | 1.11.3 | 1.10.4–1.11.12 | — | 1.12.1 | | MUnit | 3.4.0 | 3.6.3 | — | 3.7.1 | | Java | 17 | 17 | 17 | 17 | All three picked **real, released versions** and correctly used **Java 17** — no hallucinated version numbers, which is genuinely reassuring. But none reached **Mule 4.12.0**, and the plugin versions trailed further behind than the runtimes did. If you're using an AI to scaffold a project, pin your versions yourself. ## The gap all three left Every contestant skipped a **security layer** entirely — no client-ID enforcement, no policies, nothing gating the Experience API. The brief didn't explicitly ask for it, but "think like a MuleSoft architect" arguably should have surfaced it. It's the clearest example of AIs doing exactly what's asked and no more: none of them pushed back with "you probably also want auth here." ## So who wins? Depends on what you're optimizing for: - **Want a foundation to build on?** 🟦 **Claude.** The parent POM and real persistence mean you can hand this to a team and extend it without fighting version chaos. - **Want the most coverage and granularity out of the box?** 🟨 **CurieTech AI.** More tests, more error handling, a cleaner status endpoint — just add a parent POM and reconcile versions. - **Want a quick runnable demo to show the shape of API-led?** 🟩 **MuleSoft Vibes.** Fastest to something that runs, as long as you don't need it to remember anything. The meta-takeaway: all three are genuinely useful scaffolding tools, and all three need a human architect to close the same gaps — security, version pinning, and (for two of them) persistence. AI gets you to a working three-layer network fast. It doesn't yet get you to production. ## What's next? **AI Showdown: MuleSoft Edition (2026)** is an ongoing series comparing AI coding tools on the same MuleSoft and DataWeave challenges. Round 1 was DataWeave puzzles; this was API-led architecture. 🎯 What should the next round test — and which AI should join the ring? Drop a comment on YouTube or reply to this post. 🔔 Subscribe so you don't miss the next round: [youtube.com/prostdev](https://www.youtube.com/prostdev) --- ## Migrating a Blog From Wix to Astro: What Breaks Source: https://prostdev.com/post/migrating-a-blog-from-wix-to-astro | Published: Jul 9, 2026 | Category: Guides So I moved this whole blog off Wix. All 138 posts now live on a free static [Astro](https://astro.build/) site hosted on GitHub Pages, no monthly bill, and I own every byte of the content. If you're thinking about the same jump, here's the thing nobody warns you about: a Wix to Astro migration that finishes without a single error can still be quietly missing half your content. I learned that the expensive way. This post walks through what the export drops, why it drops it, and how I got it all back, so you can skip the part where you migrate the same catalog twice. ## TL;DR This is a field report, not a step-by-step tutorial. Wix silently drops a lot on export (code snippets, images, the video, galleries, buttons, comments, and it mangles tables and nested lists), all with no error to warn you. Below is what breaks, why, and how I recovered every piece without fabricating anything. If you just want the tools, the extractor and the full playbook are open-source: **[github.com/alexandramartinez/wix-to-astro](https://github.com/alexandramartinez/wix-to-astro)**. > [!NOTE] > I did this migration with Claude Code as my pair. Everything below is what actually happened on my posts, not a theory. The recovery scripts and the full playbook are in a separate repo, linked at the end. ## What "migrating from Wix to Astro" actually means A Wix to Astro migration is really two jobs, not one. The obvious job is moving your posts into a new format: Wix renders your blog, Astro builds static pages from Markdown (or MDX) files. The hidden job is *recovering the content the move leaves behind*, because Wix keeps a surprising amount of your post outside the HTML a migration script can see. Get the first job right and the site builds. Miss the second and you ship a blog that looks complete and is missing code snippets, images, and videos. That second job is where all my time went, so that's most of what this post is about. ## The first mistake: letting an AI summarize your posts My very first attempt fetched each post through an AI summarizer to turn the page into Markdown. It felt clever. It was a disaster. A summarizer paraphrases your prose and **silently drops every code block**. For a MuleSoft tutorial blog, that's the entire point of the post gone. I didn't catch it until I'd run the whole catalog through, and I had to redo the migration from scratch. Here's the fix, and it's the single most important lesson I can give you: don't summarize, *read the raw server HTML directly*. A small deterministic script that pulls the page and converts the real HTML tags to Markdown is lossless. It keeps your exact words, your links, your fenced code, all of it. Boring beats clever here. > [!WARNING] > A migration that runs without errors is not the same as a migration that's complete. Wix renders a big chunk of your post in the browser, so it was never in the HTML your script downloaded. No error fires because nothing knew that content was supposed to be there. ## What Wix renders client-side (so it was never in the HTML) This is the core insight, and it explains almost every gap I hit. Some of your post lives in the server HTML, and some of it Wix only assembles in the reader's browser. A migration script sees the first kind and is completely blind to the second. Here's what turned out to be client-side, and therefore missing after a clean export: - **Code snippets.** On Wix these were GitHub Gist embeds, little iframes that fill in on load. The code is never in the server HTML, so every snippet vanished. The tell on a live post is a line like "paste this in:" with nothing after it. - **The companion YouTube video.** Wix rendered it as a client-side player widget. Gone, leaving an orphaned "Prefer video? Watch it here" line pointing at nothing. - **Inline images.** These came through as a special Wix image block a first-pass regex never matched. Across my catalog that was 1080 images spread over 114 posts, all missing on the first real pass. - **Image galleries and buttons.** Multi-image carousels and styled call-to-action buttons (like a "Solve on the Playground" link) are the same story: client-rendered, so dropped. - **Reader comments.** Wix loads the comment thread in the browser too, so none of it was in the HTML. Some of those comments were genuinely useful (a reader posting a cleaner DataWeave solution, say), and I didn't want to lose them. None of these threw an error. They just weren't there, and you only notice when you actually read the migrated post. ## What migrated, but came out wrong A second, sneakier category: content that made it across but got mangled on the way, because the HTML-to-Markdown step didn't handle the shape Wix used. - **Tables flattened into loose paragraphs.** A two-column table became alternating one-line paragraphs, so the rows and columns were just gone as structure. The converter had no rule for ``, so each cell leaked out on its own. - **Nested lists collapsed.** Wix nests a sub-list *inside* its parent list item, and a naive match merged the parent with its first child and lost the indentation. - **Bold text glued to neighbors.** Wix likes to tuck the spacing inside the emphasis, so you'd get `**Pick this **option` with the bold swallowing the space, or one bold word split into two runs. - **Doubled quote marks on blockquotes.** The typography styling adds its own curly quotes, and the migrated text already had the author's quotes, so quotes showed up twice. These are all fixable, but you have to *know to look*. A build passes fine with a flattened table in it. ## Recovering it all without fabricating anything Here's the rule I held to the whole way through, and I'd push you to do the same: **never invent the missing content.** Every recovered code snippet came verbatim from the live rendered page or from my own source files, never from guessing what the code "probably" was. When a match was fuzzy (which dropped snippet belongs in which gap), I confirmed it by hand instead of letting a script auto-place it. That discipline is what makes the migrated post trustworthy. A tutorial with a plausible-but-wrong code block is worse than one with an honest "code was here" gap, because the reader can't tell it's wrong. So when in doubt, I left a visible hole and came back to it rather than filling it with a fabrication. The reader comments were the purest version of this rule. Unlike code or images, there's nothing to auto-recover, no script can bring them back, so I copied the handful of genuinely-useful ones straight off the live page, word for word, and preserved them as attributed notes under the post. The rest I let go. No paraphrasing, no inventing a helpful-sounding reply that nobody actually wrote. ## The upgrade the move unlocked The migration wasn't just damage control. Moving to my own Astro site let me fix things Wix never let me touch. The big one was accessibility. When I pulled the inline images across, 93% of them had no alt text at all, 977 empty out of 1047. That's a real problem for screen-reader users and for image search. So I went through and wrote genuine alt text by actually looking at each image, not keyword-stuffing, just describing what's in the shot. The other win is for AI. Every post on the new site has a plain-text `.md` version and feeds an `llms.txt`, so an answer engine can read the real content instead of scraping a JavaScript-heavy page. My old Wix blog was basically invisible to that whole world. Now the content is built to be quoted. ## Should you do it? First, to be fair to Wix: it's genuinely good at a lot of things. If you're running a shop, selling services or subscription plans, taking bookings, managing events, or you want a full CMS with a visual editor and no code, Wix handles all of that out of the box, and handles it well. The payment, scheduling, and store tooling alone can be worth the monthly fee. But if you're *just* hosting a website, mostly writing, and not accepting payments or running any of that commerce machinery, then you're paying every month for a lot of features you don't use, and getting a slower, less portable site than you could have for free. That's exactly the case where I'd make the move. So: if you're paying Wix monthly and your content is mostly writing, yes, I think it's worth it. You get free hosting, a faster site, full ownership, and content that both search engines and AI can actually read. Just go in with the right expectation: budget real time for the *recovery* phase, not only the export. The export is the easy 20%. Getting your code, images, videos, and tables back, verbatim and complete, is the other 80%, and it's the part that decides whether readers trust the result. ## What's in the repo I packaged everything I trusted into a separate open-source repo so you don't have to rediscover any of this. In short, it has: - **The extractor.** A small, dependency-free Node script that reads a Wix post's raw server HTML and converts it to clean Markdown verbatim, pulls the full-res original images, and reads the title, author, and dates from the page. There's a one-post command and a batch driver that migrates a whole blog. - **A QA linter.** After extraction it scans the output and prints exactly which posts are missing something (an empty image alt, a "paste this in:" line with no code after it, a flattened table, an MDX build trap), so you're not hunting blind for the gaps. - **The playbook.** A plain-English rundown of every trap above (the summarizer mistake, each thing Wix renders client-side, the tables and lists that come out wrong, the Astro/MDX build gotchas) with how to spot and fix each one. There's a version written for an AI pair too, so if you're migrating with Claude Code or Cursor it follows the same never-fabricate discipline. It does the honest, lossless part, tells you where the gaps are, and is explicit about what no script can recover, so you know exactly where the by-hand work starts. > [!BUTTON] > [Go to the GitHub repo](https://github.com/alexandramartinez/wix-to-astro) If you want the engineering underneath it (the tokenizer design, the Wix-specific paper cuts, and a zero-width regex that quietly pegged a CPU core), I wrote a deeper technical version on [dev.to](https://dev.to/devalexmartinez/migrating-a-wix-blog-to-markdown-the-server-html-trap-and-the-zero-width-regex-that-pegged-my-cpu-1lhc). If you hit a Wix quirk I didn't cover, I'd genuinely like to hear about it. Come find me on [YouTube](https://www.youtube.com/@prostdev) or wherever you found this post, and happy migrating. :-) --- ## Keep Your .claude Folder Private with a Git Submodule Source: https://prostdev.com/post/private-claude-folder-git-submodule | Published: Jul 5, 2026 | Category: Tutorials I build in the open. This site's whole [codebase](https://github.com/ProstDev/prostdev.github.io) is a public GitHub repo, and I like it that way. But my `.claude/` folder is a different story. It holds the skills, subagents, and `CLAUDE.md` instructions I've spent months tuning, plus notes I'd rather not hand to a scraper. I wanted the project public and that one folder private. The fix is a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). `.claude/` becomes its own private repo, wired into the public project. That folder holds all my [AI agent skills](/post/what-are-ai-agent-skills-and-how-to-use-them), subagents, and instructions, so keeping it private matters. There's one snag with `CLAUDE.md` that I'll walk you through, because Claude Code expects it at the repo root but I want the real file inside the private folder. A relative [symlink](https://en.wikipedia.org/wiki/Symbolic_link) solves it. This is exactly how prostdev.com is set up. Here's the whole thing, start to finish. > [!TIP] > You can follow these steps by hand, or you can hand them to your AI. Every step below is a plain command with the reasoning spelled out, so a coding agent like Claude Code can run the whole thing for you. If you go that route, paste the steps in, tell it your GitHub owner and the repo names you want, and ask it to stop and check with you before it creates the private repo or force-pushes anything. Those are the two steps you don't want an agent doing unsupervised. > [!NOTE] > This guide is written for macOS and Linux. The `CLAUDE.md` symlink step relies on POSIX symlinks, which Windows checkouts don't always materialize the same way. ## What you'll end up with Two repos instead of one: - **A public repo.** Your actual project (this site, in my case). Anyone can read it. - **A private repo.** Just the contents of `.claude/`: your skills, agents, docs, and the real `CLAUDE.md`. The private one is pulled into the public one as a submodule at the `.claude/` path. To anyone browsing your public repo, `.claude` looks like a pointer to a commit in another repo. They can't see what's inside unless you grant them access. ![GitHub's file view of the public prostdev.github.io repo. The file list shows a .claude entry, and the CLAUDE.md file is open, labeled "Symbolic link · 1 lines (1 loc) · 17 Bytes" with its entire contents being the single line ".claude/CLAUDE.md".](../../assets/blog/private-claude-folder-git-submodule-1.png) Two extra payoffs beyond privacy. First, you can **reuse that internal config across projects.** Point a second project's submodule at the same private repo and every skill comes along. Second, your `.claude/settings.json` (which can hold machine-specific paths) stays out of both repos entirely, which I'll cover near the end. ## Before you start You'll need the [GitHub CLI](https://cli.github.com/) (`gh`) authenticated, and a project that already has a `.claude/` folder with real content in it. Confirm `gh` is signed in: ```bash gh auth status ``` If that prints your account, you're good. Everything below runs from the root of your public project. > [!WARNING] > A few of these steps rewrite git history and delete the local `.claude/` folder before re-adding it. Commit or stash any uncommitted work first, and don't run this on a folder whose contents you haven't pushed somewhere safe. ## Step 1: Move the real CLAUDE.md inside .claude Claude Code reads `CLAUDE.md` from your project root. But I want the *real* file versioned in the private repo, so it needs to live inside `.claude/`. We'll reconcile that with a symlink in a later step. For now, move it in: ```bash git mv CLAUDE.md .claude/CLAUDE.md ``` If your `CLAUDE.md` wasn't tracked yet, use plain `mv` instead. If you already keep it at `.claude/CLAUDE.md`, skip this step. Either way, the source of truth is now `.claude/CLAUDE.md`. ## Step 2: Ignore your settings files in both repos `settings.json` and `settings.local.json` hold per-machine and personal config (local paths, permission choices). They shouldn't live in *either* repo. Add them to the `.gitignore` inside `.claude/`: ```gitignore title=".claude/.gitignore" # Claude Code: ignore personal/local settings (keep shared skills/agents/CLAUDE.md) settings.json settings.local.json .DS_Store ``` Then make sure your project root's `.gitignore` ignores them too, so they can't sneak in from the top: ```gitignore title=".gitignore" settings.json settings.local.json ``` If either file is *already* tracked, untrack it without deleting your copy on disk: ```bash git rm --cached .claude/settings.json .claude/settings.local.json ``` The `--cached` flag is the important part. It removes the file from git's index but leaves the actual file sitting in your folder, working as before. ## Step 3: Create the private repo from your .claude contents Now we give `.claude/` its own git history and push it to a new private repo. To avoid disturbing the main project's git state, do this in a throwaway copy of the folder, *outside* your project tree: ```bash cp -R .claude /tmp/claude-internal cd /tmp/claude-internal rm -rf .git # drop any leftover git metadata from the copy git init git add -A git commit -m "Initial import of .claude internal config" ``` Add a short `README.md` first if the folder doesn't have one, so the repo isn't bare. Then create the private repo and push, using `gh`: ```bash gh repo create your-name/project-internal --private --source=. --remote=origin --push ``` Swap `your-name/project-internal` for the owner and name you want. `--private` is what keeps it locked down, `--source=.` points it at this folder, and `--push` sends the commit up immediately. > [!IMPORTANT] > This is the one step that publishes something. Double-check the repo name and the `--private` flag before you run it. Everything before this was local. ## Step 4: Replace the local folder with a submodule Back in your public project, remove the plain `.claude/` folder from git and re-add it as a submodule pointing at the repo you just created: ```bash cd /path/to/your/public/project git rm -r --cached .claude rm -rf .claude git submodule add https://github.com/your-name/project-internal.git .claude ``` The `git rm -r --cached` stops tracking the folder's files directly. The `rm -rf` clears the local copy (it's safe now, everything is up in the private repo). And `git submodule add` clones the private repo into `.claude/` and records the link. You'll see a new `.gitmodules` file. It should look like this: ```ini title=".gitmodules" [submodule ".claude"] path = .claude url = https://github.com/your-name/project-internal.git ``` Confirm the submodule is healthy: ```bash git submodule status ``` A line beginning with a space and a commit hash, ending in `(heads/main)`, means the submodule is checked out and clean. ## Step 5: Recreate CLAUDE.md as a symlink Here's the trick that ties it together. The real file is now at `.claude/CLAUDE.md`, but Claude Code looks for `CLAUDE.md` at the project root. So we create a symlink at the root that points into the submodule: ```bash ln -s .claude/CLAUDE.md CLAUDE.md ``` Use the **relative** target `.claude/CLAUDE.md`, not an absolute path. A relative link resolves no matter where the repo is cloned; an absolute one would break on anyone else's machine. Verify it: ```bash ls -la CLAUDE.md ``` You should see something like `CLAUDE.md -> .claude/CLAUDE.md`, and the file will be tiny (about 17 bytes, the length of the target path). That's the tell that it's a symlink and not a copy. Reading it should show your real instructions: ```bash cat CLAUDE.md # prints the content of .claude/CLAUDE.md through the link ``` Now stage it. Git stores a symlink as a special entry (mode `120000`), not as a duplicate of the file's contents: ```bash git add CLAUDE.md ``` When you commit, `git ls-files -s CLAUDE.md` will confirm the `120000` mode. That's how you know git recorded the link itself rather than inlining the file. ## Step 6: Learn the two-repo commit rule From here on, your project is **two git repos**, and where a file lives decides which one you commit to. - Edit a **project file** (source code, a config, a blog post)? That's one ordinary commit in the public repo. - Edit an **internal file** (a skill, a subagent, a doc, or `CLAUDE.md`)? That's a **two-step commit**. The internal edit goes *inside the submodule first*, then the public repo records the new pointer: ```bash # 1. Commit inside the submodule git -C .claude add CLAUDE.md git -C .claude commit -m "Tweak instructions" git -C .claude push # 2. Bump the gitlink pointer in the public repo git add .claude git commit -m "Bump .claude submodule pointer" ``` The submodule commit has to exist and be pushed *before* the public repo points at it, or you'll reference a commit nobody else can fetch. When you run `git status` in the public repo and see `modified: .claude (new commits)`, that's not a stray change. It's git telling you the pointer is behind and you still owe step 2. For example, this is how the Source Control tab looks like in my local once I've modified files in the `.claude/` folder. First, I have to commit the files at the bottom (`.claude`), and then the files at the top (`prostdev`). Note how the `.claude` change at the top has an `S` for Submodule. ![VS Code Source Control panel showing two separate repositories stacked with their own commit boxes: the prostdev public repo on top with 3 changes, and the nested .claude submodule below with 2 changes including a modified CLAUDE.md](../../assets/blog/private-claude-folder-git-submodule-2.png) ## Step 7: Fix the clone gotcha There's one catch that trips up everyone, including future you on a new laptop. A normal `git clone` of your public repo brings down an **empty** `.claude/` folder. The submodule contents don't come automatically. Clone with the submodule in one shot: ```bash git clone --recurse-submodules https://github.com/your-name/your-project.git ``` Already cloned the plain way? Pull the submodule in after the fact: ```bash git submodule update --init ``` Because this bites everyone, say so in your public repo's README. A single line ("clone with `--recurse-submodules`") saves the next person a confused ten minutes wondering why Claude Code has no skills. ## Purge the private files from your git history Step 4's `git rm -r --cached .claude` stops git from tracking those files *going forward*. It does not remove them from the commits you already made. Git history is append-only by default, so every earlier commit still holds the full contents of `.claude/`, viewable by anyone who checks out an old commit. If the whole point is privacy, "untracked from now on" is not enough. The ideal is to never let the internal files reach the public repo's history in the first place: do this split before your first commit, or at least before you push. When that ship has already sailed, here's what to do based on where you are. > [!IMPORTANT] > Removing a file from the latest commit is not the same as removing it from history. Until you rewrite history, `git log` and any old checkout still expose it. ### If they never got committed Best case. If `.claude/` was gitignored or split out before your very first commit, there's nothing in history to clean. Confirm it: ```bash git log --all --oneline -- .claude ``` No output means the path never appeared in a commit, and you're done. ### If they're only local and not pushed yet This is the easy one, because rewriting history is safe when nobody else has your commits. The bulletproof move is to collapse everything into one fresh commit of the current, correct state (symlink plus submodule, zero private files). You lose your old commit *messages*, but you get a guaranteed-clean history: ```bash git checkout --orphan clean-history # a new branch with no parent git add -A git commit -m "Public project (private .claude lives in a submodule)" git branch -D main # drop the old history git branch -m clean-history main # promote the clean branch to main ``` Because you started from an orphan branch, the new `main` has exactly one commit and no trace of the old `.claude/` files. You never pushed, so there's nothing to force and nobody to coordinate with. ### If they were already pushed Now the files are public, and two things are true. First, you have to rewrite history *and* force-push it. Second, and this matters more: **treat anything sensitive in those files as already leaked.** Clones, forks, and GitHub's own caches may still hold the old commits, so rotate any API keys, tokens, or secrets that were exposed. A history rewrite is not a substitute for rotation. To rewrite while keeping the rest of your history, use [git-filter-repo](https://github.com/newren/git-filter-repo) (install it once with `brew install git-filter-repo`): ```bash git filter-repo --path .claude --invert-paths ``` That strips every `.claude/` entry from every commit. It also rewrites all your commit hashes and drops the `origin` remote as a safety check, so re-add your remote, re-wire the submodule (repeat Step 4), then force-push: ```bash git remote add origin https://github.com/your-name/your-project.git git push --force origin main ``` Don't want to keep the old history at all? The orphan-branch reset from the previous section works here too. Just add `git push --force origin main` at the end, and warn anyone else who cloned the repo, because their history no longer matches yours. > [!CAUTION] > A force-push rewrites shared history. If other people have cloned or forked the repo, coordinate with them first. And whatever you do, rotate any secret that was ever pushed. Removing it from history does not un-leak it. ## Wrapping up That's the whole pattern: a private submodule at `.claude/`, a relative `CLAUDE.md` symlink so Claude Code still finds its instructions, settings files ignored in both repos, a two-step commit for anything internal, and a clean history with none of those private files ever committed to the public repo. The upfront cost is one afternoon of git wrangling. What you get back is a project you can develop fully in the open without publishing the config you've invested real time in, and an internal toolkit you can drop into the next project by pointing a new submodule at the same private repo. If you set this up and Claude Code suddenly can't see your skills, the first thing to check is whether the submodule actually populated. Run `git submodule status` and `ls .claude/` before you debug anything fancier. --- ## AI Agent Skills: What They Are and How to Use Them Source: https://prostdev.com/post/what-are-ai-agent-skills-and-how-to-use-them | Published: Jul 4, 2026 | Category: Guides AI agent skills are pretty much pieces of information that you save in Markdown files. You can write whatever you want in them, or you can also let your own AI write them for you (or for itself) whenever some new information is needed. There are some standards that are good to follow on how to structure these files to keep your AI's context lower, and hopefully spend fewer tokens! But we'll talk about that in a bit. ## First of all, what really are skills? So as I said before, they're just Markdown files inside folders with specific names on them. :-) The AI would either know which one to call, or you can also manually call it at any point in your session. > [!NOTE] > I'll be talking about my experience with Claude Code because that's what I normally use on my day to day, but skills are reusable in any other AI agent. So what is inside a `SKILL.md` file? It has two parts: 1. A short YAML header (called *frontmatter*) that contains a `name` and a `description`. 2. The instructions themselves, written in Markdown. The name has to match exactly the name of the folder that contains the skill for the AI to run it. The description has to be as useful and descriptive as possible so the AI knows when exactly to run this skill. Try to always answer what this skill does and when to use it, so the AI also knows when it's useful to use or not. This first part is important because when the AI is looking for skills, it either reads only the folder names to filter out the ones that don't fit, or goes and reads the frontmatter of all the available skills to figure out which one is needed and which one is not for the specific job it's trying to accomplish. Let's see an example. This is a simple skill: ```markdown title="SKILL.md" --- name: commit-message description: >- Write a git commit message from the staged diff. Use when the user asks to commit, write a commit message, or summarize staged changes. --- # Write a commit message Read the staged diff, then write a commit message that: - Uses a concise, imperative subject line under 50 characters. - Leaves a blank line, then a body explaining *why*, not *what*. - References an issue number when the branch name contains one. ``` As you can see, the description says what the skill does (writes a git commit message from the staged diff) and when to use it (use when the user asks to commit, write a commit message, or summarize staged changes). After that is the actual skill's instructions to the AI on how you want it to write the commit message exactly. ### Why the name and description matter so much Imagine if the skill was called `commit` and the description was just `write a commit`. Remember that the AI first reads only the frontmatter and not the full body of the skill, otherwise it would defeat the purpose of getting your context under control. So if it just sees "write a commit", it's not really sure what to do with that! Are you asking it to commit your changes? Are you asking it to write some code and commit? Do you want to review the files before the commit? It doesn't even specify that you just want it to write the *message* of the commit. Of course this skill would perform poorly on any AI. This is why the name and the description are so important. Otherwise your AI will have to end up reading all your skills (or none!) to try to figure out what you want, which will cost you more turns, more context, and more tokens. ## Setting up your first skill in Claude Code Claude Code looks for skills in two places, and it loads both automatically when it starts up. The only real difference between them is *scope*: | Where | Path | Use it for | | --- | --- | --- | | **Personal** | `~/.claude/skills//SKILL.md` | Skills you want in *every* project. Your own habits and preferences. | | **Project** | `.claude/skills//SKILL.md` | Skills that only make sense in *this* codebase. Its conventions, its build steps. | Here's how you make your first personal skill: 1. **Make the folder.** The folder name becomes the skill's name, so keep it short and use kebab-case (words joined with dashes): ```bash mkdir -p ~/.claude/skills/commit-message ``` 2. **Create the `SKILL.md` inside it.** Open `~/.claude/skills/commit-message/SKILL.md` and paste in the example from above, or your own. 3. **Keep the `name` and the folder in sync.** Remember, the `name` in the frontmatter has to match the folder name exactly, or the AI won't run it. 4. **Write a description that says *when*.** "Use when the user asks to commit…" is way more useful to the AI than "a commit tool". The clearer the trigger, the more reliably the skill fires at the right moment (and only the right moment). 5. **Restart Claude Code**, or start a new session. Skills get indexed when it starts up, so a brand-new skill gets picked up on the next launch. And that's the whole setup. There's no build step, no config file, no registering anything. The folder *is* the installation. ### Calling a skill Once a skill is installed, there are two ways to trigger it: - **Let the AI decide.** Just describe your task like you normally would. If it matches the skill's description, the AI loads it and follows it on its own. - **Call it yourself by name.** Type `/commit-message` (a slash plus the skill's name) when you want to make sure that specific skill runs. ## Using skills in other AIs Like I mentioned at the top, a `SKILL.md` is just Markdown with a small header, so there's nothing Claude-Code-specific about the file itself. That gives you two ways to use it anywhere else: - **If the tool supports the format**, point it at your skills folder and it loads them the same way: reading the names and descriptions first, and the full body only when it needs it. The [Claude Agent SDK](https://docs.claude.com/en/api/agent-sdk/overview) and the Claude desktop and web apps all understand skills, and more agent tools keep adopting the same `SKILL.md` convention. - **If the tool doesn't support skills natively**, you can still use the content. Just paste the body of the `SKILL.md` right into the conversation as your instructions. You lose the automatic matching, but you keep the workflow you wrote, which is most of the value anyway. So writing your workflow as a skill is never wasted effort, even for an AI that has never heard the word "skill". It's still a really good, reusable prompt. ## Browsing the ProstDev Skills page The [Skills page](/skills) here on ProstDev is a shared library of the real skills I use with Claude Code. Each card is one skill. Here's how to get the most out of it: - **Read the description first**, exactly like the AI does. It tells you *when* the skill is meant to fire, which is the fastest way to know if it fits your workflow. - **Filter by tag** with the chips at the top (`Claude Code`, `MDX`, `AEO`, and so on) to narrow the grid down to a topic. - **Open a skill** to read my own explanation of it, plus the complete `SKILL.md` in a code block you can copy. Grab it, drop it into `~/.claude/skills//`, and tweak the parts that are specific to your own setup. Every skill page is also machine-readable. Add `.md` to the end of any skill URL, like [`/skill/save-session.md`](/skill/save-session.md), and you get the raw Markdown with none of the page around it. That plain version is exactly what you'd hand to an AI. ## Point your AI at llms.txt and let it pick the skill Here's where it gets really handy. You don't have to read the whole library yourself and decide which skill an AI needs. You can let the AI do that, the same way it already picks between its *own* skills. The site publishes an [llms.txt](/llms.txt) file. This is a small, growing web standard (you can read about it at [llmstxt.org](https://llmstxt.org)): a single Markdown index of a site, full of links, written specifically for AI crawlers. ProstDev's `llms.txt` has a section that looks like this: ```markdown title="llms.txt (excerpt)" ## Claude Code skills Reusable Claude Code / AI-agent skills Alex Martinez uses and shares. Each has a machine-readable Markdown version (append `.md` to its URL). - [save-session](https://www.prostdev.com/skill/save-session): End-of-session routine… - [add-faqs](https://www.prostdev.com/skill/add-faqs): Add an FAQ section to a post… - [blog-seo-standards](https://www.prostdev.com/skill/blog-seo-standards): SEO + AEO standards… ``` Notice this is the exact same shape as a skill's own frontmatter: a name and a one-line description of when to use it. That's not a coincidence, it's the same idea one level up. So the flow is: 1. **Give a browsing-capable AI the URL** `https://www.prostdev.com/llms.txt` along with your task. For example, "I'm about to wrap up a coding session, is there a skill for that?" 2. **The AI scans the list of names and descriptions** and matches your task to a skill, just like Claude Code matches your request to a local skill. 3. **It grabs the full instructions** by adding `.md` to that skill's URL (`/skill/save-session.md`) and follows them. You've basically handed the AI a menu it can read and order from by itself. And because it's reading descriptions first, it only pulls the full text of the one skill it actually needs, which (you guessed it) keeps your context lower and your tokens down. ## A few tips to get more out of skills - **Spend your effort on the description.** It's both the trigger and the summary the AI scans. Phrase it as "use this when…". A vague description means the skill never fires when you want it to. - **Keep each skill small and focused.** One job per skill. A skill that tries to do everything matches nothing cleanly. - **Let skills call other skills.** You can chain small skills into a bigger workflow instead of writing one giant file. One "orchestrator" skill can call several focused ones in order, so a big job stays easy to maintain. - **Check them into your repo.** A project skill in `.claude/skills/` travels with the code, so your whole team and every future session get the same behavior for free. - **Refresh them when they go stale.** Skills drift as your tools change. Updating one is usually a quick edit to the body, not a rewrite. > [!TIP] > Not sure your `SKILL.md` follows the conventions? I keep a skill for exactly that: [`claude-md-and-folder-standards`](/skill/claude-md-and-folder-standards). Point Claude Code at it and ask it to check a skill you just wrote, and it'll verify the name and description, the folder layout, the size, and the rest against the standards, so your config stays lean. ## Wrapping up Skills turn your best prompts into permanent, reusable pieces of information your AI can reach for on its own. A skill is just a folder with a `SKILL.md` inside: a `name`, a "use this when…" `description`, and some Markdown instructions. Drop it in `~/.claude/skills/` for Claude Code, paste it into any other AI that doesn't read the format natively, or publish it so other people can copy it. And because a skill's frontmatter and an `llms.txt` index both describe *when* to use each skill in one line, you can hand an AI an `llms.txt` and let it browse and pick the right skill by itself. Start with one skill for the thing you re-type the most. Once you see the AI reach for it without you asking, you'll want to write ten more! Need a head start? Browse the real skills I use with Claude Code, copy the `SKILL.md`, and make it yours. > [!BUTTON] > [Explore the ProstDev Skills library](/skills) --- ## Claude Code vs CurieTech AI: Which Writes Better DataWeave? Source: https://prostdev.com/post/claude-code-vs-curietech-dataweave | Published: May 28, 2026 | Category: Opinion Two AIs. Four DataWeave puzzles. One question: **Which one writes better MuleSoft code?** I put [Claude Code](https://claude.com/claude-code) (running in MAX effort mode) head-to-head against [CurieTech AI](https://platform.curietech.ai) to solve [Advent of Code 2025](https://adventofcode.com/2025) — Days 1 and 2, both parts. Same input. Same expected output. Side-by-side execution times in the [DataWeave Playground](https://dataweave.mulesoft.com/learn/playground). This is also the first video in a new series I'm starting: **AI Showdown: MuleSoft Edition (2026) 🥊** ## TL;DR Both AIs got the **correct answer on all 4 puzzles**. Execution times in the Playground were within milliseconds of each other. The real differences showed up in how they got there — code readability, type usage, error handling, and whether the AI actually runs the code before handing it back. ## The setup - **Claude Code** — VS Code extension, MAX effort mode, Sonnet 4.5 - **CurieTech AI** — tested two ways: - Day 1: via the CurieTech MCP server inside Claude Code - Day 2: directly through the chat at [platform.curietech.ai](https://platform.curietech.ai/) - **Challenges**: Advent of Code 2025, Days 1 and 2 (both parts) - **Benchmarking**: dw::util::Timer::time() in the DataWeave Playground ## What surprised me ### ✅ Execution times were ridiculously close We're talking millisecond-level differences — even running in the browser-based Playground. Sometimes Claude edged out, sometimes CurieTech did. Honestly? Coin-flip territory. ### ✅ CurieTech actually runs the code This is the big one. CurieTech executes the DataWeave it generates before returning the answer. Claude doesn't — it just produces code and hopes for the best. The result: I caught a runtime error in Claude's Day 2 solution that only surfaced when I ran it locally. Had to feed the error back into Claude for a second pass. CurieTech never had that problem. ### ✅ CurieTech writes typed function signatures Look at the difference: Claude (untyped): ``` fun claude(input) = ... ``` CurieTech (typed): ``` fun curietech(input: Array): Number = ... ``` When you're reading code cold (or coming back to it months later), those types make a huge difference. ### ✅ CurieTech adds performance notes On Day 2 Part 1, CurieTech proactively warned me: > "Ranges like this are tiny (~7 IDs), so brute forcing works here. If you ever face huge ranges, you'd want to generate candidates, loop over half-lengths, and concatenate instead of scanning every number." That kind of contextual insight is genuinely useful — especially for folks newer to DataWeave. ### ✅ Claude Code wins on workflow If you live in VS Code or your terminal, Claude Code's integration is hard to beat. File context, multi-step tasks, the whole ecosystem. CurieTech's MCP server brings some of that into Claude Code, but the native experience is still tighter. ### ❌ Claude can ship broken code As mentioned — Day 2 had a runtime error that wasted time. Not a dealbreaker, but worth knowing. ### ❌ CurieTech needed a nudge on Day 1 Part 2 The first attempt got the wrong answer. After I gave it more context about the input format, it produced a verified solution. Slight friction, but the recovery was clean. ## So who wins? Honestly? There's no clear winner. Each has real strengths: - **Pick Claude Code** if you value workflow integration, live in VS Code, and don't mind catching the occasional runtime error. - **Pick CurieTech AI** if you value verified-running code, typed function signatures, and inline performance notes. For someone newer to DataWeave or AI coding tools, **CurieTech is probably the gentler entry point** — the chat UI is approachable and the verified-solution guarantee is reassuring. For experienced devs already deep in the Claude Code ecosystem, Claude Code with the CurieTech MCP server is a great hybrid — you get the workflow you want plus the verification when you need it. ## All the code Every solution, every benchmark script, every Playground link is in the GitHub repo: 🔗 [github.com/alexandramartinez/adventofcode-2025](http://github.com/alexandramartinez/adventofcode-2025) ## What's next? This is the first video in **AI Showdown: MuleSoft Edition (2026)** — an ongoing series comparing AI coding tools on the same MuleSoft and DataWeave challenges. As I work through more Advent of Code days (and test more AIs), I'll keep posting the results here. 🎯 What AI tool should I compare next? Drop a comment on the YouTube video or reply to this post. 🔔 Subscribe so you don't miss the next round: [youtube.com/prostdev](https://www.youtube.com/prostdev) *Looking for the previous series? Check out "Adventures in MuleSoft + AI" on the channel for last year's explorations.* --- ## How I Use ChatGPT to Brainstorm Newsletter Topics Source: https://prostdev.com/post/how-i-use-chatgpt-to-brainstorm-newsletter-topics | Published: Sep 25, 2025 | Category: Opinion Every week, I need a juicy, relevant topic for [AI Picked, Alex Approved](https://www.linkedin.com/newsletters/%F0%9F%94%AE-ai-picked-alex-approved-7348737640038350849/). Lately, I’ve been experimenting with ChatGPT as my brainstorming partner — and it’s been surprisingly effective. Here’s how the process went for my latest edition: 1️⃣ I fed it my previous newsletter topics so it knew what I’d already covered. 2️⃣ I asked for fresh angles, from AI in IDEs to Docs vs AI debates. 3️⃣ We drilled down into real-world developer opinions, research, and stats. 4️⃣ I used it to structure a fast-read newsletter with **fast facts, real quotes, and a deeper dive** for readers who want more. The result: a newsletter draft ready to go, packed with credible sources and actual dev stories — all while keeping my voice front and center. [Read the full edition here](https://www.linkedin.com/pulse/ide-cage-match-can-you-trust-ai-your-editor-alex-martinez--pdvic/). Pro tip: AI is amazing for **idea generation and research**, but the human touch is still essential for context, nuance, and personality. Note that this conversation took around 30 minutes but it did give me two different posts based on the same conversation (the newsletter and THIS post you're reading right now!). ![ChatGPT prompt feeding it the AI Picked, Alex Approved newsletter, then suggesting AI, MuleSoft, and content topics](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-1.png) ![ChatGPT given the list of past newsletter topics, then proposing fresh angles grouped by theme](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-2.png) ![ChatGPT comparing "Docs vs AI" and "IDE Cage Match" topics with a pros and cons breakdown](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-3.png) ![ChatGPT weighing pros and cons of the IDE Cage Match topic and recommending it as the next edition](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-4.png) ![ChatGPT's first newsletter draft comparing expectations vs reality of AI coding copilots](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-5.png) ![ChatGPT listing what devs praise and what breaks with AI IDE tools, with cited sources](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-6.png) ![ChatGPT suggesting hands-on experiments to try and real developer quotes to include](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-7.png) ![ChatGPT framing the editorial angle and offering a TL;DR shareable soundbite for the piece](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-8.png) ![ChatGPT condensing the draft into Fast Facts, What This Means, and My Take sections](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-9.png) ![ChatGPT's "IDE Cage Match" draft with numbered fast facts, takeaways, and a deeper dive section](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-10.png) ![ChatGPT offering real developer testimonials and a "Deeper Dive + Real-Dev Stories" section](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-11.png) ![ChatGPT quoting developer complaints about AI copilots plus supporting research and stats](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-12.png) ![ChatGPT's polished Deeper Dive split into "The Good" and "The Bad" with quoted dev reviews](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-13.png) ![ChatGPT's "The Ugly" and "The Takeaway" sections with a Cursor refactor quote and security stats](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-14.png) ![ChatGPT's near-final newsletter draft combining fast facts, takeaways, and quoted dev reviews](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-15.png) ![Continuation of the draft ending with The Ugly and The Takeaway, then offering a cleaned-up version](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-16.png) ![Cleaned-up "IDE Cage Match" draft with formatted fast facts, what-this-means, and deeper-dive sections](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-17.png) ![End of the draft, then ChatGPT offering to write a short LinkedIn caption to promote the edition](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-18.png) ![ChatGPT drafting a LinkedIn caption, then a separate post about the brainstorming process itself](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-19.png) ![ChatGPT producing a LinkedIn post titled "How I Use ChatGPT to Write About AI in IDEs Without Losing My Voice"](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-20.png) ![ChatGPT's final post styled after this article, recapping the ChatGPT brainstorming workflow](../../assets/blog/how-i-use-chatgpt-to-brainstorm-newsletter-topics-21.png) --- ## How I Use ChatGPT for Writing Without Losing My Voice Source: https://prostdev.com/post/how-i-use-chatgpt-to-create-articles | Published: Sep 18, 2025 | Category: Opinion A lot of people either misuse AI tools or don’t use them at all—even though these tools could make their work so much more efficient. From what I’ve seen, the main reason is simple: most people aren’t sure how to get the best results from them. For me, AI has become part of my writing process. I run a LinkedIn newsletter, and every edition is built with the help of tools like ChatGPT. But here’s the key—I don’t just copy-paste whatever the AI spits out. I guide it with my own tone, I curate the content, and I make sure it’s factually accurate. In other words, the AI helps me move faster, but I stay in control of the message. Take my [latest newsletter on CurieTech AI](https://www.linkedin.com/pulse/curietech-ais-glow-up-chat-first-create-later-alex-martinez--l8ikc) as an example. I used ChatGPT to brainstorm and structure the post, then shaped it into something that sounded like me. That way, I saved time without losing my voice or letting the AI drift off track. Note that this whole conversation took place in less than 10 minutes. ![ChatGPT prompt asking for a newsletter article about the new CurieTech AI chat-first experience.](../../assets/blog/how-i-use-chatgpt-to-create-articles-1.png) ![ChatGPT's first draft titled "A Smarter Way to Work with AI" with a "Why This Matters" bullet list.](../../assets/blog/how-i-use-chatgpt-to-create-articles-2.png) ![Continuation of the first draft with "Example in Action" and "My Take" sections.](../../assets/blog/how-i-use-chatgpt-to-create-articles-3.png) ![Follow-up prompt asking ChatGPT to spell CurieTech correctly and make the tone funnier and more playful.](../../assets/blog/how-i-use-chatgpt-to-create-articles-4.png) ![Playful, emoji-heavy rewrite titled "CurieTech's Glow-Up: Chat First, Create Later" with a benefits list.](../../assets/blog/how-i-use-chatgpt-to-create-articles-5.png) ![Continuation of the playful rewrite with emoji-laden "Example" and "My Take" sections.](../../assets/blog/how-i-use-chatgpt-to-create-articles-6.png) ![Prompt telling ChatGPT to make the tone "a bit less gay", and its reply offering a more balanced version.](../../assets/blog/how-i-use-chatgpt-to-create-articles-7.png) ![Toned-down rewrite of "CurieTech's Glow-Up: Chat First, Create Later" with a "Why this matters" list.](../../assets/blog/how-i-use-chatgpt-to-create-articles-8.png) ![Continuation of the balanced rewrite with "Example" and "My Take" sections.](../../assets/blog/how-i-use-chatgpt-to-create-articles-9.png) ![Prompt clarifying that Curie is an AI dev tool for MuleSoft projects usable inside Anypoint Studio.](../../assets/blog/how-i-use-chatgpt-to-create-articles-10.png) ![Rewrite grounded in MuleSoft context: "CurieTech's New AI Experience: Chat Before You Build".](../../assets/blog/how-i-use-chatgpt-to-create-articles-11.png) ![Continuation with "Why This Is a Big Deal" and an "Example in Action" using a DataWeave-tests scenario.](../../assets/blog/how-i-use-chatgpt-to-create-articles-12.png) ![The "My Take" section calling Curie a collaborative development partner, plus a follow-up caption offer.](../../assets/blog/how-i-use-chatgpt-to-create-articles-13.png) ![Prompt to keep the "Chat First, Create Later" title, and the polished article reopening with it.](../../assets/blog/how-i-use-chatgpt-to-create-articles-14.png) ![Polished article's "How It Works" and "Why This Matters" sections referencing API type and DataWeave tests.](../../assets/blog/how-i-use-chatgpt-to-create-articles-15.png) ![Polished article's "Example in Action" and "My Take" sections framing Curie as a development partner.](../../assets/blog/how-i-use-chatgpt-to-create-articles-16.png) ![Prompt simply saying "linkedin post", with ChatGPT offering a short scroll-stopping promo post.](../../assets/blog/how-i-use-chatgpt-to-create-articles-17.png) ![Short LinkedIn promo caption "New in CurieTech: Chat First, Create Later" with a newsletter-link placeholder.](../../assets/blog/how-i-use-chatgpt-to-create-articles-18.png) At the end of the day, AI is just a tool—it won’t replace your voice or your perspective. But if you know how to guide it, you can save time, avoid burnout, and still produce content that feels authentic and valuable. The trick isn’t to let AI do the work for you, but to let it work with you. --- ## Implement MySQL with Docker in Anypoint Code Builder (ACB) Source: https://prostdev.com/post/connect-mysql-docker-to-mulesoft-acb-vs-code | Published: Jul 28, 2025 | Category: Tutorials In this tutorial, you’ll learn how to connect a **MySQL database running in Docker** to a MuleSoft project in **Anypoint Code Builder (ACB)** — all inside **VS Code**, without ever leaving your IDE. We’ll use three powerful VS Code extensions: - [Container Tools](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-containers) → run MySQL with a single click - [Database Client](https://marketplace.visualstudio.com/items?itemName=cweijan.vscode-database-client2) → create tables and test queries - [Postcode](https://marketplace.visualstudio.com/items?itemName=rohinivsenthil.postcode) → test your API endpoints By the end, you’ll have a Mule flow that returns live MySQL data in JSON. **Follow up with the GitHub repo**: [https://github.com/alexandramartinez/todo-api-impl](https://github.com/alexandramartinez/todo-api-impl) ## Prerequisites Make sure you already have: - An **Anypoint Code Builder project scaffolded** - **VS Code** with the following extensions installed: - Anypoint Extension Pack - Container Tools - Database Client - Postcode - **Docker Desktop** running ## Step 1: Launch MySQL with Container Tools - In your project folder, create a file called `docker-compose.yml` and paste this in: - Click on the **Run All Services** button inside this file (you have to have the Containers extension for VS Code) ```yaml version: '3' services: db: image: mysql:latest environment: MYSQL_ROOT_PASSWORD: admin MYSQL_DATABASE: mysqldb MYSQL_USER: user MYSQL_PASSWORD: password ports: - 3306:3306 volumes: - db_data:/var/lib/mysql volumes: db_data: ``` ✅ MySQL is now running inside a Docker container. ## Step 2: Connect Using the Database Client Extension - Open the **Database** panel in VS Code. - Click **+ New Connection** and select **MySQL**. - Enter: - **Host**: 127.0.0.1 - **Port**: 3306 - **User**: user - **Password**: password - **Database**: mysqldb - Save and test the connection — you should see it succeed. - Run the following SQL to create your table: ```sql CREATE TABLE tasks( id int NOT NULL PRIMARY KEY AUTO_INCREMENT, title VARCHAR(255) NOT NULL, description VARCHAR(255), dueDate VARCHAR(255), completed BOOLEAN NOT NULL ); ``` ✅ The table is ready — no data inserted yet. ## Step 3: Implement the get-all-tasks Sub-Flow in ACB Now that the database is running and the table is created, let’s implement the flow that will fetch tasks from MySQL. In your scaffolded Mule project, add the following sub-flow to your implementation XML file (for example: `get-all-tasks.xml`): ```xml ``` ### How This Sub-Flow Works - **Start Logger -** Marks the beginning of the flow. This is helpful for debugging and tracking execution in your logs. - **Build the SQL Query -** Uses a DataWeave script (`dw/get-all-tasks-request.dwl`) to generate the SQL query dynamically. This keeps SQL logic separate from the flow. - **Database Select -** Executes the SQL query against MySQL using the connection defined in Mysql_Database_Config. - #[payload] tells Mule to use the SQL string produced by the previous step. - **Transform DB Results to JSON -** Applies a second DataWeave script (`dw/get-all-tasks-response.dwl`) to shape the raw DB results into a clean JSON response for the API. - **End Logger -** Marks the successful completion of the sub-flow. Here are the two DataWeave script files: ```dataweave %dw 2.0 output application/java var completed = attributes.queryParams.completed var dueDate = attributes.queryParams.dueDate var whereClause = if (isEmpty(completed) and isEmpty(dueDate)) "" else "WHERE " ++ ( [ ("completed = $completed") if (!isEmpty(completed)), ("dueDate = $dueDate") if (!isEmpty(dueDate)) ] joinBy " AND " ) --- "SELECT * FROM tasks $whereClause" ``` ```dataweave %dw 2.0 output application/json --- payload map { id: $.id as String, title: $.title, description: $.description, dueDate: $.dueDate, completed: $.completed } ``` ## Step 4: Run and Test with Postcode - Start your Mule app from ACB. - Open the Postcode extension in VS Code. - Create a new request: ``` GET http://localhost:8081/api/tasks ``` - Click Send ✅ You’ll see an empty JSON array ([]) until you insert rows later — but your flow is working and ready for data. ## Final Thoughts You’ve successfully implemented a **MySQL-backed flow in Anypoint Code Builder** using Docker and VS Code extensions. - **Container Tools** handles your database with one click - **Database Client** makes schema management simple - **Postcode** keeps testing inside VS Code ⚡ Next step: Add a **POST /tasks** flow to insert new tasks into your database! ## 🔔 Want More Tutorials? Subscribe to [ProstDev on YouTube](https://www.youtube.com/@prostdev?sub_confirmation=1) for weekly videos on: - Anypoint Code Builder (ACB) - API design and implementation - MUnit testing - MuleSoft best practices ## 🤖 FAQ ### 1. Why should I use Docker for my database in MuleSoft projects? Docker makes it easy to spin up a clean, isolated database environment without needing a manual install. It ensures consistency across development machines and avoids “works on my machine” issues. ### 2. Do I need to know Docker commands to follow this approach? Not necessarily. With the right VS Code extensions (like Container Tools), you can manage containers with simple clicks instead of terminal commands. ### 3. What’s the benefit of using a Database Client extension instead of an external tool like MySQL Workbench? Using a Database Client inside VS Code keeps everything in one place. You can manage schemas, run queries, and debug directly alongside your MuleSoft code. ### 4. How do DataWeave scripts improve database flows? DataWeave lets you dynamically generate SQL queries and format responses. This makes your flows more flexible, avoids hardcoding, and ensures cleaner separation between logic and data. ### 5. Is it safe to build SQL queries dynamically with DataWeave? Yes, as long as you validate and sanitize inputs properly. For production, always guard against SQL injection by using parameters or prepared statements when possible. ### 6. What are common use cases for integrating MuleSoft flows with a database? Typical use cases include CRUD operations for APIs, syncing data between systems, generating reports, or validating user information during API requests. ### 7. How do I test MuleSoft endpoints without leaving VS Code? You can use REST client extensions like Postcode. They let you send HTTP requests and view responses without switching to external tools. ### 8. Why should I separate request and response transformations into different DataWeave files? This improves maintainability. If your SQL logic or JSON response format changes, you only need to update the relevant .dwl file without touching the main flow. ### 9. What’s the advantage of using sub-flows in MuleSoft projects? Sub-flows let you break down complex logic into reusable, modular components. This improves readability, makes testing easier, and allows you to reuse common logic across multiple flows without duplicating code. ### 10. Can I reuse this approach with other databases besides MySQL? Absolutely. You can adapt the same setup for PostgreSQL, Oracle, SQL Server, or any database supported by MuleSoft’s Database connector — just update the Docker image and connector configuration. --- ## Scaffold an API Spec in ACB + VS Code Tips for Beginners Source: https://prostdev.com/post/scaffold-api-spec-anypoint-code-builder-vscode-tips | Published: Jul 21, 2025 | Category: Tutorials If you’ve ever used **Anypoint Studio** to implement an API from a spec, you already know how easy it is to generate flows from a RAML or OpenAPI file. But what about **Anypoint Code Builder (ACB)**? In this guide, I’ll walk you through what happens when you scaffold an API spec in ACB — from clicking **“Implement an API”** to running your Mule app locally. Along the way, you’ll learn essential **VS Code tips**, how to handle `.gitignore`, where your **flows** and **configs** live, and how to switch between **UI and XML views**. > [!TIP] > Whether you're a beginner with MuleSoft or just transitioning to ACB, this walkthrough gives you the confidence to explore and edit scaffolded projects like a pro. ## 🛠️ Step 1: Scaffold the API Spec from Exchange Start with an API spec you’ve published to **Anypoint Exchange** (we used a simple To-Do API in OpenAPI 3.0 format). - Open Anypoint Code Builder inside **VS Code** - Click **Implement an API** - Select your API asset from Exchange - Name your project (e.g., todo-api-impl) - Choose a local folder and hit **Create Project** ACB will automatically generate: - Mule flows for each endpoint - HTTP Listener configuration - Error handling logic (400, 404, 405, 406, 415, etc.) - `mule-artifact.json`, `pom.xml`, .gitignore, and other setup files ## 🧭 Step 2: Explore the Scaffolded Project Here’s what you get out of the box: ### ✅ Auto-generated Flows Each method in your API (GET, POST, etc.) becomes its own flow, wired to the right path. ### 🔀 Error Handling ACB scaffolds global error flows like: - 400 Bad Request - 404 Resource Not Found - 405 Method Not Allowed - and more… You don’t need to define these in the API spec — ACB just adds them. ### 🧩 Configuration Files - `mule-artifact.json`: runtime + project metadata - `pom.xml`: dependencies (e.g., APIkit, HTTP connector) - `log4j2.xml`, embedded sample folders, etc. ## 🔧 Step 3: Use Mule Project Properties UI Right-click your project > **Open Mule Project Properties**. This UI helps you: - View/set Mule runtime and Java version - See which **connectors** were added - Check compatibility for versions (runtime vs connector vs Java) - Apply missing runtimes automatically ## 🗂️ Step 4: Clean Up with .gitignore To keep your repo clean: - Add `src/test/resources/embedded*/` to .gitignore - Ignore .vscode or other machine-specific files - VS Code makes this easy from the Source Control tab — right-click any file > Add to .gitignore ## 🌀 Step 5: Initialize Git & Use Source Control - Open the **Source Control** tab in VS Code - Click “Initialize Repository” - You’ll see all tracked changes - You can commit everything after staging, or one file at a time ## 🧑‍💻 Step 6: Canvas vs XML — Switch Views Easily Want XML to open by default? - Open a .xml file (e.g., a flow) - Click the three-dot icon at the top right > **Configure Editors** - Edit the file association so XML is default instead of Canvas - Switch it back anytime to mulesoft.projectFile.canvas if you prefer UI Bonus: You can even open **both views at once** with "Open in Canvas and Code Editor." ## 🚀 Step 7: Run & Test Your App in VS Code Click the **Run** button or use the command palette to launch the Mule app locally. Test your endpoints with: - Postman - Postcode (VS Code extension for REST requests) - cURL or any HTTP client You should see 200 OK responses — even though the implementation is empty. That’s scaffolding done right. ## 🎯 Recap: What You Learned ✅ How to scaffold an API spec into a running Mule project ✅ What files and flows ACB generates for you ✅ How to explore and customize your project like a pro ✅ Useful VS Code tips for Git, views, testing, and organization ## 🔔 Want More Tutorials? Subscribe to [ProstDev on YouTube](https://www.youtube.com/@prostdev?sub_confirmation=1) for weekly videos on: - Anypoint Code Builder (ACB) - API design and implementation - MUnit testing - MuleSoft best practices ## 🤖 FAQ – Anypoint Code Builder (ACB) API Scaffolding & VS Code ### ❓ What does "scaffold an API" mean in Anypoint Code Builder? Scaffolding an API in ACB means automatically generating Mule flows, error handlers, and configuration files (like `mule-artifact.json` and `pom.xml`) based on an API spec, such as OpenAPI or RAML. This allows developers to instantly start implementing logic without manually creating flows from scratch. ### ❓ How do I implement an API spec from Anypoint Exchange in ACB? You can implement an API by: Opening Anypoint Code Builder in VS Code Clicking **“Implement an API”** Selecting your API spec from Exchange Letting ACB generate flows and configs for you ### ❓ What files are auto-generated when scaffolding an API in ACB? ACB generates: Mule flows for each method Error handlers `mule-artifact.json` `pom.xml` Logging configs (`log4j2.xml`) HTTP listener configurations Sample test files (under `src/main/test/embedded*`) ### ❓ Can I edit the Mule flows in both XML and visual UI (Canvas) in ACB? Yes. ACB supports both views: **Canvas UI** for visual editing **XML editor** for precise config control You can open both views side-by-side or configure VS Code to always default to one view using the **Configure Editor** setting. ### ❓ How do I configure .gitignore in a new ACB project? ACB automatically creates a .gitignore file for you when creating a new project, but you still need to add some files or folders like `src/test/resources/embedded*` or .vscode/ ### ❓ What is mule-artifact.json used for? The `mule-artifact.json` file defines project-level metadata like: The required Mule runtime version Minimum Mule version ACB generates this automatically during scaffolding. ### ❓ How do I run my Mule app locally in ACB? Use the Run/Debug tab in VS Code or use the command palette: Select **Run/Debug Mule Application Locally** Check the terminal for server logs and endpoint availability Use Postman or Postcode (VS Code REST client) to test your API --- ## Design a REST API in Anypoint Code Builder (OpenAPI + VS Code) Source: https://prostdev.com/post/how-to-design-a-rest-api-in-anypoint-code-builder-openapi-vs-code-exchange | Published: Jul 14, 2025 | Category: Tutorials Whether you’re new to MuleSoft or finally starting to explore Anypoint Code Builder (ACB), this post will walk you through designing a full REST API specification using OpenAPI 3.0 — directly inside VS Code, with no XML or flows yet. By the end, you’ll be able to: - Define a reusable OpenAPI spec for a To-Do API - Mock and test the endpoints inside ACB - Publish the spec to Anypoint Exchange so your team can reuse or scaffold flows from it later Let’s build the foundation for your next MuleSoft project — the right way. ## 🛠️ What You’ll Need - New to ACB? Read our [Getting Started guide](https://www.prostdev.com/post/getting-started-with-anypoint-code-builder-in-vs-code-beginner-guide) first. - [Anypoint Platform](https://anypoint.mulesoft.com/) account to publish to Exchange. ## 🧪 What We’re Building We’re designing a To-Do Task Management API that supports full CRUD operations for /tasks: - GET /tasks to list tasks with optional filters - POST /tasks to create a task - GET /tasks/{taskId}, PUT, and DELETE for individual task management - A schema for Task and a separate input model TaskInput > If you want to follow with the YAML file, you can check it out [here](https://gist.github.com/alexandramartinez/c7680921369627323b763905b9b28490). ## 🧑‍💻 Step-by-Step API Design in ACB ### 1. Create a New OpenAPI Spec From the ACB Welcome screen, choose "Design an API", then select: - Type: API Specification - Language: OpenAPI 3.0 (YAML) - Project name: todo-task-api You don’t need to log into Anypoint Platform just yet. ### 2. Fill Out the API Info Block Update the metadata like this: ```yaml info: version: 1.0.0 title: To-Do Task Management API description: A simple API for managing to-do tasks. Supports full CRUD operations for tasks in a single-user context. ``` This info will be displayed in Exchange later — so make it clear and helpful. ### 3. Define Your Endpoints Start with the /tasks resource and support: - GET with optional query parameters (completed, dueDate) - POST with a JSON request body ```yaml paths: /tasks: get: summary: List all tasks description: Retrieve a list of all tasks, with optional filtering by completion status and due date. parameters: - in: query name: completed schema: type: boolean description: Filter tasks by completion status (true or false) - in: query name: dueDate schema: type: string format: date description: Filter tasks by due date (YYYY-MM-DD) responses: "200": description: A list of tasks content: application/json: schema: type: array items: $ref: "#/components/schemas/Task" post: summary: Create a new task description: Add a new task to the to-do list. requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/TaskInput" responses: "201": description: Task created successfully content: application/json: schema: $ref: "#/components/schemas/Task" ``` Then add the /tasks/{taskId} resource with GET, PUT, and DELETE methods. Make sure to handle the URI parameter. ```yaml /tasks/{taskId}: parameters: - in: path name: taskId required: true schema: type: string get: summary: Get a task by ID description: Retrieve a specific task by its unique ID. responses: "200": description: Task details content: application/json: schema: $ref: "#/components/schemas/Task" "404": description: Task not found put: summary: Update a task description: Update an existing task by its ID. requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/TaskInput" responses: "200": description: Task updated successfully content: application/json: schema: $ref: "#/components/schemas/Task" "404": description: Task not found delete: summary: Delete a task description: Remove a task from the list by its ID. responses: "204": description: Task deleted successfully "404": description: Task not found ``` ### 4. Add Reusable Schemas In the components section, define two schemas: #### Task ```yaml components: schemas: Task: type: object properties: id: type: string example: "1" title: type: string example: Buy groceries description: type: string example: Milk, bread, eggs dueDate: type: string format: date example: 2026-01-01 completed: type: boolean default: false example: false required: - id - title - completed ``` #### TaskInput ```yaml TaskInput: type: object properties: title: type: string example: Buy groceries description: type: string example: Milk, bread, eggs dueDate: type: string format: date example: 2026-01-01 completed: type: boolean default: false example: false required: - title - completed ``` ### 5. Test Using the API Console > Your API Specification should look like [this](https://gist.github.com/alexandramartinez/c7680921369627323b763905b9b28490). Click API Console to try out each endpoint — no flows required. [You](http://required.You)’ll see: - Query parameter validation - 404 handling - Schema errors if you send the wrong type - Simulated responses using your example data This is perfect for quick iteration or mocking out APIs. ### 6. Publish to Anypoint Exchange When you're ready to share: - Log into Anypoint Platform via ACB - Press Cmd/Ctrl + Shift + P → "Publish API Project to Exchange" - Choose a business group and version, then publish! You can now discover and reuse this API inside your org — or scaffold flows from it in the next step. ## 📦 What’s Next? In the next tutorial, we’ll take this exact API spec and use it to auto-generate the MuleSoft flows. You’ll implement the endpoints inside VS Code without writing a line of XML manually. 👉 Subscribe to the [YouTube playlist](https://www.youtube.com/playlist?list=PLb61lESgk6hh9SszYvsAnLYiV9-WCpqPP) for new ACB videos every week. ## **💡 Pro Tip** **If you ever get stuck, I offer** **Anypoint Code Builder Office Hours** **to help you troubleshoot and get unblocked.** **👉 Book a session** [here](https://calendly.com/mule-alexmartinez/acb-troubleshooting)**.** --- ## Getting Started with Anypoint Code Builder in VS Code (Beginner Guide) Source: https://prostdev.com/post/getting-started-with-anypoint-code-builder-in-vs-code-beginner-guide | Published: Jul 7, 2025 | Category: Tutorials If you're new to MuleSoft—or making the switch from Anypoint Studio—you might be wondering: **how do I even begin with Anypoint Code Builder (ACB)?** This post will walk you through the **entire setup process of ACB locally in Visual Studio Code**, from installation to building and testing your very first “Hello World” Mule application. Whether you're a developer curious about the future of MuleSoft or a seasoned Studio user adjusting to ACB, this is your go-to beginner’s guide. ## 🛠️ Step 1: Install the Required Tools Before using ACB, make sure you have the following installed: - **Git**: [Download Git here](https://git-scm.com/downloads) - **Visual Studio Code**: [Download VS Code](https://code.visualstudio.com/) ## 🔌 Step 2: Install the Anypoint Extension Pack ✅ **Important:** Only install the Anypoint Extension Pack *after* Git and VS Code are installed and up to date. If needed, uninstall and reinstall to get the latest version of VS Code. - Open VS Code. - Go to the Extensions tab. - Search for Anypoint Extension Pack. - **Install the full pack** (not individual extensions). - Restart VS Code completely after installation. 🔁 If you've already installed extensions one by one, uninstall them, restart VS Code, and then install the full pack. ## 🧠 Step 3: Get Comfortable with VS Code Open the **Command Palette**: - Mac: Shift + Command + P - Windows: Shift + Ctrl + P You’ll use this for everything—from changing themes to running Mule apps. Try searching for: - Preferences: Color Theme Customize your theme or install fun ones like *Halloween* or *MuleSoft Community Theme*. ## 🐴 Step 4: Understand the ACB Layout Unlike Anypoint Studio, ACB runs inside VS Code. Here's how it differs: - **Start a Project** from the MuleSoft tab (not from the “File > New Project” menu). - You can’t import JAR files or projects directly. - Always **open the root folder** (with your src, `pom.xml`, etc.) to work on projects. ### Quick Actions in ACB: - Design an API - Implement an API - Develop an Integration (what we’ll use) ## 🧩 Step 5: Install Additional VS Code Extensions (Optional) VS Code has a powerful extension ecosystem. For example: - Docker - Kubernetes - Postman - MySQL Viewer - GitHub Actions These aren’t required for ACB, but they supercharge your workflow. ## ⚙️ Step 6: Configure ACB Settings In the MuleSoft tab: - Click the gear ⚙️ icon to adjust **autocompletion**, **default runtimes**, and **VM arguments**. - You *don’t need to install Java manually*—ACB does it for you. - In future versions, ACB may also install Git automatically. ## 🔐 Step 7: Log In to Anypoint Platform Click your profile icon or use the “Sign in with Anypoint Platform” option in the ACB panel. (You can skip this for now if you’re just testing locally.) ## 👋 Step 8: Build a "Hello World" Mule App Let’s create our first integration: - Go to the **MuleSoft tab** → “Develop an Integration.” - Name it: helloworld - Choose a folder and select the latest runtime and Java version. - Once the project is created: - Use the UI builder to add an **HTTP Listener**. - Set the path to /hello. - Add a **Transform Message** component and output "Hello World". - Add a **Logger** to print the payload. You now have a fully working Mule app that returns “Hello World” at [http://localhost:8081/hello](http://localhost:8081/hello). ## 🐞 Step 9: Run and Debug Your App - Use the **Run and Debug** panel in VS Code. - Click “Start Debugging.” - Test with a tool like Postcode (or Postman). - Hit [localhost:8081/hello](http://localhost:8081/hello) to get your response. ### Debugging Tips: - Add breakpoints directly in the XML or use the UI. - Watch variables like payload or check flow attributes. - Use the Command Palette to format XML files with Format Document. ## 🧪 Final Thoughts You’ve now: ✅ Installed ACB ✅ Created a Hello World Mule App ✅ Tested and Debugged it—all inside VS Code! This is just the beginning. In the next tutorial, we’ll build a real REST API using API specifications and flow logic. ## 💡 Pro Tip If you ever get stuck, I offer **Anypoint Code Builder Office Hours** to help you troubleshoot and get unblocked. 👉 Book a session [here](https://calendly.com/mule-alexmartinez/acb-troubleshooting). --- ## Building a YouTube MCP Server with MuleSoft and CurieTech AI in 10 Minutes Source: https://prostdev.com/post/building-a-youtube-mcp-server-with-mulesoft-and-curietech-ai-in-10-minutes | Published: Jun 19, 2025 | Category: Tutorials Curious how to integrate the YouTube Data API v3 into a fully functioning MCP (Model Context Protocol) server in record time? With the help of CurieTech AI and MuleSoft, I did it in just 10 minutes, without ever having created an MCP server before. In this article, I’ll walk you through what I did step-by-step — including how I used CurieTech to generate the entire integration, converted it into an MCP server, and ran it using SuperGateway and the MCP Inspector. I also show how you can connect this to Claude or other apps and use natural language to run YouTube searches via the MCP server. 🔗 **GitHub Repo:** [https://github.com/alexandramartinez/youtube-mcp-server-mule](https://github.com/alexandramartinez/youtube-mcp-server-mule) ## 🧠 What Is MCP? MCP stands for **Model Context Protocol**, a new protocol designed for agents, AI tools, and other automated systems to interact with structured tools in a flexible and self-describing way. MCP tools define schemas, input parameters, and responses — and are ideal for agent-based use cases. > In other words... It's an API for your agents. ## 🛠️ What We’re Building A MuleSoft app that: - Connects to the **YouTube Data API v3** - Accepts search queries and API keys as parameters - Returns YouTube search results - Runs as an **MCP server** with full schema and parameter validation ## 🔑 Step 1: Get Your YouTube API Key To use the YouTube API, you'll need an API key from Google Cloud: - Go to [Google Cloud Console](https://console.cloud.google.com/). - Create a project. - Enable the **YouTube Data API v3**. - Under **APIs & Services > Credentials**, create a new **API Key**. - Under **API Restrictions**, select **YouTube Data API v3**. - Save your changes and copy the key — you'll use it in your requests. ## 🧪 Step 2: Generate the API with CurieTech AI Inside [CurieTech AI](https://curietech.ai): - Go to the **Integration Generator**. - Set the language to **Java 17**, build tool to **Maven 3.9**, and **Mule 4.9**. - Use a prompt like: "Make an API that uses YouTube Data API v3. Include all query parameters, especially key, and make it as simple as possible." - Paste the list of parameters from the [official docs](https://developers.google.com/youtube/v3/docs/search/list) or paste the URL. - CurieTech generates the full integration — copy the code. ## 💻 Step 3: Run the App in MuleSoft - Create a new Mule project in **Anypoint Code Builder** or **Studio**. - Paste the code from CurieTech. - Run the project (default port is 8081). - Use Postman to test: ``` GET http://localhost:8081/search?key=YOUR_API_KEY&q=mulesoft ``` ## 🤖 Step 4: Convert to an MCP Server - Push your Mule project to GitHub (or use VS Code's repo initializer). - In CurieTech, open the **Coding Agent** and select your repo. - Prompt: "Turn this into an MCP server using the beta MCP connector." - CurieTech updates the code, modifies the `pom.xml`, adds a global MCP config, and replaces the HTTP listener with an MCP listener. - Approve the PR in GitHub and merge. ## 🚦 Step 5: Run SuperGateway - Open a terminal window. - Run the following command (ensure the port matches your Mule app's HTTP Listener): ```bash npx -y supergateway --sse http://localhost:8081/sse --ssePath /sse --messagePath /message ``` ## 🔍 Step 6: Open MCP Inspector - Open a new terminal window and run: ```bash npx @modelcontextprotocol/inspector ``` - Copy the tokenized URL (e.g., http://localhost:6274/?MCP_PROXY_AUTH_TOKEN=...) into your browser. - Select SSE as the transport type - Paste the following URL: ``` http://0.0.0.0:8081/sse ``` - Click Connect - Select the youtube-search tool - Enter your API Key and query - Click on Run Tool ## 🤝 Step 7: Add the Server to Your App (e.g., Claude) Add the following to your Claude or other AI agent config: ```json { "mcpServers": { "youtube": { "command": "npx", "args": [ "-y", "supergateway", "--sse", "http://0.0.0.0:8081/sse", "--ssePath", "/sse", "--messagePath", "/message" ] } } } ``` Restart your app, and you're ready to use it in natural language prompts! ## 🧠 Example Prompts You Can Use Inside Claude (or any compatible tool): - “Search for the ProstDev YouTube channel and return the last 4 videos.” - “Generate a LinkedIn post that promotes the first video. Include the title, description, and video link.” - “Find trending content about ‘API design’ using YouTube search.” ## 📦 Repo 🔗 **GitHub Repo:** [https://github.com/alexandramartinez/youtube-mcp-server-mule](https://github.com/alexandramartinez/youtube-mcp-server-mule) ## 🧭 Final Thoughts I had never worked with MCP before — and I built this whole server in 10 minutes thanks to **CurieTech AI**. The entire flow, from integration to deployment and testing, was smoother than I imagined. If you learn by reading code like I do, this is one of the fastest ways to master new patterns like MCP. Whether you're building for fun, research, or AI-driven platforms, give this a try and let me know what you build next! --- ## Why is my Array not filtering correctly? It may be an Array instead Source: https://prostdev.com/post/array-string-key-dataweave | Published: Feb 5, 2025 | Category: Tutorials ## The Problem Ok so here's what was happening to me and I didn't know why. I was doing a bunch of transformations and at one point I'd add some keys and/or values into an array. So, some of the arrays were of type `Array` (because no key was added here) while some of them where `Array` I didn't think much about it at first, since I would -incorrectly- see all the arrays in the JSON output were of type `Array`. Like this: > [!NOTE] > The item in line 4 was coerced into a Key for demonstration purposes. ```dataweave title="transform.dwl" %dw 2.0 output application/json var arr1 = [ "key" as Key, "string" ] var arr2 = [ "key", "string" ] --- [arr1, arr2] ``` ```json title="output.json" [ [ "key", "string" ], [ "key", "string" ] ] ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Farray-string-key%2Fscript1) So imagine my surprise when I tried to use a [distinctBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-distinctby) function with the above arrays and I'd still get the same two arrays in the output. ```dataweave title="transform.dwl" %dw 2.0 output application/json var arr1 = [ "key" as Key, "string" ] var arr2 = [ "key", "string" ] --- [arr1, arr2] distinctBy $ ``` ```json title="output.json" [ [ "key", "string" ], [ "key", "string" ] ] ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Farray-string-key%2Fscript2) I started losing my mind trying to troubleshoot the problem a million different ways because I didn't notice the items in the arrays were of different types! It was until one of those million times finally led me to see the array types were actually different 🤦 ## Possible Solutions ### Solution 1 There are different solutions for this problem. One would be to make sure any new items are of type String and not Key. This completely removes the issue since now you'd be comparing two arrays of type `Array`, which correctly returns only one array in the output. ```dataweave title="transform.dwl" %dw 2.0 output application/json var arr1 = [ "key" as Key, "string" ] var arr2 = [ "key", "string" ] --- [arr1, arr2] map ((item) -> item map ($ as String) ) distinctBy $ ``` ```json title="output.json" [ [ "key", "string" ] ] ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Farray-string-key%2Fscript3) ### Solution 2 If you can't change the type of the items or it would take too many iterations to do, another thing you can do is to transform the items inside the [distinctBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-distinctby) function itself. Like this: ```dataweave title="transform.dwl" %dw 2.0 output application/json var arr1 = [ "key" as Key, "string" ] var arr2 = [ "key", "string" ] --- [arr1, arr2] distinctBy ($ joinBy ",") ``` ```json title="output.json" [ [ "key", "string" ] ] ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Farray-string-key%2Fscript4) Since we're using the [joinBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-joinby) function, we're transforming the values of arr1 and arr2 into concatenated strings and THEN comparing them. Both arr1 and arr2 would be "key,string", which makes these values the same. I hope this post brings you more DataWeave knowledge or helps you fix some code :D Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## How to connect your local Ollama AI using the MuleSoft AI Chain (MAC) project and Anypoint Code Builder (ACB) Source: https://prostdev.com/post/mac-project-ollama-ai | Published: Nov 20, 2024 | Category: Tutorials This post guides you through building an AI-powered API using Ollama and MuleSoft. You'll start by installing Ollama locally, designing and implementing the API, and testing it in your development environment. With ngrok, you'll expose the API publicly, and finally, deploy it to CloudHub for production use. For guidance, you can follow along using [this GitHub repository](https://github.com/ProstDev/mac-ollama-proj). ## 1 - Install Ollama locally - Go to [ollama.com](http://ollama.com) and follow the prompts to install in your local computer - Make a note of which version you started running (like llama3 or llama3.2) - To verify ollama is running, you should either be able to interact with it in the terminal or you should be able to run "ollama list" or "ollama ps" - We will verify the local installation in MuleSoft before deploying to CloudHub ![Terminal running ollama run llama3, pulling the model and answering a sample prompt](../../assets/blog/mac-project-ollama-ai-1.png) ## 2 - Create the API specification - Open Anypoint Code Builder > Design an API - Name it MAC-Ollama-API - Select REST API > RAML 1.0 - **Create Project** - Paste the following into this RAML file ```yaml #%RAML 1.0 title: MAC-Ollama-API version: 1.0.0 description: Simple API to connect a Mule application and Ollama locally using the MAC project. mediaType: application/json /chat: post: body: description: The question to ask Ollama type: string example: "Hello!" responses: 200: description: The response from Ollama body: type: string example: Hey! How's it going? ``` - Publish to Exchange using the Command Palette in VS Code (cmd+shift+P or ctrl+shift+P) ![VS Code Command Palette showing the "Publish API Specification to Exchange" command](../../assets/blog/mac-project-ollama-ai-2.png) - You can leave all the defaults (organization, project name, version, etc.) and wait for it to be published ## 3 - Implement the API - Select **Yes** to implement this API in ACB - Name it mac-ollama-proj, select the folder where you want to keep this, select Mule Runtime 4.8 and Java 17 - Once the project finishes loading, open the `mac-ollama-proj.xml` file under `src/main/mule` - Click on the **Flow List** button and switch to the one that starts with "post" ![Anypoint Code Builder Flow List dropdown listing the project's flows, including the post flow](../../assets/blog/mac-project-ollama-ai-3.png) - Remove the logger from the flow - Click on the plus button to add a new connector - Click on the Exchange button to search in Exchange ![Add Component panel with the Search in Exchange button highlighted, listing connectors](../../assets/blog/mac-project-ollama-ai-4.png) - Search for the MuleSoft AI Chain module and click on it - Select the **Chat answer prompt** connector ![MuleSoft AI Chain Connector operations list with Chat answer prompt selected](../../assets/blog/mac-project-ollama-ai-5.png) - Click on the new connector from the canvas - Click on the plus button next to the Connection Config ![Chat answer prompt connector config panel with the Add Connector Config plus button](../../assets/blog/mac-project-ollama-ai-6.png) - Add the following details: | Setting | Value | Notes | |---|---|---| | Name | `MAC_Config` | | | LLM type | OLLAMA | | | Config type | Configuration Json | | | File path | `mule.home ++ "/apps/" ++ app.name ++ "/llm-config.json"` | *Make sure to click on the fx button to make this a formula instead of a hardcoded value | | Model name | llama3 | *This will depend on the model you installed locally | | Temperature | 0.7 | | | LLM timeout | 60 | | | LLM timeout unit | SECONDS (Default) | | | Max tokens | 500 | | > [!NOTE] > This post was created when the MAC project was on version 1.0.0. This functionality may change in future versions. - Click Add - Back in the canvas, click on the plus button at the end to add a new connector - Add a Transform - Set it up like the following: ```dataweave output application/json --- payload ``` ![Transform component after Chat answer prompt, set to output application/json payload](../../assets/blog/mac-project-ollama-ai-7.png) > [!NOTE] > If you see a red line/dot at the beginning of the script like in the previous screenshot, head to the XML view and remove the surrounding #[ ] - You can add some loggers at the beginning and/or at the end of the flow to show the input/output payloads from the console - The Global Configuration / MAC Connection Config should look like the following (located at the beginning of the XML file): ```xml ``` - The flow should look like the following (located at the end of the XML file): ```xml ``` - Create a new file called `llm-config.json` under `src/main/resources` and paste the following: ```json { "OLLAMA": { "OLLAMA_BASE_URL": "http://localhost:11434" } } ``` ## 4 - Test the app locally - Run the application locally - Once it has started, send a POST request to [localhost:8081/api/chat](http://localhost:8081/api/chat) with a JSON body including your question ![API client POST to localhost:8081/api/chat returning a 200 OK chat response from Ollama](../../assets/blog/mac-project-ollama-ai-8.png) - If everything was successful, continue to the next step. Otherwise, please troubleshoot before continuing - Stop the app ## 5 - Use ngrok for the public endpoint - Download and install [ngrok](https://download.ngrok.com/mac-os) to make your Ollama endpoint publicly available from the internet. This way, CloudHub will be able to access the URL since it's not only in your local (localhost:11434) - Run the following from your Terminal: ```bash ngrok http 11434 --host-header="localhost:11434" ``` - Copy the address from the Forwarding field ![ngrok session screen with the public Forwarding URL pointing to localhost:11434 highlighted](../../assets/blog/mac-project-ollama-ai-9.png) - Paste it in your `llm-config.json` file under `src/main/resources` ![llm-config.json with OLLAMA_BASE_URL set to the ngrok forwarding URL](../../assets/blog/mac-project-ollama-ai-10.png) - Save the file and run the app again to verify everything still works correctly - Stop the app once you verify it works ## 6 - Deploy to CloudHub - If everything still works, you are ready to deploy to CloudHub - Go to your `pom.xml` file and change the version to 1.0.0 (remove the SNAPSHOT) ![pom.xml version changed to 1.0.0 with the SNAPSHOT suffix removed](../../assets/blog/mac-project-ollama-ai-11.png) - Save the file - Head to your `mac-ollama-proj.xml` file and click on the Deploy to CloudHub button ![mac-ollama-proj.xml editor toolbar with the Deploy to CloudHub button highlighted](../../assets/blog/mac-project-ollama-ai-12.png) - Select CloudHub 2.0 - Select the `US-EAST-2` (or whichever region is available on your account) - Select the Sandbox environment - Make sure everything looks good in the newly created `deploy_ch2.json` file and click on Deploy - Select the Runtime version (including the patch) - It will first publish the project to Exchange (as an application) - After that, it will be deployed to CloudHub - If you're experiencing a lot of issues deploying from ACB, you can also take the generated JAR from the target/ folder and deploy it manually to Runtime Manager - Once the deployment is done, get the Public Endpoint from your application and call it to verify the app works ![API client POST to the deployed CloudHub endpoint returning a 200 OK chat response](../../assets/blog/mac-project-ollama-ai-13.png) Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## How to develop AsyncAPIs on Anypoint Code Builder with Solace PubSub+ locally (Docker) Source: https://prostdev.com/post/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker | Published: Nov 7, 2024 | Category: Tutorials First of all, I am brand-new to any Solace stuff. So there I was, working on a demo to connect Mule with [Solace](https://solace.com/) PubSub+ (Message Broker) but I didn't want to create a 30 or 60-day cloud trial account with them because I wanted to be able to re-run this demo as many times as needed. So, I saw you could run the broker using Docker. The problem is that all the documentation from the MuleSoft side were for Solace in the cloud and not local. So, here I am creating this post for you! If you want to run Solace locally, using their Docker image/container, and then connect that to a MuleSoft AsyncAPI specification/application -- then, keep reading. > [!NOTE] > Take a look at the following GitHub repo to follow along with the code: > [asyncapis-accounts-email](https://github.com/alexandramartinez/asyncapis-accounts-email) ## 1 - Start the Solace Docker Container I started by following the [Getting Started with PubSub+ Standard Edition](https://solace.com/products/event-broker/software/getting-started/) page, which pretty much gives you the command to run to start the new Docker container: ```bash docker run -d -p 8080:8080 -p 55554:55555 -p 8008:8008 -p 1883:1883 -p 8000:8000 -p 5672:5672 -p 9000:9000 -p 2222:2222 --shm-size=2g --env username_admin_globalaccesslevel=admin --env username_admin_password=admin --name=solace solace/solace-pubsub-standard ``` > [!NOTE] > The command above is used for Mac only. Please refer to the website for the Linux/Windows command. However, I had one issue with the port 9000, so I had to change that to map to the port 9001 instead. If you have a port issue, you can also change it. Make sure you change the port at the **left** side and not the one at the right side: ```bash : ``` The other thing you'll notice from the website is that the commands are different depending on the operating system you're running. It states: *Your applications can use Solace APIs to connect to the message broker on this port. Note that for Mac users we are mapping port 55554 to the default Solace SMF port of 55555 because Mac OS now reserves port 55555.* Anyway, long story short, if you can run the command from the website without errors, good. If not, modify it to match your available ports. For example, since my port 9000 was busy (and I'm running Mac), this is the command I had to use: ```bash docker run -d -p 8080:8080 -p 55554:55555 -p 8008:8008 -p 1883:1883 -p 8000:8000 -p 5672:5672 -p 9001:9000 -p 2222:2222 --shm-size=2g --env username_admin_globalaccesslevel=admin --env username_admin_password=admin --name=solace solace/solace-pubsub-standard ``` Once your container is running, you should be able to see a green icon next to it from your Docker Desktop application. ![Docker Desktop Containers list with the running solace container and a green status dot](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-1.png) ## 2 - Set up the queue Once everything looks good, open your browser and head to [http://localhost:8080/](http://localhost:8080/). This will open a log-in screen to connect to Solace's management tool. If you're not seeing this screen immediately, wait a few minutes. ![Solace PubSub+ Standard login screen with the admin username filled in](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-2.png) Write the credentials admin/admin and log in. Once inside, click on the default VPN. ![Solace Message VPNs screen showing the default VPN card with status Up](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-3.png) From the left side, click on Queues and add a new Queue by clicking on the green **+Queue** button on the top right. Write the name of your queue and leave all the defaults. For our example, we'll use the name user-signup. ![Solace Queues tab with a new user-signup queue and the +Queue button](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-4.png) Click on this new queue to open its properties and head to the **Subscriptions** tab at the top. Add a new subscription/topic with the same name (user-signup). ![user-signup queue Subscriptions tab with a user-signup topic subscription added](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-5.png) You will need to do this configuration in order to receive the messages from MuleSoft. ## 3 - Create the AsyncAPI specifications Head to Anypoint Code Builder and create a new AsyncAPI specification called **Accounts Service**. We will be using version 2.6 YAML for this example. Paste the following spec. ```yaml asyncapi: '2.6.0' info: title: Account Service version: '1.0.0' description: Publishes the UserSignedUp event when a new user account is created. contact: name: Alex Martinez email: alexandra.martinez@salesforce.com url: alexmartinez.ca license: name: test url: test defaultContentType: application/json tags: - name: solace description: makes use of Solace PubSub+ servers: Solace: protocol: solace bindings: solace: msgVpn: default url: localhost:55554 channels: user-signup: subscribe: description: Publishes the UserSignedUp event operationId: emitUserSignUpEvent message: $ref: "#/components/messages/UserSignedUp" components: messages: UserSignedUp: payload: type: object properties: firstName: type: string examples: - Alex lastName: type: string examples: - Martinez email: type: string examples: - fake@email.email createdAt: type: string examples: - 2024-01-01T15:00:00Z ``` Note that we are using the **default** VPN and the `localhost:55554` URL. From the first step, the installation, you can see which port you're exposing locally to make use of port 55555 in Docker. In my case, since I'm using a MacOS, my port was changed to be 55554:55555. If you're using Linux/Windows, you probably set it up as 55555:55555. If this is the case, then please change the URL to match your port (the one on the left of the colon - 55555:). Publish this spec to Exchange and create a new one called **Email Service**. Paste the following code. ```yaml asyncapi: '2.6.0' info: title: Email Service version: '1.0.0' description: Subscribed to receive the UserSignedUp event to send the new user a welcome email. contact: name: Alex Martinez email: alexandra.martinez@salesforce.com url: alexmartinez.ca license: name: test url: test defaultContentType: application/json tags: - name: solace description: makes use of Solace PubSub+ servers: Solace: protocol: solace bindings: solace: msgVpn: default url: localhost:55554 channels: user-signup: publish: description: Subscribed to receive the UserSignedUp event operationId: onUserSignUp message: $ref: "#/components/messages/UserSignedUp" components: messages: UserSignedUp: payload: type: object properties: firstName: type: string examples: - Alex lastName: type: string examples: - Martinez email: type: string examples: - fake@email.email createdAt: type: string examples: - 2024-01-01T15:00:00Z ``` Same as before, make sure your VPN and URL match what you set up in Docker. Publish it to Exchange. > [!DOCS] > For more information about why we're using these two specifications, see [Design AsyncAPI Specifications With a Practical Example](https://blogs.mulesoft.com/dev-guides/how-to-design-asyncapi-specifications/). Video version can be found at the end of that article. ## 4 - Develop the Mule apps From Anypoint Code Builder, select the **Implement an API** option. Search for the **Accounts Service** specification you published to Exchange and start implementing it. Take a look at the `global-configs.xml` file. Since we are running Solace locally, there's no need to use the OAuth configuration. We can comment this out. ![global-configs.xml in ACB with the Solace OAuth configuration block commented out](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-6.png) Apart from that, let's just add an HTTP Listener connection. ```xml ``` Now head to the `flows.xml` file. Add a flow with an HTTP Listener and a Publish operation (from the APIKit AsyncAPI module). You can copy and paste the following XML code. ```xml ``` Finally, head to the `dev-properties.properties` file. Add the **default** value for the username and password properties. Since we won't be using OAuth, you can comment out the clientId, tokenProviderUrl, and clientSecret properties. ![dev-properties.properties with default username/password and OAuth properties commented out](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-7.png) Once you're done with the Accounts Service application, open a new VS Code window and create the Email Service application (based on the Email Service specification we just published to Exchange). Change the properties and remove the OAuth configuration from the `global-configs` file just as we did with the Accounts app. In this case, you don't need to change anything in the `flows` file. ## 5 - Run apps and test locally Run both projects locally. Since ACB doesn't allow this, as workaround, change the Mule runtime version from one of the projects to a different one. For example, accounts with runtime 4.8 and email with runtime 4.7. You can do this by right-clicking any of the files under `src/main/mule` and selecting **Project Properties**. ![ACB file explorer right-click menu with the Project Properties option highlighted](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-8.png) ![Project Properties pane setting the Mule Runtime to 4.8.0 and Java to 17](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-9.png) Once both projects are running at the same time on your local, open any REST application like Postman and send a request to localhost:8081/publish with the following JSON body: ```json { "firstName": "Alex", "lastName": "Martinez", "email": "fake@email.email", "createdAt": "2024-01-01T15:00:00Z" } ``` You should be able to see this payload from the Email app's console in ACB. ## Troubleshooting I had to do some troubleshooting because it wasn't clear to me which port I should be using to connect to my Solace broker. [This post](https://stackoverflow.com/questions/50641285/solace-connectivity-issue-using-spring-4-x) in StackOverflow was incredibly helpful! I also discovered that MuleSoft is connecting to Solace via [JCSMP](https://docs.solace.com/API/Messaging-APIs/JCSMP-API/jcsmp-api-home.htm) - it makes a LOT of sense since Mule is Java/SpringBoot under the hood, it just wasn't obvious from reading the main docs. Anyway, what I gathered from the StackOverflow post was that, in case your port (55555) was modified for whatever reason, you could double-check it with the following commands: - On a regular terminal in your local (not in Docker), run *docker exec -it solace /usr/sw/loads/currentload/bin/cli -A* - This connects to the Docker container using the Solace CLI. Then, execute *show service* - Press any key to continue You should see an output like the following: ![Solace CLI "show service" output listing the SMF over TCP port as 55555](../../assets/blog/how-to-develop-asyncapis-on-anypoint-code-builder-with-solace-pubsub-locally-docker-10.png) And from the StackOverflow post, "You are looking for the SMF over TCP port. The default port number is 55555." And as you can see from the previous image, it indeed is port 55555. If you have a different port, you will see a different number here. Then, you will just have to check which local port is exposing *that* port from Docker and use that one instead. That's all for this article! I hope this helped guide you to run Solace PubSub+ locally and understand how to run it from MuleSoft! It took me a while to understand it so I really hope if helps someone. Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## How to upsert fields from an object in an array with the update operator in DataWeave Source: https://prostdev.com/post/upsert-object-from-array | Published: Sep 16, 2024 | Category: Tutorials In this post, we'll learn how to use the update operator along with the upsert and conditional options. We'll also learn different ways of handling null values for our fields. > [!TIP] > You can follow along by clicking on the "Open in Playground" buttons to see how the code is being developed and to try it out yourself! ## Use case Let's say you have an array of objects `Array` like so: ```json [ { id: 1, name: "alex1", notes: "NA" }, { id: 2, name: "alex2", notes: "NA" }, { id: 3, name: "alex3", notes: "NA" } ] ``` And you want to update one of those objects depending on the ID. For example, using this variable: ```dataweave var updateUser = { id: 2, notes: "this is my new note", newField: "abc" } ``` The first observations we can make, based on this information, are: - The field `name` is not available from the `updateUser` variable. If we don't want to lose this field, we can't just replace the whole object. We'll need to check each field separately. - The field `newField` is not available from the original user. We will need to insert this field and not just update the current user. A quick solution to this problem is to use the [update operator](https://docs.mulesoft.com/dataweave/latest/dw-operators#update-operator) because it has an upsert option to it. Let's see how to do this. ## Solution First of all, we need to find in the array the user we want to update. To do this, we'll use a [map](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-map) function and a conditional based on the ID field: ```dataweave users map ((user) -> if (user.id ~= updateUser.id) ??? else user ) ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Fupsert-object-from-array%2Fscript1) Once we meet the condition, we can get started with the `update` operator. Since we want to have the possibility of updating all the fields (except the ID), we'll have to create `case`s for all of these. Like so: ```dataweave users map ((user) -> if (user.id ~= updateUser.id) user update { case na at .name -> updateUser.name case no at .notes -> updateUser.notes case nf at .newField -> updateUser.newField } else user ) ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Fupsert-object-from-array%2Fscript2) In summary, the object we want to update has two issues that we need to solve: ```json { "id": 2, "name": null, "notes": "this is my new note" } ``` - The field `name` now has a `null` value - The field `newField` is not being inserted yet Issue 1 is happening because we are updating the field `name` with this: `updateUser.name` - and since this field does not exist, it's returning a `null` value. The solution is quite simple: we need to add a `default` keyword with the original value, like so: ```dataweave case na at .name -> updateUser.name default na ``` Now it will take the value from the original object and use that if the `updateUser` variable does not contain this value. We can add this to the other fields just in case. Issue 2 is where we will be using our `upsert` operator because we do not have this field in our original value, so, we have to tell the operator that we want it to insert it if it doesn't exist. We do that by adding an exclamation point (`!`) right next to the field. Like so: ```dataweave case nf at .newField! -> updateUser.newField ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Fupsert-object-from-array%2Fscript3) Ta-da! 🎉 Now we are correctly inserting this new value and updating the other fields from the original object. ## Handling null values What would happen though, if the `newField` is not available in either of the two objects? Not in the original and not in the `updateUser` variable. In that case, the `update` operator will still add the field because you are using the `upsert` operator. It will just be added with a `null` value. ```dataweave "newField": null ``` Depending on your business requirements, maybe this is the way it should work. But in case it's not, you can fix it two ways: adding a conditional to each field or adding a `skipNullOn` configuration to the output directive. ### With a conditional This is the less fancy approach, but you may need to choose this if you only need to remove specific fields and not ALL the fields with null values. ```dataweave case nf at .newField! if(!isEmpty(updateUser.newField default nf)) -> updateUser.newField default nf ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Fupsert-object-from-array%2Fscript4) > [!TIP] > You can create a local scope with [do](https://docs.mulesoft.com/dataweave/latest/dataweave-flow-control#control_flow_do) if you don't want to re-type `updateUser.newField default nf` twice. ### With skipNullOn This will get rid of all the fields that have a null value. It's super effective if that's what you're looking for, but it may be confusing if later on you're expecting to see actual null values from other fields. To add this, you have to change the output directive on the top of your DataWeave script. Like so: ```dataweave output application/json skipNullOn="everywhere" ``` > [!PLAYGROUND] > [Open in Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Fupsert-object-from-array%2Fscript5) > [!DOCS] > To learn more about the JSON writer properties, see [JSON Format](https://docs.mulesoft.com/dataweave/latest/dataweave-formats-json#writer_properties). There are a lot more ways to personalize your DataWeave scripts. These are just a few options that you can use to make the best of the operators available! I hope you found this article useful. Add a comment if you found an easier way to do this! ;) Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## Part 5: Data Cloud + MuleSoft integration - Insert data with the BULK operations Source: https://prostdev.com/post/part-5-data-cloud-mulesoft-integration | Published: Jul 30, 2024 | Category: Tutorials Hello, hello! By this point, you should already have a full Mule application running in CloudHub and you have been able to call the Query/Streaming operations. In July 2024, I released a new JAR (version 2.1.0) where I added the Bulk operations. Let's take a look at what other updates have been done to this version! ## Prerequisites - **Previous configurations** - Make sure you followed at least parts 1-3 before reading this article. Part 4 is preferred for security reasons, but not necessary. You will need to do the Data Cloud, MuleSoft, and Postman configuration before doing this post. - **Deployed Mule app -** Make sure you're using the JAR version 2.1.0 or newer. You can find the releases [here](https://github.com/alexandramartinez/datacloud-mulesoft-integration/releases/). - **Postman collection -** If you have already imported the previous Postman collection, make sure you re-import the latest one (released on July 2024) from [this link](https://github.com/alexandramartinez/datacloud-mulesoft-integration/tree/main/rest-clients-requests/postman). This collection includes both the Streaming and the Bulk operations. Below is a screenshot of what the new collection looks like. ![Postman Data Cloud Integration collection listing the streaming and new bulk requests](../../assets/blog/part-5-data-cloud-mulesoft-integration-2.png) ## Understanding the Bulk process Before jumping right away into the new operations, let's take a moment to understand how Data Cloud processes this type of data. Take a look at the following diagram: ![Diagram of the bulk Job lifecycle: create, upload, close, then Data Cloud's InProgress/JobComplete states](../../assets/blog/part-5-data-cloud-mulesoft-integration-3.png) The blue circles are the actual operations you can do with the Data Cloud connector in MuleSoft and the purple squares are what Data Cloud changes on its own. A happy path processing would go like this: - Create the Job - Upload the data to the Job - Close the Job with the *UploadComplete* status - Data Cloud queues this data and changes the Job's state to *InProgress* - Once the data is processed, the Job's status is changed to *JobComplete* The Aborted state would happen if you wanted to stop the processing of the Job after it's already been created. It is irrelevant if you already uploaded the data or not. Once you abort it, the data will be deleted from the Job. The Failed state would happen if there was an error processing the Job in Data Cloud. For example, if the data you sent is corrupted, wrong, or incomplete. You can always delete a Job at any point while the job is not being processed and is already closed. This means after you close it (Aborted/UploadComplete state) but before it is queued (not in the InProgress state), or after it has been processed (Failed/JobComplete). If a specific object already contains an open/in progress Job, you won't be able to create more Jobs for the same object and you will receive a 409-Conflict error when you try to create a new Job. In this case, you will need to wait until the Job is Failed/JobComplete, or you will have to abort the other Open Job first (if it hasn't been closed). A lot of this process is already taken care of by the Mule application in the JAR file. You only need to send your data to the upsert operation and the app will take care of creating, uploading, and closing the Job for you. ## Get All Jobs Once you import the Postman collection, you will see a request called "get all jobs". This is a GET request calling the path `/api/bulk` of our Mule application. You don't need to add Query Params or a body. The credentials you set up in CloudHub are good enough to perform the call. This request will return all the Jobs with their corresponding information, like the ID and its state. *Note that the next screenshot has some blurred information for security reasons. ![Postman "get all jobs" GET returning 200 OK with a JSON array of Jobs and their states](../../assets/blog/part-5-data-cloud-mulesoft-integration-4.png) ## Upsert Once you import the Postman collection, you will see a request called "upsert (CSV)" and another one called "upsert (JSON)". Both are the same request, calling a POST in `/api/bulk/upsert`, but you have the possibility to send different bodies on each. Internally, the Mule application is transforming the input body to a CSV that can be read by Data Cloud. You need to add two Query Parameters, same as you did for the streaming insertion: - `sourceApiName` - `objectName` This request will create a new Job, upload the data, and close it with the *UploadComplete* status. If anything goes wrong while uploading the data, the Job will be aborted instead. Either way, you will receive a response with the details of the Job. *Note that the next screenshot has some blurred information for security reasons. ![Postman upsert (JSON) POST with a customer array body returning the created Job details](../../assets/blog/part-5-data-cloud-mulesoft-integration-5.png) ## Get Job Once you import the Postman collection, you will see a request called "get job info". This is a GET request calling the path `/api/bulk/{id}` of our Mule application. You need to make sure to add your Job ID in the URI of the call and you need to add two Query Parameters, same as you did for the streaming insertion: - `sourceApiName` - `objectName` This request will return the given Job's information (from the ID at the URI). You can use this to track its state. *Note that the next screenshot has some blurred information for security reasons. ![Postman "get job info" GET returning a single Job's JSON with state InProgress](../../assets/blog/part-5-data-cloud-mulesoft-integration-6.png) ## Delete Job Once you import the Postman collection, you will see a request called "delete job". This is a DELETE request calling the path `/api/bulk/{id}` of our Mule application. You only need to make sure to add your Job ID in the URI of the call. This request will delete the Job you pass in the URI. It will only be successful if the given Job is already closed. You can only delete a Job that has one of the following states: - UploadComplete (before it's InProgress, once it's queued you can't delete it) - JobComplete - Aborted - Failed If the Job is in a different state, it will try to Abort it first (this will only be successful if the Job is in an Open state). *Note that the next screenshot has some blurred information for security reasons. ![Postman delete-job DELETE request to the bulk job endpoint returning 200 OK](../../assets/blog/part-5-data-cloud-mulesoft-integration-7.png) And that's all for this post! I hope this helps to understand how to use the bulk operations with our Mule application. Remember that you can always go to my [GitHub repo](https://github.com/alexandramartinez/datacloud-mulesoft-integration) if you want to take a look at the code :) Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## Part 6: CI/CD pipeline with MuleSoft and GitHub Actions - Deploying to CloudHub 2.0 Source: https://prostdev.com/post/part-6-ci-cd-pipeline-with-mulesoft-and-github-actions-deploying-to-cloudhub-2-0 | Published: May 7, 2024 | Category: Tutorials We already learned how to set up secured properties, MUnit coverage, and even a connected app to authenticate when you're using Multi-Factor Authentication (MFA) in your Anypoint Platform account. From now on, we will continue authenticating using a connected app for best practices. What are we missing then? Well, we haven't learned how to enable the CI/CD pipelines for a CloudHub 2.0 deployment. You'd think it's very similar to CloudHub 1.0, but turns out it's very different! Don't worry, I got you :) If you haven’t been following the series or you’re not familiar with GitHub Actions, we recommend you start from the [first article](https://www.prostdev.com/post/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions) to understand how we are setting up all the configurations we need. In this post, we'll learn the basics of setting up our GitHub Actions pipeline to deploy a Mule application to CloudHub 2.0. Once more, using a Connected App. ## Prerequisites - **Anypoint Platform** - You should have an Anypoint Platform account. You can create a new free trial account [here](https://anypoint.mulesoft.com/login/signup). - **GitHub Repo** - You should already have a GitHub repo with a base Mule application and know how to configure secrets in your GitHub repo. If this is not the case, please return to the first article of this series. ## Create a connected app in Anypoint Platform We learned how to create a connected app in the [previous article](https://www.prostdev.com/post/part-5-ci-cd-pipeline-with-mulesoft-and-github-actions-enabling-mfa-through-a-connected-app), but let's go ahead and create a new one. **Sign in to** [Anypoint Platform](https://anypoint.mulesoft.com/) **and navigate to Access Management > Connected Apps.** ![Anypoint Access Management Connected Apps page with the Create app button on the Owned Apps tab.](../../assets/blog/part-6-ci-cd-pipeline-with-mulesoft-and-github-actions-deploying-to-cloudhub-2-0-2.png) **Click on Create app. Add any name you want to identify this app, like github-actions. Select App acts on its own behalf and click on Add Scopes.** > [!DOCS] > You can refer to [this article](https://help.salesforce.com/s/articleView?id=001119708&type=1) to learn more about the minimum scopes for CloudHub 2.0. **Select the following scopes:** - Exchange Contributor - View Environment - View Organization - Create Applications - Delete Applications - Manage Application Data - Manage Runtime Fabrics - Manage Settings - Read Applications Click on Next. Select your Business Group and the Sandbox environment. Review the scopes and click on Add Scopes. You should end up with something like the following. ![Connected app set to act on its own behalf with the nine selected Exchange, General, and Runtime Manager scopes.](../../assets/blog/part-6-ci-cd-pipeline-with-mulesoft-and-github-actions-deploying-to-cloudhub-2-0-3.png) After that, click on Save. Make sure you copy both ID and Secret from the generated connected app. ## Set up your credentials on GitHub Just as we’ve done before, go to your GitHub repository and click on the **Settings** tab. Select **Secrets and variables** > **Actions** and add these two new secrets. - `CONNECTED_APP_CLIENT_ID` - `CONNECTED_APP_CLIENT_SECRET` The values should match what you previously extracted from Anypoint Platform. ## Modify your pom.xml We'll start from the beginning on this so we don't get confused. Starting from a brand new project with no MUnits and no secured properties, open your `pom.xml` file from the root folder. The first thing we'll modify is the groupId. The value here has to match your Business Group ID in Anypoint Platform. Please refer to [this post](https://dev.to/devalexmartinez/4-ways-to-retrieve-your-orgidgroupid-from-anypoint-platform-2e71) to learn how to extract this value. You should end up with something like the following: ```xml 4500658a-7637-4fcf-bc7d-51d1feb28edb ``` Next, change your version so it doesn't say SNAPSHOT. If this is the first version of your app, you can leave it as 1.0.0, like so: ```xml 1.0.0 ``` Then, in the `` tag, change the Mule Maven Plugin version to at least 4.1.0 (we'll be using 4.1.1). ```xml 4.1.1 ``` And here comes the fun part. Inside the mule-maven-plugin, replace the configuration with the following: ```xml org.mule.tools.maven mule-maven-plugin ${mule.maven.plugin.version} true mule-application https://anypoint.mulesoft.com MC Sandbox Cloudhub-US-East-2 Repository ${project.artifactId} NONE 1 0.1 true ``` A few notes regarding this configuration: - `` must always be set to MC when deploying to CloudHub 2.0. - `` in this case is Sandbox, but you can change it if you have a different environment set up. - There is only one `` available for your trial account. Before setting this up, you can go to Runtime Manager > Deploy Application and see which region is available for your shared space. In our case, it's US East (Ohio). Refer to [this table](https://docs.mulesoft.com/cloudhub-2/ch2-architecture#regions-and-dns-records) to retrieve the target name you need to set up here. - Whatever you write on `` needs to match in three other places. We'll see exactly where in the rest of the article. In our case, we're setting it up as Repository. - You can use the `` field to choose between NONE, LTS, or EDGE. - NONE will set up Mule Runtime 4.4.0. - LTS will set up the latest available LTS version (changes each year). The last LTS version when this post was written was 4.6.x. This is the most stable option if you want to use 4.6. - EDGE will set up the latest available EDGE version (changes every 4 months). This is a good option if you want to always be on the latest possible version, but a few things might break along the way, so choose carefully. > [!NOTE] > You can set up additional options to customize your deployment even more. For more information about this, see [CloudHub 2.0 Deployment Parameters Reference](https://docs.mulesoft.com/mule-runtime/latest/deploy-to-cloudhub-2#cloudhub2-deploy-reference). Almost done. Now, keep scrolling down the file until you reach the `` tag. Make sure the Anypoint Exchange version is 3 instead of 2. If this is not the case, change it. It should look like the following. ```xml anypoint-exchange-v3 Anypoint Exchange https://maven.anypoint.mulesoft.com/api/v3/maven default ``` Add the following repository. Note that this `` must match the same name you set up before as ``. ```xml Repository Private Exchange repository https://maven.anypoint.mulesoft.com/api/v3/organizations/${project.groupId}/maven default ``` Aaaand finally, make sure you add a `` section at the end (before the `` closing tag) with the following values. Note that this `` must be the same as the previous repository. ```xml Repository Corporate Repository https://maven.anypoint.mulesoft.com/api/v3/organizations/${project.groupId}/maven default ``` Here's the full `pom.xml` for you to confirm everything's good. ```xml 4.0.0 4500658a-7637-4fcf-bc7d-51d1feb28edb test 1.0.0 mule-application test UTF-8 UTF-8 4.1.1 org.apache.maven.plugins maven-clean-plugin 3.1.0 org.mule.tools.maven mule-maven-plugin ${mule.maven.plugin.version} true mule-application https://anypoint.mulesoft.com MC Sandbox Cloudhub-US-East-2 Repository ${project.artifactId} NONE 1 0.1 true anypoint-exchange-v3 Anypoint Exchange https://maven.anypoint.mulesoft.com/api/v3/maven default mulesoft-releases MuleSoft Releases Repository https://repository.mulesoft.org/releases/ default Repository Private Exchange repository https://maven.anypoint.mulesoft.com/api/v3/organizations/${project.groupId}/maven default mulesoft-releases mulesoft release repository default https://repository.mulesoft.org/releases/ false org.mule.connectors mule-http-connector 1.9.0 mule-plugin Repository Corporate Repository https://maven.anypoint.mulesoft.com/api/v3/organizations/${project.groupId}/maven default ``` Did you notice we don't have the connected app's client id/secret set up in the `pom.xml`? That is because we will now authenticate using a server in the Maven `settings.xml` file instead of the `pom.xml` file. ## Modify your settings.xml In the previous articles, we've created our own `settings.xml` file in the project to simplify things. Let's do the same here. Create a new .maven folder under the root folder. Then, create a `settings.xml` file inside it. Copy and paste the following into the file. ```xml org.mule.tools Repository ~~~Client~~~ ${client.id}~?~${client.secret} ``` We will be passing the client id/secret through the GitHub Actions file so we're not hardcoding them in the project. > [!NOTE] > If you want to authenticate using username and password instead of connected app, simply add your credentials in this server. Just make sure your user has the necessary permissions. > For more information, see [Authentication Methods](https://docs.mulesoft.com/mule-runtime/latest/deploy-to-cloudhub-2#authentication-methods). ## Modify your build.yml As we've done before, create a .github folder in the root folder of the project, then, create a workflows folder, and a `build.yml` file inside it. Copy and paste the following. ```yaml name: Publish to Exchange & Deploy to CH2.0 on: push: branches: [ main ] jobs: build: runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v4 - name: Cache dependencies uses: actions/cache@v4 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: ${{ runner.os }}-maven- - name: Set up JDK 1.8 uses: actions/setup-java@v4 with: distribution: "zulu" java-version: 8 - name: Publish to Exchange run: | mvn deploy --settings .maven/settings.xml -DskipMunitTests \ -Dclient.id="${{ secrets.CONNECTED_APP_CLIENT_ID }}" \ -Dclient.secret="${{ secrets.CONNECTED_APP_CLIENT_SECRET }}" - name: Deploy to CloudHub 2.0 run: | mvn deploy --settings .maven/settings.xml -DskipMunitTests -DmuleDeploy \ -Dclient.id="${{ secrets.CONNECTED_APP_CLIENT_ID }}" \ -Dclient.secret="${{ secrets.CONNECTED_APP_CLIENT_SECRET }}" ``` As you can see, we are extracting the secrets you set up at the beginning of the article which contain your connected app's client id/secret, and passing them to the Mule runtime as the client.id and client.secret properties. ## Run the pipeline That's it! **Once you’re done with the changes, simply push a new change to the main branch and this will trigger the pipeline.** There is something very important that you need to remember tho. Every time you push a new change, you have to change the version in the `pom.xml`. Otherwise, you will receive an error saying the version has already been published. This happens because the asset is being published to Exchange and then it's deployed to CloudHub and each Exchange asset has to have a different version. ![Successful GitHub Actions build run with steps including Publish to Exchange and Deploy to CloudHub 2.0 all passing.](../../assets/blog/part-6-ci-cd-pipeline-with-mulesoft-and-github-actions-deploying-to-cloudhub-2-0-4.png) Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## New MuleSoft Community Theme for Visual Studio Code! Source: https://prostdev.com/post/new-mulesoft-community-theme-for-visual-studio-code | Published: Apr 30, 2024 | Category: News Hello! I just wanted to take a moment to share this news with you all. I created a theme for Visual Studio Code that is inspired by the MuleSoft colors. Please note that it is NOT an official theme. It's just something I created on my own. I based most colors on Anypoint Platform or other official sites under the [MuleSoft.com](http://MuleSoft.com) domain. However, that doesn't mean they're the official colors :-) I just wanted to mention that. That said...The theme is already published in the VSCode Marketplace. You can download it and start using it right away! > [!BUTTON] > [Download theme](https://marketplace.visualstudio.com/items?itemName=ProstDev.mulesoft-community-theme) ## Contribute You can start contributing via GitHub whenever you see something is missing or *just not right*. You can either create a Pull Request directly in the [GitHub repository](https://github.com/ProstDev/mulesoft-community-theme) or [raise an issue](https://github.com/ProstDev/mulesoft-community-theme/issues) for me to take a look. ## Overview Ok, enough chatting. Let's just see some screenshots! ![Anypoint Code Builder home screen with the Develop an Integration dialog open, in the MuleSoft community theme](../../assets/blog/new-mulesoft-community-theme-for-visual-studio-code-1.png) ![Anypoint Code Builder editing a Mule XML flow with the HTTP Listener config panel and code view, themed in MuleSoft colors](../../assets/blog/new-mulesoft-community-theme-for-visual-studio-code-2.png) ![Anypoint Code Builder Settings page showing Mule runtime and Language Server options in the MuleSoft theme](../../assets/blog/new-mulesoft-community-theme-for-visual-studio-code-3.png) ![Visual Studio Code welcome screen with no folder opened, showing the MuleSoft community theme colors](../../assets/blog/new-mulesoft-community-theme-for-visual-studio-code-4.png) ![Postman running inside VS Code with a Data Cloud SQL query, themed with the MuleSoft palette](../../assets/blog/new-mulesoft-community-theme-for-visual-studio-code-5.png) ![A DataWeave mapping project in VS Code with payload, MyMapping script, and Preview Output panels in the MuleSoft theme](../../assets/blog/new-mulesoft-community-theme-for-visual-studio-code-6.png) ![VS Code command palette listing DataWeave and MuleSoft commands, in the MuleSoft community theme](../../assets/blog/new-mulesoft-community-theme-for-visual-studio-code-7.png) So what do you think?? You can find more information like the color palette and the full list of sections that were customized for this theme in the [Marketplace](https://marketplace.visualstudio.com/items?itemName=ProstDev.mulesoft-community-theme) site. I am always open to feedback. This was my first version of what I envisioned for this theme. And as you can see, I am not a designer! So, a lot of things can be improved for sure :) Don't forget to leave a review in Marketplace so I can improve it. Oh! And this is just the light theme for now. I hope to create a dark theme in the future :D Thank you!! Alex --- ## Interactive tutorial - MuleSoft Anypoint Platform API Catalog by Rolando Carrasco Source: https://prostdev.com/post/interactive-tutorial-mulesoft-anypoint-platform-api-catalog-by-rolando-carrasco | Published: Apr 10, 2024 | Category: Tutorials Have you ever wanted to do some interactive tutorials while learning MuleSoft? There are not a lot of options out there for this purpose. However, MuleSoft Ambassador [Rolando Carrasco](https://www.linkedin.com/in/rolandocarrasco/) took the time to create one for us! Watch the video or follow through the article to learn how to use this awesome platform called [Killercoda](https://killercoda.com/) to learn more about API Catalog. ## Getting started To get started with the interactive tutorial, first go to [this link](https://killercoda.com/borlandc/scenario/mule-tester). This will open the Killercoda platform with the scenario that Rolando created for us. A bunch of stuff is going to be running in the terminal. Let that finish. It'll take a few minutes to get started. ![Killercoda scenario with the intro on the left and packages installing in the terminal on the right](../../assets/blog/interactive-tutorial-mulesoft-anypoint-platform-api-catalog-by-rolando-carrasco-2.png) Once the progress is finished, you will see a new prompt asking you about the sshd_config. Make sure you select "Keep the local version currently installed" and press Enter. ![openssh-server sshd_config prompt with "keep the local version currently installed" highlighted](../../assets/blog/interactive-tutorial-mulesoft-anypoint-platform-api-catalog-by-rolando-carrasco-3.png) It will now continue installing stuff. Once it's 100% finished, you will be able to see the Ubuntu $ prompt on the right screen. At this point, you can click on START (scroll down on the left screen) and get started with the tutorial. ![Killercoda scenario steps with a START button and the Ubuntu terminal prompt ready](../../assets/blog/interactive-tutorial-mulesoft-anypoint-platform-api-catalog-by-rolando-carrasco-4.png) ## Follow the instructions The tutorial's instructions are located to the left side of the screen while the actual terminal is located to the right. Once you start the tutorial, you will see some commands that you need to run in the terminal. Killercoda lets you click on those commands on the left to automatically execute them on the right (on the terminal). You do not need to copy and paste the commands. ![Tutorial step "Install cURL, jq, Node and NPM" with a clickable command block and exec button](../../assets/blog/interactive-tutorial-mulesoft-anypoint-platform-api-catalog-by-rolando-carrasco-5.png) There will also be some instructions for you to follow outside of Killercoda. For example, creating a Connected App in Anypoint Platform or gathering your Organization ID. However, the screenshots in the interactive tutorial are very useful. ![Connected App scopes screenshot in the tutorial instructions, selecting the View Environment scope](../../assets/blog/interactive-tutorial-mulesoft-anypoint-platform-api-catalog-by-rolando-carrasco-6.png) There is also an Editor tab located at the top of the terminal window. ![Killercoda Editor tab open showing a VS Code-style file explorer and the Ubuntu terminal](../../assets/blog/interactive-tutorial-mulesoft-anypoint-platform-api-catalog-by-rolando-carrasco-7.png) This Editor is based on Visual Studio Code and is useful to use the File Explorer to verify certain files have been downloaded to the remote system. You will be using this Editor in the tutorial at some point to modify the Catalog files. ## Other considerations Killercoda lets you use the platform for free but it asks you to finish the tutorial in less than 60 minutes. This is ok. The total time of this tutorial is around 20 minutes. You will have plenty of time to finish. If you have any feedback about this interactive tutorial, make sure to comment on this article or message [Rolando Carrasco](https://www.linkedin.com/in/rolandocarrasco/) directly. We hope to see more interactive tutorials like these from the community! Thanks again Rolando for taking the time and create this amazing tutorial!! Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## Exposing DataWeave: Map+Filter vs. Reduce - which is faster? Source: https://prostdev.com/post/exposing-dataweave-map-filter-vs-reduce-which-is-faster | Published: Mar 26, 2024 | Category: Opinion I was working on some code that was using both map and filter together and I started thinking if there was a better way to refactor this code to make it more performant. Let me first talk about the use case so you get the context of the problem. ## The use case It all starts with a JSON payload that is an array of objects with only three fields (to simplify the example): ```json [ { "id": 1, "criteria": 5, "yearOfBirth": 2000 }, { "id": 2, "criteria": 0, "yearOfBirth": 1990 } ] ``` The goal is to filter out the objects in which the criteria is less than a number (let's say 3 for this example). Plus, some fields are being added to each object. So, the expected output would be something like this: ```json [ { "id": 2, "criteria": 0, "yearOfBirth": 1990, "isValid": true, "additionalField": "something", "years": 34 } ] ``` The object with id 1 was removed because the criteria is not less than 3. I'm sure you have already started thinking of ways to generate this output but we're not there yet. Stay with me. ## Using map and filter The original solution was making use of both map and filter like this: ```dataweave items map { ($), isValid: $.criteria < 3, // needed for filter additionalField: "something", years: now().year - $.yearOfBirth } filter ($.isValid) ``` You might think that I should've done the filter before the map, but that's not the point of this article. Again, stay with me! So I have this code. First I'm doing the map and adding the three new fields, and then I'm doing the filter. The first thing that came to my mind was that I was doing two sets of iterations to the whole array: one for the map and one for the filter. > [!NOTE] > Even if I did the filter before the map, it would've been more than one iteration to the whole array (one full iteration for filter and a partial iteration to map). But again, not the point xD So I started thinking about how I could *reduce* the number of iterations to just one instead of two. And of course, the reduce function came to my mind. ## Using reduce Here we go then. The code I first thought to use was this: ```dataweave items reduce (item, acc=[]) -> if (item.criteria < 3) acc + { (item), isValid: item.criteria < 3, // no longer needed additionalField: "something", years: now().year - item.yearOfBirth } else acc ``` It's pretty much the same thing as the map, but it's only appending the objects in which the criteria is matched and leaving the other objects behind. I figured this was definitely the better approach because now we're only doing one iteration of the whole array and not more than one! ![Smiling Anakin and Padmé Star Wars "for the better, right?" meme top panel](../../assets/blog/exposing-dataweave-map-filter-vs-reduce-which-is-faster-2.png) Right? ![...right?](../../assets/blog/exposing-dataweave-map-filter-vs-reduce-which-is-faster-3.png) ## Timing the approaches I hope you're in disbelief by this point and thinking "*There's no way map+filter is quicker than reduce"* because that's what I thought. So let's see...We can use the [Timer](https://docs.mulesoft.com/dataweave/latest/dw-timer) module to check this out. Especially the [time](https://docs.mulesoft.com/dataweave/latest/dw-timer-functions-time) function. And because I wanted to test these - like, *really* test them - I created some code to generate 10,000 objects. If you want to try it yourself, you can simply up this number in line 4. Also, just because some of you are gonna be wondering about the times of filter+map as opposed to the map+filter, I did that too. Here's the code: > [!PLAYGROUND] > [Open in the DW Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=ProstDev%2Fdataweave-playground-previews&path=scripts%2Ftiming-map-filter-reduce) ```dataweave %dw 2.0 output application/json import time from dw::util::Timer var items = (1 to 10000) as Array map { id: $$, criteria: randomInt(100), yearOfBirth: 1900 + randomInt(123) } fun mapAndFilter() = items map { ($), isValid: $.criteria < 50, // needed for filter additionalField: "something", years: now().year - $.yearOfBirth } filter ($.isValid) fun onlyReduce() = items reduce (item, acc=[]) -> if (item.criteria < 50) acc + { (item), isValid: item.criteria < 50, // can be removed additionalField: "something", years: now().year - item.yearOfBirth } else acc fun filterAndMap() = items filter ($.criteria < 50) map { ($), isValid: $.criteria < 50, // can be removed additionalField: "something", years: now().year - $.yearOfBirth } --- { mapAndFilter: time(() -> mapAndFilter()) then $.end - $.start, onlyReduce: time(() -> onlyReduce()) then $.end - $.start, filterAndMap: time(() -> filterAndMap()) then $.end - $.start } mapObject ((value, key, index) -> (key): (value as String)[2 to -2] as Number ) orderBy $ ``` And here's the result of the previous code (I ran it in the DW Playground) ```json { "filterAndMap": 0.000046, "mapAndFilter": 0.000063, "onlyReduce": 0.124531 } ``` The exact timings will vary every time, but it mostly stays in the same order. We had already predicted that doing filter+map would be faster than map+filter, but had you predicted that reduce would be the slowest?! And by so much more?! If you are skeptical about the results being generated correctly, you can also change the script to the following and run it for every function: ```dataweave time(() -> filterAndMap()) then ( { totalTime: (($.end - $.start) as String)[2 to -2] as Number } ++ $ ) ``` I still see different times depending on the function. Check it out: ![filterAndMap](../../assets/blog/exposing-dataweave-map-filter-vs-reduce-which-is-faster-4.png) ![onlyReduce](../../assets/blog/exposing-dataweave-map-filter-vs-reduce-which-is-faster-5.png) Both generate around 40,000 lines of output. ## Conclusion Nothing is as it seems 🫨 If some milliseconds are a difference for your use case, time your approaches before assuming! Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## Part 4: Data Cloud + MuleSoft integration - Secure your API with basic authentication in API Manager Source: https://prostdev.com/post/part-4-data-cloud-mulesoft-integration | Published: Feb 28, 2024 | Category: Tutorials We already finished our Data Cloud integration with MuleSoft. We have everything deployed and running in CloudHub, and we know how to call this integration to query, insert, or delete records from/to Data Cloud. However, our integration has a fatal flaw: we have no security. In [Part 2](https://www.prostdev.com/post/part-2-data-cloud-mulesoft-integration) of this series, we learned that we should not share the CloudHub URL because everyone had access to our Data Cloud credentials with just the link. There is a simple solution to this problem. In this post, we'll learn how to connect our Mule app (deployed in CloudHub) to API Manager to add basic authentication. This way, even if you share your URL by mistake, people would still need your username/password to access it. ## Prerequisites - **Anypoint Platform** - You should have an Anypoint Platform account. You can create a new free trial account [here](https://anypoint.mulesoft.com/login/signup). It will expire in 30 days but you can create a new one using the same details to register, just making sure to change the username to a different one. - **Deployed Mule app** - Make sure you followed [Part 2](https://www.prostdev.com/post/part-2-data-cloud-mulesoft-integration) of this series before starting. If you have already followed it and deployed version 1.0.0, I'll teach you how to update it to 2.0.0. If there are newer versions, feel free to use those. - **Mule app's JAR** - If you deployed your app with version 1.0.0, make sure you download the version 2.0.0 JAR file. You can find the releases for this Mule project [here](https://github.com/alexandramartinez/datacloud-mulesoft-integration/releases). - **Anypoint Platform's Environment Credentials** - In Anypoint Platform, go to Access Management > Business Groups > select your business group > Environments > Sandbox. Copy the Client ID and Client Secret. See the [docs](https://docs.mulesoft.com/access-management/environments#to-view-the-client-id-and-client-secret-for-an-environment) for more information. - **Postman & the Postman collection** - You can use any other REST Client of your choice, but we used Postman in the [previous article](https://www.prostdev.com/post/part-3-data-cloud-mulesoft-integration), so we'll continue with it. You can download other collections from [this folder](https://github.com/alexandramartinez/datacloud-mulesoft-integration/tree/main/rest-clients-requests). ## Create API in API Manager Sign in to your [Anypoint Platform](https://anypoint.mulesoft.com/) account and head to **API Manager** using the hamburger menu on the top left. ![Anypoint Platform hamburger menu with API Manager highlighted under the APIs section](../../assets/blog/part-4-data-cloud-mulesoft-integration-2.png) In API Manager, click **Add API** > **Add new API**. In the Runtime section, select the **Mule Gateway** runtime and click on **Next**. ![Add API Runtime step with Mule Gateway selected and connect-to-existing-application proxy type](../../assets/blog/part-4-data-cloud-mulesoft-integration-3.png) In the API section, select the **Create new API** option. Add the name **Data Cloud API** and the **HTTP API** asset type. Click on **Next**. ![Add API step creating a new API named Data Cloud API with the HTTP API asset type](../../assets/blog/part-4-data-cloud-mulesoft-integration-4.png) Leave the defaults for Downstream and Upstream (empty) and review the configurations. Click **Save**. ![Review step summarizing the Mule Gateway runtime, Data Cloud API v1, and HTTP downstream scheme](../../assets/blog/part-4-data-cloud-mulesoft-integration-5.png) Copy the **API Instance ID**. We will use it in later steps. ![API Summary page showing the API Instance ID 19307742 and an Unregistered API status](../../assets/blog/part-4-data-cloud-mulesoft-integration-6.png) ## Upgrade the deployed Mule app The previous posts were based on the version 1.0.0 of the JAR file. If you already deployed your Mule app with that version, you'll just have to upload the new 2.0.0 JAR (or newer) to the same application in Runtime Manager. If you deployed your Mule app with this new version already, you can skip this part. > [!TIP] > If you're unsure of the JAR version you deployed, you can check. it by going to your Anypoint Platform account > Runtime Manager > open your app > go to the Settings tab > take a look at the JAR file under **Application File**. It will have the version number. Let's see how to update the JAR from Runtime Manager. Sign in to your Anypoint Platform account and go to **Runtime Manager**. Select the `data-cloud-integration` application you had previously deployed. In the **Settings** tab, locate the **Application File**. Click on **Choose file** > **Upload file** and select the new 2.0.0 JAR (or newer version) you downloaded from the Prerequisites. Don't apply your changes yet. Let's add some more properties first. ![Runtime Manager Settings for the running app, uploading a new JAR via Choose file then Upload file](../../assets/blog/part-4-data-cloud-mulesoft-integration-7.png) ## Add Autodiscovery properties + credentials Before applying your changes, head to the **Properties** tab and switch to the **Text view**. ![Properties tab switched to Text view, with existing protected Salesforce and Data Cloud values below](../../assets/blog/part-4-data-cloud-mulesoft-integration-8.png) Copy the following snippet and replace the placeholders with your values. ```properties anypoint.platform.gatekeeper=flexible api.id=your_api_instance_id anypoint.platform.client_id=your_client_id anypoint.platform.client_secret=your_client_secret ``` > [!NOTE] > If you had already set up the `anypoint.platform.gatekeeper` property to disabled, make sure you change it to flexible. See the [docs](https://docs.mulesoft.com/mule-gateway/mule-gateway-gatekeeper) for more info. ![Properties text view with the gatekeeper, api.id, client_id, and client_secret values filled in](../../assets/blog/part-4-data-cloud-mulesoft-integration-9.png) Go back to the **Table view** and click on **Protect** for your two new Anypoint Platform credentials. Once you're done, click **Apply Changes**. It will take some time to redeploy. ## Add the Basic Authentication policy As you learned from [Part 3](https://www.prostdev.com/post/part-3-data-cloud-mulesoft-integration), you can use Postman to call your integration. If you try sending a request to the Query request now, you should still receive a 200-OK response because we haven't added the security policy yet. ![Postman SELECT query returning a 200 OK with a Data Cloud profile record, before the policy is applied](../../assets/blog/part-4-data-cloud-mulesoft-integration-10.png) Head back to Anypoint Platform > **API Manager**. Select the Data Cloud API we created and make sure the API Status now appears as 🟢 **Active**. You may need to refresh the page. If it doesn't appear as Active yet, make sure your request works and your properties in Runtime Manager are correct. ![API Summary now showing the Data Cloud API status as Active with Mule version 4.4.0](../../assets/blog/part-4-data-cloud-mulesoft-integration-11.png) Then, head to the **Policies** tab on the left and click on **Add policy** under the API-level policies section. ![Policies tab with no policies applied and an Add policy button under API-level policies](../../assets/blog/part-4-data-cloud-mulesoft-integration-12.png) Search for the **Basic Authentication - Simple** policy and click on **Next**. ![Add a policy search for basic authentication with Basic Authentication - Simple selected](../../assets/blog/part-4-data-cloud-mulesoft-integration-13.png) Create your own username/password to access your integration. **Do NOT** use the foo/bar example from the screenshot. Create your own credentials - they can be anything you want. Once you're done, click **Apply**. ![Configure Basic Authentication policy with example User Name foo and User Password bar](../../assets/blog/part-4-data-cloud-mulesoft-integration-14.png) Try calling your Query request once more. After a few seconds, you should receive a **401-Unauthorized** error code. This means the security policy was successfully applied. ![Postman query now returning 401 Unauthorized with a missing-security-context error message](../../assets/blog/part-4-data-cloud-mulesoft-integration-15.png) ## Add your credentials to Postman Now we need to send our new username/password credentials with Postman. There are several ways of doing this. You can: - Hardcode them to each one of the requests - Hardcode them in the collection - Add them to the environment and reference them from each one of the requests - Add them to the environment and reference them from the collection The last one sounds more secure. Let's do that one. In Postman, open. the **Environments** tab and select the **CloudHub** environment we had previously created. Add two new variables: **username** and **password**. Add your values to the Current value field. ![Postman CloudHub environment with host, username, and password variables and their current values](../../assets/blog/part-4-data-cloud-mulesoft-integration-16.png) Click on **Save** and open the **Collections** tab. Click on the Data Cloud Integration collection. This will open a new window with different tabs for the collection. Select the **Authorization** tab and the **Basic Auth** type from the dropdown. Once you're able to see the Username/Password fields with their textboxes, add the reference to your variables with the following syntax: ``` {{variableName}} ``` ![Postman collection Authorization tab set to Basic Auth referencing the username and password variables](../../assets/blog/part-4-data-cloud-mulesoft-integration-17.png) Make sure you **Save** your changes. Now all the requests located inside this collection will inherit the same authentication settings. Try sending the Query request one more time. You should now receive a **200-OK** status code. ![Postman query returning 200 OK again with the Data Cloud profile record after adding credentials](../../assets/blog/part-4-data-cloud-mulesoft-integration-18.png) ## Troubleshooting If you receive a 503 error code when calling the integration from Postman after setting up Autodiscovery (`api.id`), you'll have to wait a few seconds for the security policies to be applied to your API. If a long time has passed and you're still receiving 503 instead of 200, make sure the `api.id` property is correctly set and your API in API Manager appears as Active (make sure to refresh). If you're still experiencing the error, try changing the `anypoint.platform.gatekeeper` property to disabled and try again. Once it works, try changing it back to flexible. ## Conclusion Anypoint Platform offers security policies for your APIs that you can apply using clicks, not code. We used one of the most basic policies for this tutorial, but there are a lot of other policies you can apply to improve your APIs. Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## Part 3: Data Cloud + MuleSoft integration - Call your integration with Postman Source: https://prostdev.com/post/part-3-data-cloud-mulesoft-integration | Published: Feb 20, 2024 | Category: Tutorials A little bird told me that deleting records in Data Cloud is actually not that easy to do. So, I did my research and came up with a Mule application for you all to reuse to (hopefully) make your lives easier when dealing with Data Cloud! In this third part, we'll learn how to use our integration. We'll use [Postman](https://www.postman.com/) for this article, but you can use any other REST client like [Thunder Client](https://www.thunderclient.com/), [cURL](https://curl.se/), or [Advanced REST Client](https://www.advancedrestclient.com/). ## Prerequisites - **Postman collection** - Download the Postman collection from [this link](https://github.com/alexandramartinez/datacloud-mulesoft-integration/tree/main/rest-clients-requests/postman). You can also find other collections/requests [here](https://github.com/alexandramartinez/datacloud-mulesoft-integration/tree/main/rest-clients-requests). - **Postman** - Make sure you download [Postman](https://www.postman.com/). You can also use the web version by creating a Postman account, instead of installing the app locally. - **Mule app URL** - By the end of the [previous article](https://www.prostdev.com/post/part-2-data-cloud-mulesoft-integration), we retrieved the public Endpoint/URL of our deployed Mule application. Make sure you have this before starting. ## Import the collection In Postman, make sure you're located inside the **Collections** tab and click the **Import** button. ![Postman Collections tab with the Import button highlighted](../../assets/blog/part-3-data-cloud-mulesoft-integration-2.png) Select the Postman collection you downloaded from the Prerequisites. After you import it, you should now have something like the following. > [!NOTE] > The following screenshot contains only the **Streaming** insert/delete operations (from when the post was created). The **Bulk** operations are introduced after the JAR version 2.1.0, updated on July 2024. ![Imported Data Cloud Integration collection with schema, query, insert and delete requests](../../assets/blog/part-3-data-cloud-mulesoft-integration-3.png) ## Set up the environment Select the **Environments** tab from the left (next to Collections). We could create a Global variable, but we're going to create a new environment to follow best practices. Click on **Create Environment**. ![Empty Postman Environments tab with a Create Environment link](../../assets/blog/part-3-data-cloud-mulesoft-integration-4.png) Let's name this environment **CloudHub**. Add a variable called **host** and the current value field will be your Mule app URL. ![CloudHub Postman environment with a host variable set to the Mule app URL](../../assets/blog/part-3-data-cloud-mulesoft-integration-5.png) Save the variable by clicking on the **Save** button at the top-right. Select this new environment from the environment dropdown located at the top-right (over the Save button). ![Postman environment dropdown listing No Environment and the CloudHub environment](../../assets/blog/part-3-data-cloud-mulesoft-integration-6.png) Go back to the **Collections** tab. You can now run the requests! ## schema With this endpoint, you can send a JSON object to be transformed into the OpenAPI YAML schema. This is needed to create your Ingestion API and Data Stream in Data Cloud. Because the Ingestion API doesn't accept nested objects on the schema, this endpoint will transform your multi-level object into the single level needed for Data Cloud. For example, if you paste the following input payload under the **Body** tab of the request: ```json { "customer": { "id": 1, "first_name": "Alex", "last_name": "Martinez", "email": "alex@sf.com", "address": { "street": "415 Mission Street", "city": "San Francisco", "state": "CA", "postalCode": "94105", "geo": { "lat": 37.78916, "lng": -122.39521 } } } } ``` ![Postman schema POST request with a nested customer JSON object in the raw body](../../assets/blog/part-3-data-cloud-mulesoft-integration-7.png) it would be first flattened and transformed into the following output: ```json { "customer": { "id": 1, "first_name": "Alex", "last_name": "Martinez", "email": "alex@sf.com", "street": "415 Mission Street", "city": "San Francisco", "state": "CA", "postalCode": "94105", "lat": 37.78916, "lng": -122.39521 } } ``` then, based on this new input, you will receive the YAML schema like the following: ```yaml openapi: 3.0.3 components: schemas: customer: type: object properties: id: type: number first_name: type: string last_name: type: string email: type: string street: type: string city: type: string state: type: string postalCode: type: string lat: type: number lng: type: number ``` If you want to change the OpenAPI version, you will have to set a different version using the openapiversion query parameter. To do this, go to the **Params** tab inside the request and modify the value. ![Postman Params tab showing the openapiversion query parameter](../../assets/blog/part-3-data-cloud-mulesoft-integration-8.png) > [!NOTE] > This is the only request that does not connect to Data Cloud and is not using your Data Cloud credentials. If you need to use this request before setting up your Salesforce configurations, you can go through the Mule app deployment (Part 2) and just input random credentials as a placeholder for the secured properties. You can modify them later in CloudHub. ## query Send your SOQL query in the body of the request in a text/plain format. You can modify the query by going to the **Body** tab inside the request. ![Postman query POST request with a SOQL SELECT statement in the body](../../assets/blog/part-3-data-cloud-mulesoft-integration-9.png) You will receive a JSON response with the result from Data Cloud. For example, from the previous query, you'd receive a JSON Array with the results of the SELECT: ```json [ { "DataSourceObject__c": "MuleSoft_Ingestion_API_runner_profiles_38447E8E", "DataSource__c": "MuleSoft_Ingestion_API_996db928_2078_4e3a_9c67_1c80b32790aa", "city__c": "Toronto", "created__c": "2017-07-21", "email__c": "alex@sf.com", "first_name__c": "Alex", "gender__c": "NB", "last_name__c": "Martinez", "maid__c": 1.000000000000000000, "state__c": "ON" } ] ``` If there are no records matching the query, you'll receive an empty array ([ ]). ## insert Make sure you add the following query parameters (in the **Params** tab) to let Data Cloud know more information about where you want to insert new records: ![Postman insert request Params tab with sourceApiName and objectName query parameters](../../assets/blog/part-3-data-cloud-mulesoft-integration-10.png) Next, in the body of the request (**Body** tab), make sure to use a JSON Array. Each Object inside this Array is a new record. For example: ```json [ { "maid": 1, "first_name": "Alex", "last_name": "Martinez", "email": "alex@sf.com", "gender": "NB", "city": "Toronto", "state": "ON", "created": "2017-07-21" } ] ``` ![Postman insert request body with a JSON array of records to add](../../assets/blog/part-3-data-cloud-mulesoft-integration-11.png) > [!IMPORTANT] > Streaming in Data Cloud in limited to max 200 records per insertion. If everything runs smoothly, you will receive a **200 - OK** successful response. > [!NOTE] > It may take a few minutes for your data to be updated in Data Cloud. You can manually check the records in Data Cloud or wait to attempt the `/query` from your MuleSoft API. ## delete Make sure you add the following query parameters (in the **Params** tab) to let Data Cloud know more information about where you want to insert new records: ![Postman delete request Params tab with sourceApiName and objectName query parameters](../../assets/blog/part-3-data-cloud-mulesoft-integration-12.png) Next, in the body of the request, make sure to use a JSON Array. Each Object inside this Array is the Primary Key of the record to delete. For example: ```json [ 1 ] ``` ![Postman delete request body with a JSON array of primary keys to remove](../../assets/blog/part-3-data-cloud-mulesoft-integration-13.png) > [!IMPORTANT] > Streaming in Data Cloud in limited to max 200 records per deletion. If everything runs smoothly, you will receive a **200 - OK** successful response. > [!NOTE] > It may take a few minutes for your data to be updated in Data Cloud. You can manually check the records in Data Cloud or wait to attempt the /query from your MuleSoft API. ## Conclusion That's it! That's all you need to know to get your Mule application going. I will be working on more enhancements to this app and I'll be writing more articles for you to learn how to implement/edit this code on your own. Keep posted! Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## Part 2: Data Cloud + MuleSoft integration - Deploy your own Mule app on Anypoint Platform (CloudHub) Source: https://prostdev.com/post/part-2-data-cloud-mulesoft-integration | Published: Feb 13, 2024 | Category: Tutorials A little bird told me that deleting records in Data Cloud is actually not that easy to do. So, I did my research and came up with a Mule application for you all to reuse to (hopefully) make your lives easier when dealing with Data Cloud! In this second part, we'll go through the MuleSoft side of the integration and you'll deploy your own Mule app to CloudHub. You do not have to know MuleSoft beforehand. I will guide you through every step. ## Prerequisites - **Anypoint Platform** - You should have an Anypoint Platform account. You can create a new free trial account [here](https://anypoint.mulesoft.com/login/signup). It will expire in 30 days but you can create a new one using the same details to register, just making sure to change the username to a different one. - **Mule app's JAR** - You will need to download the Mule application in a JAR format. You can find the releases for this Mule project [here](https://github.com/alexandramartinez/datacloud-mulesoft-integration/releases). By the time this post was created, the latest release was v1.0.0. However, if there is new functionality added later, it might be better for you to use the latest version. - **Salesforce credentials** - Make sure you followed the [previous post](https://www.prostdev.com/post/part-1-data-cloud-mulesoft-integration) because you will need some of those credentials to follow this post. You will need: - `salesforce.username` - The username you use to log in to [Salesforce](https://login.salesforce.com/) - `salesforce.password` - The password you use to log in to [Salesforce](https://login.salesforce.com/) - `cdp.consumer.key` - The Consumer Key for the Connected App you created in Salesforce - `cdp.consumer.secret` - The Consumer Secret for the Connected App you created in Salesforce ## Deploy application Sign in to your [Anypoint Platform](https://anypoint.mulesoft.com/) account and navigate to Runtime Manager from the left-hand side menu. ![Anypoint Platform header with the hamburger Products menu button](../../assets/blog/part-2-data-cloud-mulesoft-integration-2.png) ![Anypoint Platform products menu with Runtime Manager highlighted](../../assets/blog/part-2-data-cloud-mulesoft-integration-3.png) If this is your first time opening Runtime Manager, you will be asked to choose an environment. Choose **Sandbox**. Once inside, click on the blue **Deploy application** button. ![Empty Runtime Manager Applications view in the Sandbox with a Deploy application button](../../assets/blog/part-2-data-cloud-mulesoft-integration-4.png) Add any application name—for example, `data-cloud-integration`. **CloudHub 2.0** should be selected by default under Deployment Target, if not, please select it. Under Application File, select **Choose file** > **Upload File** and select the JAR file you downloaded from the Prerequisites. ![Deploy Application form with app name, CloudHub 2.0 target and the uploaded JAR file](../../assets/blog/part-2-data-cloud-mulesoft-integration-5.png) > [!NOTE] > We will deploy to CloudHub 2.0, but if you're deploying to CloudHub 1.0, the application name has to be unique across all applications worldwide. If the application name is not available, you can add your username at the end to make it unique. For example: `data-cloud-integration-amartinez`. ## Runtime tab Under the **Runtime** tab (which should be already open), make sure the Release Channel is set to **None** so we can select the Runtime Version **4.4.0**. You can experiment with different runtimes if you want! However, when this app was created and this post was published, I used version 4.4.0. I'll document in future versions of the JAR file if a new runtime is supported. > [!NOTE] > If you use the JAR version 2.1.0 or newer, you can select the runtime 4.7.0 and/or Java 11/17. You can leave the rest of the settings with the default values. ![Runtime tab with Release Channel set to None and Runtime Version 4.4.0](../../assets/blog/part-2-data-cloud-mulesoft-integration-6.png) ## Properties tab Now go to the **Properties** tab. Click on the **Text view** switch so you can copy and paste the following properties (add your actual credentials for each property): ```properties salesforce.username=your_username salesforce.password=your_password cdp.consumer.key=your_key cdp.consumer.secret=your_secret anypoint.platform.gatekeeper=disabled ``` > [!NOTE] > The last property, `anypoint.platform.gatekeeper` has to be added for the JAR version 2.0.0 and newer. You can skip it if you're using the 1.0.0 version. ![Properties tab Text view with the Salesforce and Data Cloud credentials pasted in](../../assets/blog/part-2-data-cloud-mulesoft-integration-7.png) Switch back to the **Table view** and click on the **Protect** button next to each of the values (this button is only available for CloudHub 2.0). ![Properties tab Table view with a Protect button next to each property value](../../assets/blog/part-2-data-cloud-mulesoft-integration-8.png) Now your credentials are not visible for added security. After this, click on **Deploy Application**. There is no need to modify the Ingress or Logging tabs. ![Properties table showing each credential value as a protected value](../../assets/blog/part-2-data-cloud-mulesoft-integration-9.png) ## Copy the URL Wait a few seconds for the JAR file to be uploaded. After this, you might see your app has a 🔴 red circle. Don't worry about this. If you go to the **Settings** tab you will see the yellow circle saying the configuration is still being applied. Wait a few more minutes until it's deployed 🟢 (green circle). It can take up to 10 minutes. ![App Settings showing data-cloud-integration applying configuration with status Not running](../../assets/blog/part-2-data-cloud-mulesoft-integration-10.png) Once the app has been deployed and the Application Status says 🟢 Running, you can retrieve the **public endpoint**/URL from either the **Settings** or the **Dashboard** tab. ![App Settings with status Running and the public endpoint CloudHub URL shown](../../assets/blog/part-2-data-cloud-mulesoft-integration-11.png) Copy this URL. We will use it in the next article to call the integration. > [!CAUTION] > **DO NOT** share this URL publicly. Anyone with this URL will be able to access your Data Cloud API (my URL from the previous screenshot is no longer available). > > If you wish to create a new URL, you will have to delete this app from the Settings tab (where it says Stop at the top-right) and deploy a new one with a new name following the same steps. > > You can **Stop** and **Start** your app only when you are using it to avoid unwanted requests. ## Conclusion After you follow all these steps, you should now have the public endpoint/URL that we will use to call our Mule application. The Mule app will then connect with Data Cloud using the credentials we provided in the settings. Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## Part 1: Data Cloud + MuleSoft integration - Connected App, Ingestion API & Data Stream settings in Salesforce Source: https://prostdev.com/post/part-1-data-cloud-mulesoft-integration | Published: Feb 7, 2024 | Category: Tutorials A little bird told me that deleting records in Data Cloud is actually not that easy to do. So, I did my research and came up with a Mule application for you all to reuse to (hopefully) make your lives easier when dealing with Data Cloud! In this first part, we'll go through the Salesforce/Data Cloud settings that we need to set up before even calling Data Cloud through the Mule app. > *I want to thank* [Mohammad Arsha](https://www.linkedin.com/in/mohammad-arsha-7b18381b6/) *for creating this useful* [article](https://medium.com/another-integration-blog/connect-data-cloud-with-mulesoft-bfad40fe2e8e) *that got me started on the Data Cloud/Salesforce configuration. Keep creating content!* ## Prerequisites - **Salesforce Account with Data Cloud** - You should have a Salesforce account where you will be able to access Data Cloud. For more info, see [Salesforce Data Cloud](https://www.salesforce.com/products/data/). ## Connected App settings The first thing we're going to do is to create a Connected App in [Salesforce](https://login.salesforce.com/) to authenticate to Data Cloud. To do this, first enter the **Setup** from the top-right of the screen. ![Salesforce gear menu showing the Setup option for the current app](../../assets/blog/part-1-data-cloud-mulesoft-integration-2.png) Search for **App Manager** using the Quick Find input from the left and click on **New Connected App** on the top-right of the screen. ![Salesforce App Manager with the New Connected App button highlighted](../../assets/blog/part-1-data-cloud-mulesoft-integration-3.png) Fill in the following fields and click **Save** and **Continue** once you're done. | **Field** | **Value** | |---|---| | Connected App Name | MuleSoft Integration Connected App | | API Name | `MuleSoft_Integration_Connected_App` | | Contact Email | your email | | Enable OAuth Settings | ✅ | | Callback URL | [https://login.salesforce.com/services/oauth2/callback](https://login.salesforce.com/services/oauth2/callback) | | Selected OAuth Scopes | - Access Interaction API resources (`cdp_api`)
- Access all Data Cloud API resources (`cdp_api`)
- Manage Data Cloud Calculated Insight data (`cdp_calculated_insight_api`)
- Manage Data Cloud Identity Resolution (`cdp_identityresolution_api`)
- Manage Data Cloud Ingestion API data (`cdp_ingest_api`)
- Manage Data Cloud profile data (`cdp_profile_api`)
- Manage user data via Web browsers (`web`)
- Perform ANSI SQL queries on Data Cloud data (`cdp_query_api`)
- Perform segmentation on Data Cloud data (`cdp_segment_api`) | | Enable Client Credentials Flow | ✅ | In the created Connected App detail page, click on **Manage Consumer Details** ![Connected App API section with the Manage Consumer Details button](../../assets/blog/part-1-data-cloud-mulesoft-integration-4.png) Enter the code sent to your email and click on **Verify**. Once inside the Consumer Details page, click on **Generate** and copy the newly generated **Staged Consumer Key** and **Staged Consumer Secret**. You will use these credentials for your Mule integration. After that, click on **Apply** and **Apply** again. Your staged consumer details should be now your main consumer details. ![Connected App Consumer Details page with masked Consumer Key and Secret and Copy buttons](../../assets/blog/part-1-data-cloud-mulesoft-integration-5.jpg) > [!IMPORTANT] > These credentials will be used as `cdp.consumer.key` and `cdp.consumer.secret` in your Mule application's settings in Anypoint Platform (the properties in Runtime Manager). You can close this window after you've copied the credentials. ## OAuth settings Still inside the **Setup** page, search for **OAuth and OpenID Connect Settings** in the Quick Find input box and click on it. Make sure the **Allow OAuth Username-Password Flows** option is **On**. ![OAuth and OpenID Connect Settings with Allow OAuth Username-Password Flows toggled On](../../assets/blog/part-1-data-cloud-mulesoft-integration-6.png) ## Ingestion API settings If you already have an Ingestion API **with a YAML schema**, just take note of the name of the Ingestion API you created. Otherwise, follow the next steps to create one. Still inside the **Setup** page, search for **Ingestion API** in the Quick Find input box and click on it. Click on **New**. ![Data Cloud Ingestion API setup page with the New button to create a connector](../../assets/blog/part-1-data-cloud-mulesoft-integration-7.png) Add a Connector Name like **MuleSoft Ingestion API** and click on **Save**. ![MuleSoft Ingestion API connector details showing Source API Name MuleSoft_Ingestion_API](../../assets/blog/part-1-data-cloud-mulesoft-integration-8.png) > [!IMPORTANT] > Take note of the **Source API Name**. This name will be used as the `sourceApiName` query parameter to call the Mule application. In this case, the correct value would be `MuleSoft_Ingestion_API`. If you already have a YAML schema, upload it here and take note of the name of the object(s). If not, upload the following example schema (the names of the objects are `runner_profiles` and `exercises`). ```yaml openapi: 3.0.3 components: schemas: runner_profiles: type: object properties: maid: type: number first_name: type: string last_name: type: string email: type: string gender: type: string city: type: string state: type: string created: type: string format: date exercises: type: object properties: runid: type: number datetime: type: string format: date-time km_run: type: number calories_burned: type: number duration_minutes: type: number maid: type: number type: type: string ``` ![Ingestion API schema listing the exercises and runner_profiles objects, connector In Use](../../assets/blog/part-1-data-cloud-mulesoft-integration-9.png) Click on **Save**. > [!IMPORTANT] > Whichever object you wish to insert/delete records to, will be used as the `objectName` query parameter to call the Mule application. In this case, the correct value(s) would be `runner_profiles` or `exercises`. ## Data Stream settings Exit the Setup page or go back to the main page after logging in to Salesforce. Click on the apps button at the top-left of the screen and search for **Data Cloud**. Click on the app. ![Salesforce App Launcher searching for and showing the Data Cloud app](../../assets/blog/part-1-data-cloud-mulesoft-integration-10.png) Select the **Data Streams** tab from the top. Click **New**. ![Data Cloud Data Streams tab with the New button to create a data stream](../../assets/blog/part-1-data-cloud-mulesoft-integration-11.png) Select **Ingestion API** and click **Next**. ![New Data Stream source picker with the Ingestion API tile selected](../../assets/blog/part-1-data-cloud-mulesoft-integration-12.png) Select your **MuleSoft Ingestion API** from the dropdown. Select all the objects and click on **Next**. - In the `exercises` configuration, select Category **Profile** and Primary Key `maid` - In the `runner_profiles` configuration, select Category **Profile** and Primary Key `maid` Click **Next**. Select the **default** Data Space and click **Deploy**. ![Two deployed data streams: MuleSoft Ingestion API-runner_profiles and -exercises](../../assets/blog/part-1-data-cloud-mulesoft-integration-13.png) If you used the example YAML schema, you should have ended up with something like the following. ![runner_profiles data stream fields table with maid set as the Primary Key](../../assets/blog/part-1-data-cloud-mulesoft-integration-14.png) > [!NOTE] > In this Data Stream, we selected `maid` as the Primary Key. This is the field we're going to need per record to delete them using the Mule app. ## Conclusion After finishing this configuration in Salesforce/Data Cloud, you should now have the following data that you will use later when calling the integration. | **Field** | **Value** | |---|---| | `salesforce.username` | The username you used to log in to Salesforce at [login.salesforce.com](http://login.salesforce.com) | | `salesforce.password` | The password you used to log in to Salesforce at [login.salesforce.com](http://login.salesforce.com) | | `cdp.consumer.key` | The Consumer Key you generated from the Connected App | | `cdp.consumer.secret` | The Consumer Key you generated from the Connected App | | `sourceApiName` | The Source API Name from the Ingestion API (`MuleSoft_Ingestion_API`) | | `objectName` | The objects created from the YAML schema in the Ingestion API (`runner_profiles` or `exercises`) | | Object ID / Primary Key | The field selected as Primary Key from the Data Stream settings (`maid`) | Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## How to add JVM/Command-line arguments to the Mule 4 Runtime in Anypoint Code Builder (ACB) Source: https://prostdev.com/post/how-to-add-jvm-command-line-arguments-to-the-mule-4-runtime-in-anypoint-code-builder-acb | Published: Jan 25, 2024 | Category: Tutorials I've received this question a lot lately, so I figured I'd create a quick post about it. Adding JVM or Command-line arguments to your Mule Runtime is especially useful when you're using properties files that are separated by environment. It is also useful to pass a secure/encryption key to your Mule application when you have secured properties. Let's learn how to pass these arguments in ACB. ## Prerequisites - **Visual Studio Code** - To use Anypoint Code Builder (Desktop), you must first install VS Code. You can download it [here](https://code.visualstudio.com/). - **Anypoint Extension Pack** - To enable Anypoint Code Builder in your Visual Studio Code, you must install this extension. You can install it from [here](https://marketplace.visualstudio.com/items?itemName=salesforce.mule-dx-extension-pack). - **REST Client** - A lot of people use [Postman](https://www.postman.com/). You can also use [curl](https://curl.se/) if you're more into the command line, or [Thunder Client for VS Code](https://marketplace.visualstudio.com/items?itemName=rangav.vscode-thunder-client). We'll use Thunder Client for this post. - **GitHub Repo** - If you want to follow along with the code I generated, you can check it out [here](https://github.com/ProstDev/args-in-acb). ## Create the Mule project You can use the GitHub repository listed in the prerequisites, or you can generate your code by following the next instructions. In ACB, click on **Develop an Integration** and give a name to the project. In our case, it will be test-project. Click on **Create project**. Paste the following code before the `` tag. ```xml ``` This will create an HTTP Listener and a Set Payload component in the flow. You will also create two Global Element Configurations: one for the HTTP Listener configuration and another for the properties, in this case, using the ${env}.properties syntax. > [!NOTE] > You can use .properties or .yaml extensions for your properties files. Create a new `dev.properties` file under `src/main/resources` and paste the following properties: ``` http.listener.host=0.0.0.0 http.listener.port=8081 ``` > [!NOTE] > You should have a different properties file per environment. For now, we are only creating the basic configuration for the dev environment to demonstrate. ## Add arguments to the runtime Once you have the basic setup to run the Mule app, we'll now add the env argument to the list. So, every time you run this Mule app, the env property will be provisioned to the runtime. Open the ACB Settings by clicking on the ⚙️ icon on the home screen. ![Anypoint Code Builder home screen with the gear icon tooltip reading "Open ACB Settings".](../../assets/blog/how-to-add-jvm-command-line-arguments-to-the-mule-4-runtime-in-anypoint-code-builder-acb-2.png) Go to **Mule > Runtime: Default Arguments** and append the following argument at the end of the value. ``` -M-Denv=dev ``` ![VS Code settings showing Mule Runtime Default Arguments with -M-Denv=dev appended to the value.](../../assets/blog/how-to-add-jvm-command-line-arguments-to-the-mule-4-runtime-in-anypoint-code-builder-acb-3.png) ## Run your app locally You can start your Mule app from the **Run and Debug** tab and click on the green run button. ![VS Code Run and Debug tab with the green "Debug Mule" run button to start the app.](../../assets/blog/how-to-add-jvm-command-line-arguments-to-the-mule-4-runtime-in-anypoint-code-builder-acb-4.png) Finally, call your Mule application from your REST Client with the following URL: ``` localhost:8081/hello ``` You should receive a "Hello World" response. ![Thunder Client GET to localhost:8081/hello returning 200 OK with a "Hello World" response.](../../assets/blog/how-to-add-jvm-command-line-arguments-to-the-mule-4-runtime-in-anypoint-code-builder-acb-5.png) Ta-da!! I hope that was helpful. Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## How to RE-scaffold Mule flows from an Exchange API specification in Anypoint Studio Source: https://prostdev.com/post/re-scaffold-mule-flows-exchange-api-spec-studio | Published: Dec 12, 2023 | Category: Tutorials What happens when there is a change to a published API in Anypoint Exchange? How can we get these changes re-scaffolded to our API implementation once we have already done the scaffolding? In this post, we'll learn how to re-scaffold our Mule project based on new changes done to the published API Specification in Exchange. ## Prerequisites - **Anypoint Platform** - You should have an Anypoint Platform account. You can create a new free trial account [here](https://anypoint.mulesoft.com/login/signup). - **API specification** - You should already have an API specification published in Anypoint Exchange. - **Anypoint Studio** - MuleSoft's IDE based on Eclipse. You can download it [here](https://www.mulesoft.com/lp/dl/anypoint-mule-studio). - **Scaffolded Mule project** - You should already have scaffolded the first version of the Mule project based on a previous version of the API specification since we're going to re-scaffold the same project with the new change in the specification. If you haven't done this yet, please refer to [part 1](https://www.prostdev.com/post/scaffold-mule-flows-from-published-api-specification) of this series. - **GitHub Repo** - If you want to follow along with the code I generated, you can check it out [here](https://github.com/ProstDev/codetober23/tree/main/day3). ## Update the API specification's dependency version Once our API Specification has been updated and published to Anypoint Exchange, our Mule project will become outdated and this is where we'll do the re-scaffolding to update the project to match the latest API specification version from Exchange. In your Package Explorer in Anypoint Studio (located at the left of the screen), you will be able to find the name of your API Specification with the first version (in this case v1.0.0). If you open the dependency, you'll be able to see the .zip associated with the specification. ![Studio Package Explorer showing the my-to-do-api v1.0.0 dependency and its RAML zip](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-2.png) We want to change this version of the specification to the next available version, which in our case is v2.0.0, but it could be v1.0.1 or any other version your specification has as the latest. To do this, right-click on your API specification's dependency and select the name of the spec (in this case **my-to-do-api**). After that, select **Update Version**. ![Right-click menu on the API dependency with the Update Version option highlighted](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-3.png) This will open a new window where you'll be able to refresh the APIs associated with the Mule Project by clicking the **Refresh** or **Update** button on the top-right side. ![Mule Project APIs settings listing my-to-do-api at version 1.0.0 with a refresh button](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-4.png) > [!IMPORTANT] > If nothing happens when you click this button, it might mean that your credentials have expired. You will need to refresh them from **Anypoint Studio > Settings > Anypoint Studio > Authentication**. Remove your current Anypoint Platform credentials from this window and re-add them. Try refreshing the APIs one more time after refreshing the credentials. Once the APIs are refreshed, you'll be able to see that there is a newer version of the API specification you're using in your project. ![Tooltip on the API version 1.0.0 reading "There is a newer version: 2.0.0"](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-5.png) Click on the **Update version** button for this API to be refreshed. ![API dependency detail showing "New version available" with an Update version button](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-6.png) Once it's refreshed, it will appear with the newest version. In this case, 2.0.0. Click **Apply and Close**. ![Properties APIs view with My To-Do API now at version 2.0.0 and an Apply button](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-7.png) Immediately after that, you will get a message asking if you want to scaffold the API specification. Select **Yes** to re-scaffold it. ![Dialog asking "Do you want to scaffold 'My To-Do API' specification?" with Yes and No](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-8.png) After that, your API specification's dependency will appear with the new version in the Package Explorer and you'll be able to see your new changes in the API implementation. ![Package Explorer showing the My To-Do API dependency updated to v2.0.0](../../assets/blog/re-scaffold-mule-flows-exchange-api-spec-studio-9.png) Subscribe to receive notifications as soon as new content is published ✨ 💬 Prost! 🍻 --- ## How to scaffold Mule flows from a published API spec in Anypoint Code Builder (ACB) Source: https://prostdev.com/post/scaffold-mule-flows-from-published-api-spec-acb | Published: Dec 7, 2023 | Category: Tutorials In this post, I am going to show you how to scaffold your flows in Anypoint Code Builder from a published API specification. > [!DOCS] > To do this same thing in Anypoint Studio, refer to [How to scaffold Mule flows from a published API spec in Anypoint Studio](https://www.prostdev.com/post/scaffold-mule-flows-from-published-api-specification) Why would you want to scaffold Mule flows from an API specification? This way you will be able to get started on your Mule application with a base project that is created upon your specification, instead of starting the Mule project and Mule flows from scratch. Once you scaffold the Mule project, you will have: - Mule flows for each HTTP method in your specification - Basic error handling for different HTTP status codes - Initial Mule project with some Transform Message components where applicable ## Prerequisites - **Anypoint Platform** - You should have an Anypoint Platform account. You can create a new free trial account [here](https://anypoint.mulesoft.com/login/signup). - **API specification** - You should already have an API specification published in Anypoint Exchange. - **Visual Studio Code** - To use Anypoint Code Builder, you must first install VS Code. You can download it [here](https://code.visualstudio.com/). - **Anypoint Extension Pack** - To enable Anypoint Code Builder in your Visual Studio Code, you must install this extension. You can install it from [here](https://marketplace.visualstudio.com/items?itemName=salesforce.mule-dx-extension-pack). - **GitHub Repo** - If you want to follow along with the code I generated, you can check it out [here](https://github.com/ProstDev/codetober23/tree/main/day2). > [!DOCS] > If this is your first time creating an API specification, refer to [How to use MuleSoft's visual API Designer to create a To-Do API specification using clicks, not code](https://www.prostdev.com/post/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code) ## Create a Mule project in ACB Once you are inside Anypoint Code Builder, select the **Implement an API** quick action. ![Anypoint Code Builder Quick Actions panel with the Implement an API option highlighted](../../assets/blog/scaffold-mule-flows-from-published-api-spec-acb-2.png) This will open a new menu or window, and it will ask you to sign in to Anypoint Platform to retrieve the API Specification from your account. Click **Allow** and follow the prompts to sign in. Once you are signed in, you will be able to search an API Specification from Exchange to implement it. Simply search for your asset's name on the search bar and select it from the list by clicking on **Add Asset**. ![New project form with name todoapp and an Exchange search listing the My To-Do API with Add Asset](../../assets/blog/scaffold-mule-flows-from-published-api-spec-acb-3.png) > [!NOTE] > If you're having issues searching for your asset, you can use the **Show filters** toggle to search for more specific assets. For example, using your own organization or the asset's type. After the API Specification has been selected, click on **Create Project**. Wait a few seconds for everything to finish processing and you'll have the finalized project after that. ## Navigate the generated project Once the project has been created, you should be able to see the flows in the configuration file located under `src/main/mule`. ![Scaffolded todoapp.xml shown as a flow canvas alongside its generated Mule XML](../../assets/blog/scaffold-mule-flows-from-published-api-spec-acb-4.png) You can click on the **Flow List** button from the top-left of the graphical view (or the flow canvas) to see a complete list of all the flows that have been implemented in the current XML file. ![Flow List panel listing the generated GET, POST, PUT, and DELETE flows for the to-do API](../../assets/blog/scaffold-mule-flows-from-published-api-spec-acb-5.png) The Error Handling has been added automatically to the main flow as well. This contains the HTTP status codes for some basic errors like 400 Bad Request or 404 Not Found. ![Auto-generated error handling with On Error Propagate blocks each routing to a Transform Message](../../assets/blog/scaffold-mule-flows-from-published-api-spec-acb-6.png) Notice how some of the flows already have a Transform Message component for you to see which DataWeave code is needed to retrieve some data like the URI Parameter(s). This, of course, is just in the case that your API Specification contains a URI Parameter. ![Transform Message in a GET flow with DataWeave setting a variable from attributes.uriParams.id](../../assets/blog/scaffold-mule-flows-from-published-api-spec-acb-7.png) There will also be Transform Message components added for some cases where you have set up an example in the API Specification. ![Transform Message with DataWeave returning a sample JSON to-do payload built from the spec example](../../assets/blog/scaffold-mule-flows-from-published-api-spec-acb-8.png) Pretty cool, right? Keep posted for some more articles on scaffolding in Anypoint Studio or Anypoint Code Builder! Remember to subscribe at the bottom of the page to receive email notifications as soon as new content is published. ✨ 💬 Prost! 🍻 --- ## How to scaffold Mule flows from a published API spec in Anypoint Studio Source: https://prostdev.com/post/scaffold-mule-flows-from-published-api-specification | Published: Nov 29, 2023 | Category: Tutorials In this post, I am going to show you how to scaffold your flows in Anypoint Studio from a published API specification. > [!DOCS] > To do this same thing in Anypoint Code Builder, refer to [How to scaffold Mule flows from a published API spec in Anypoint Code Builder (ACB)](https://www.prostdev.com/post/scaffold-mule-flows-from-published-api-spec-acb) Why would you want to scaffold Mule flows from an API specification? This way you will be able to get started on your Mule application with a base project that is created upon your specification, instead of starting the Mule project and Mule flows from scratch. Once you scaffold the Mule project, you will have: - Mule flows for each HTTP method in your specification - Basic error handling for different HTTP status codes - Initial Mule project with some Transform Message components where applicable ## Prerequisites - **Anypoint Platform** - You should have an Anypoint Platform account. You can create a new free trial account [here](https://anypoint.mulesoft.com/login/signup). - **API specification** - You should already have an API specification published in Anypoint Exchange. - **Anypoint Studio** - MuleSoft's IDE based on Eclipse. You can download it [here](https://www.mulesoft.com/lp/dl/anypoint-mule-studio). - **GitHub Repo** - If you want to follow along with the code I generated, you can check it out [here](https://github.com/ProstDev/codetober23/tree/main/day1). > [!DOCS] > If this is your first time creating an API specification, refer to [How to use MuleSoft's visual API Designer to create a To-Do API specification using clicks, not code](https://www.prostdev.com/post/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code) ## Create a Mule project in Studio First of all, let's open Anypoint Studio and click on **Create a Mule project** from the options on the left. ![Anypoint Studio Package Explorer with the "Create a Mule Project" link among project options](../../assets/blog/scaffold-mule-flows-from-published-api-specification-2.png) Add any name of your choice in **Project Name**. Click on the green ➕ button under **Import a published API** and select **from Exchange**. ![New project dialog: the add API button expanded to "from Exchange" or "from Maven"](../../assets/blog/scaffold-mule-flows-from-published-api-specification-3.png) Follow the prompts to authenticate to your Anypoint Platform account by clicking **Add Account** and adding your Username and Password. Once you are signed in, search for your API Specification, select it, and click on **Add >**. Once you selected it on the right-hand panel, click on **Finish**. ![Add Dependencies dialog with the To-Do API selected from Exchange before clicking Finish](../../assets/blog/scaffold-mule-flows-from-published-api-specification-4.png) After the API appears under **Import a published API**, you can click on **Finish** to close the window and see the generated project. ## Navigate the generated project Once the project has been created, you should be able to see the flows in the configuration file located under `src/main/mule`. ![Generated main flow with an HTTP Listener connected to an APIkit Router](../../assets/blog/scaffold-mule-flows-from-published-api-specification-5.png) The Error Handling has been added automatically to the main flow as well. This contains the HTTP status codes for some basic errors like 400 Bad Request or 404 Not Found. ![Auto-generated error handling with On Error Propagate scopes for BAD_REQUEST and NOT_FOUND](../../assets/blog/scaffold-mule-flows-from-published-api-specification-6.png) Notice how some of the flows already have a Transform Message component for you to see which DataWeave code is needed to retrieve some data like the URI Parameter(s). This, of course, is just in the case that your API Specification contains a URI Parameter. ![Generated get-by-id flow whose Transform Message reads attributes.uriParams.'id'](../../assets/blog/scaffold-mule-flows-from-published-api-specification-7.png) There will also be Transform Message components added for some cases where you have set up an example in the API Specification. ![Transform Message generated from an API example outputting sample JSON to-do data](../../assets/blog/scaffold-mule-flows-from-published-api-specification-8.png) Pretty cool, right? Keep posted for some more articles on scaffolding in Anypoint Studio or Anypoint Code Builder! Remember to subscribe at the bottom of the page to receive email notifications as soon as new content is published. Prost! --- ## #Codetober 2023 ~ 31 MuleSoft-related videos under 10min each! Source: https://prostdev.com/post/codetober-2023 | Published: Oct 3, 2023 | Category: News Here at [ProstDev](https://www.prostdev.com/), we dedicate ourselves to sharing knowledge with everyone who needs it. Because of that, every October we create this series of 31 videos ranging from various tech topics. > [!BUTTON] > [See YouTube Playlist](https://www.youtube.com/playlist?list=PLb61lESgk6hhKwJpM565UJVS5CxVZ4D1u) That's right...we'll publish ONE VIDEO A DAY! This year, the videos will be edited for your better digestion and will be less than 10 minutes each (hopefully). All of the scheduled videos will be listed below. If you can't access some of them yet, it is because they haven't been released as of now. Each video will be published at 12:00 a.m. ET. You can **subscribe** to our [YouTube channel](https://www.youtube.com/prostdev) to get notifications as soon as we publish new videos! Please comment on our videos to suggest future content or ask any questions about a specific topic (discussed in the video). Happy [#Codetober](https://www.prostdev.com/blog/hashtags/Codetober)! > [!TIP] > Find this year's GitHub repository here: [ProstDev/codetober23](https://github.com/ProstDev/codetober23) Remember to subscribe to our [YouTube channel](https://www.youtube.com/prostdev) to get notifications as soon as we publish new videos! Prost! --- ## How to use MuleSoft's visual API Designer to create a To-Do API specification using clicks, not code Source: https://prostdev.com/post/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code | Published: Sep 26, 2023 | Category: Tutorials Did you know MuleSoft provides this neat Graphical User Interface (GUI) where you can configure your own API specification with just clicks? That's right, you don't have to learn RAML or OAS anymore. You can just follow this visual tool to create them and it gives you the code you need in whichever format you need it! Let's create a To-Do API specification to see how this tool works. ## Prerequisites Before starting, create a new free trial account on [Anypoint Platform](https://anypoint.mulesoft.com/login/signup). Once you are logged in, Navigate to **Design Center**. You can access it using the left-hand menu or by clicking **Start designing** from the welcome screen. ## 1. Create a new API specification Inside Design Center, click on the **Create +** button and select **New API Specification**. Name your project however you prefer and make sure you select the **Guide me through it** option to use the visual tool. Click on **Create API**. ![New API specification dialog naming the project "My To-Do API" with the "Guide me through it" option selected.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-2.png) On the first screen, we'll see the API Summary. Add the following fields: ![API Summary with Title "My To-Do API", Version 1.0.0, and Media type set to application/json.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-3.png) ## 2. Create a data type Click on the plus (**+**) button next to the **DATA TYPES** tab on the left. Change the name to **To-Do** and the Type to **Object**. Next, click on the **Add Property** button and create the following properties for this object: > [!TIP] > Click on the Details (**v**) button to see the additional configurations for each property. ![To-Do object properties: id (Number), title (String), checked (Boolean, default False), dueDate (Date-Only).](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-4.png) We are changing the default value for the *checked* property from *True* to *False* because normally when we create a new To-Do, it means that it hasn't been completed :-) ## 3. Create a /todos resource Click on the plus (**+**) button next to the **RESOURCES** tab on the left. Name the resource path **/todos**. ### 3.1. Read all To-Dos Under **GET > Responses** click **Add New Response** and leave the **200 - OK** option selected. Click on **Add Body** and change the type to **Array**. Open the details of the array and select Items Type **To-Do**. ![GET on /todos with a 200 OK response whose body is an Array of To-Do items.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-5.png) In other words, this means that whenever we send a GET request to the /todos resource (or path), we'll receive a 200-OK response with a list of our existing To-Dos. ### 3.2. Add Query Parameters Since we will be receiving a list of all the To-Dos, perhaps we could make things easier for the user by providing some filters for this array. We can do this by adding some Query Parameters to the GET method we just created. For this, go to **GET > Query Parameters** and click on **Add Query Parameter**. Add the following parameters: ![GET on /todos with four optional query parameters: titleContains, isChecked, fromDate, toDate.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-6.png) As you can see, these are all optional parameters. We do this by unchecking the Required option. Now the user can choose to either use these filters or not to retrieve a list of To-Dos. ### 3.3. Create a new To-Do To create a new To-Do in our design, we have decided to use the POST method under the /todos resource. For this, navigate to **POST > Responses** and click on **Add New Response**. You could leave this as 200 if you wanted to, but let's spice things up a bit by selecting **201 - Created** instead. You know, just for fun. Now click on **Add Body** and change the type to **To-Do**. ![POST on /todos with a 201 Created response whose body type is To-Do.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-7.png) This is not all! Now head to the **Body** tab right next to the Responses tab that we're in right now and click on **Add Body**. Change this type to **To-Do** as well. ![POST on /todos Body tab with one request body of type To-Do.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-8.png) This means that when the user wants to create a new To-Do, they'll have to send in a To-Do in the body of the request and they will receive a 201-Created with the details of the To-Do that was just created. > [!NOTE] > **❓ Question**: Can you see a potential problem for us to be using the To-Do data type in the creation of a To-Do? We'll discuss this at the end of the post :) ## 4. Create a /todos/{id} resource Click on the plus (**+**) button next to the **RESOURCES** tab on the left. Name the resource path **/todos/{id}**. > [!TIP] > You can also click on the plus button next to the /todos resource to automatically create a nested resource. ### 4.1. Add a URI Parameter Click on the **URI Parameters** button next to the resource path and select **Add URI Parameter**. Add the following values: ![Single to-do resource with a required URI parameter named id of type Number.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-9.png) This *id* will act as a sort of variable for the real value that will be passed in the URI. For example, to refer to the To-Do with id 1, you can call /todos/1. ### 4.2. Read a To-Do Under **GET > Responses** click **Add New Response** and leave the **200 - OK** option selected. Click on **Add Body** and change the type to **To-Do**. ![GET on a single to-do resource with a 200 OK response whose body type is To-Do.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-10.png) In this case, we are not creating any Query Parameters since we are solely returning one To-Do. The one that matches the id from the URI Parameter in the path. ### 4.3. Update a To-Do Under **PUT > Responses** click **Add New Response** and leave the **200 - OK** option selected. Click on **Add Body** and change the type to **To-Do**. ![PUT on a single to-do resource with a 200 OK response whose body type is To-Do.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-11.png) Now head to the **Body** tab right next to the Responses tab that we're in right now and click on **Add Body**. Change this type to **To-Do** as well. ![PUT on a single to-do resource, Body tab with one request body of type To-Do.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-12.png) This is awfully similar to the configurations we did for the POST method in the /todos resource, right? Honestly, we could've chosen to use POST here as well instead of PUT. It really depends on how you prefer to design your API specification. It's all very subjective sometimes! But I have an article for you on why I prefer to use PUT here instead of POST. Check it out 👇 > [!DOCS] > **Article**: [Understanding APIs (Part 3): What are HTTP Methods?](https://www.prostdev.com/post/understanding-apis-part-3-what-are-http-methods) ### 4.4. Delete a To-Do Under **DELETE > Responses** click **Add New Response** and select the **204 - No content** option. We won't add a body here in either the response or the request body. This is because we are already retrieving the id of the To-Do we want to delete (from the URI Parameter) and once we remove it, we won't have to return this information back in the response's body. ![DELETE on a single to-do resource with a 204 No Content response and no body.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-13.png) If you're curious about why I'm not just choosing 200-OK for all the status responses, you can check out this post I wrote about HTTP Status Codes 👇 > [!DOCS] > **Article**: [Understanding APIs (Part 6): What are HTTP Status Codes?](https://www.prostdev.com/post/understanding-apis-part-6-what-are-http-status-codes) ## Conclusion And that's all!! That's how you create an API specification for a To-Do app. I don't know if you've been noticing that there's some code being updated on the right-hand side of the screen. If you look at the bottom of that section, you'll notice there are two tabs: one that says RAML and one that says OAS. You can totally use this tool to generate a base of your RAML/OAS code so you don't have to start from scratch! Please note that there's an option to edit the code directly from the API Designer, however, if you do that, you'll lose the capability to come back to the GUI. If you intend to use this tool only to generate the code, I would recommend you download it and open it in a new API spec project instead of modifying it from here. Just in case you need this GUI in the future for reference :-) ## Answer to the question There was a question I dropped somewhere in the middle: - Can you see a potential problem for us to be using the To-Do data type in the creation of a To-Do? The answer is that because we created the To-Do data type to have a required property called *id,* and since we are using this data type to create a new To-Do, the id property will be required in the body of the request. For example, if we were to create a new To-Do, our request body would have to look like this: ```json { "id": 1, "title": "Example", "checked": false, "dueDate": "2023-09-25" } ``` When in reality, we wouldn't know what the appropriate id would be for this new To-Do. One should have to be assigned from the backend, the user shouldn't have to provide it. A solution. to this would be to go back to the POST method in the /todos resource and change the request body's data type from To-Do to Object. Then, we would have to manually add the rest of the properties here (title, checked, dueDate), which is not ideal. ![POST /todos body changed to an Object with manually re-added title, checked, and dueDate properties.](../../assets/blog/how-to-use-mulesoft-s-visual-api-designer-to-create-a-to-do-api-specification-using-clicks-not-code-14.png) If our initial To-Do data type changes, we would have to change the structure from both places. This will eventually lead to human error and there is no reusing in place for this. This is why the GUI is very helpful in generating an initial base of our API specification, but it currently doesn't hold all the answers to create the best RAML/OAS code. I still use it pretty much all the time to get started, though. Tools are exactly that: tools. They are meant to help you in your work. But they don't hold all the answers. We do :-) Prost! --- ## DataWeave scripts to clean your XML/HTML code snippets for a WordPress blog post Source: https://prostdev.com/post/dataweave-scripts-to-clean-your-xml-html-code-snippets-for-a-wordpress-blog-post | Published: Sep 5, 2023 | Category: Tutorials In case you're not familiar with my [dataweave-scripts](https://github.com/alexandramartinez/dataweave-scripts) GitHub repo, it's the place where I keep some of the scripts I've created to help the community with transformation questions or simply some scripts that have been handy to me. In this post, I want to introduce you to two transformations I added because of a use case I came up with last week. Basically to help clean an XML or HTML to publish a script in a WordPress article. ## The problem This problem started because I had written a blog post in a WordPress-based blog. I was sharing a Maven snippet (XML format). The issue is that WordPress mistook the XML tags as HTML code. So, instead of having a regular XML snippet, the article was showing something like this: ![Maven XML snippet in WordPress with its tags stripped out, leaving only the bare values](../../assets/blog/dataweave-scripts-to-clean-your-xml-html-code-snippets-for-a-wordpress-blog-post-2.png) The fix was simple. Instead of having the regular `<` and `>` characters pasted in the code snippet, I had to use `<` and `>` respectively. *(Thanks so much Julian Duque for providing the fix! I had no idea about this issue in WordPress* 🤗*)* For example, instead of writing ``, I had to replace it with `<plugin>` I thought to myself: If I need to keep doing this for future blog posts, maybe I can create a DataWeave transformation to fix this for me so I can just easily copy and paste the new clean snippet. These are the two approaches I came up with. ## First approach: XML input The first thing I tried to do since I was using an XML format for the script, was to take an input XML format, transform it to a String, and then clean the text. This is the script I came up with: ```dataweave %dw 2.0 output text/plain --- write(payload,"application/xml") replace "\n" with "" replace "<" with "<" replace ">" with ">" ``` > [!PLAYGROUND] > [Open in the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-scripts&path=functions%2Fclean-xml) However, I quickly ran into issues when I tried to clean an HTML code snippet using this same transformation. This is how I came up with the second approach. ## Second approach: plain text input This time I decided to use a plain text input instead of an XML input format. This way, both XML and HTML code snippets could be used as the input and I wouldn't need to use the `write()` function in the first place. ```dataweave %dw 2.0 output text/plain --- payload replace "<" with "<" replace ">" with ">" ``` > [!PLAYGROUND] > [Open in the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-scripts&path=functions%2Fclean-html) Plus, I got rid of one `replace()` because I no longer needed to remove the XML header. It's a short post, but I hope it's insightful for you all 🤗 I'm sure I'll keep using this example in the Playground to modify my WordPress posts in the future. Let me know if you've faced similar issues with WordPress before! --- ## Down the Rabbit Hole vol.2 - GUIs in MuleSoft Source: https://prostdev.com/post/down-the-rabbit-hole-vol-2-guis-in-mulesoft | Published: Aug 8, 2023 | Category: Opinion *GUIs, you love em or you hate em.* Software these days often gets shipped with a fancy user interface, though many developer tools are still CLI based. We devs seem to love the CLI. It is powerful and scares away the annoying managers and *The Business*, but most importantly: it works nicely with our scripts, builds, and pipelines! We do use GUIs though and cannot live without them, think of your APIKit Console, or Postman. Yes, once your collections are done you’ll end up running them with CLI variant Newman, but that’s for some other blog. No, What I want to cover today is GUIs in your integrations. 99/100 times this won’t be needed, but we’ll discuss those edge cases where a GUI might solve your problems. GUIs are there to help and visualize, but if you ask me there is a gap in the development lifecycle where we’re not used to providing them. We as developers are content to work with the console, we print DEBUG statements and ERRORs and can solve any problem with that. But that’s technical operations, what about the functional operations? Do I want to give the technical logs to the functional maintainer? Will they understand, or will they even check those logs regularly? As a developer I don’t mind checking if a batch has run, or which records failed, but if those questions arise regularly that quickly becomes a time sink. Now let's assume our business user is as non-technical as they get, they don’t want to see logs, even if we’ve limited those to the notifications intended for them. We can hook up an in-flow SMTP connector and send them a mail, or use a platform alert, but that might not be a good way of sharing the information they want. Let's discuss an alternative and some examples of that. I promise you, a well-designed interface can make your life as a Dev/Ops engineer more relaxed, and keep your focus on those tasks that actually matter. The first Mule GUI I ever made was years ago and used the static resource handler to display a simple HTML form, if you’re interested look at my horrible project here:[Mule Hackathon — RealLifeExploration Game](https://dev.to/wds444/real-life-exploration-game-powered-by-mulesoft-3bl6). It’s a mess of code and was made under a time crunch without all the dedicated software and services a real company has. I didn’t have a web server, so Mule had to do it. Same for the app logic, or the “Database”. Which was just a CSV file getting updated whenever an action happened. So far, I’ve only had to **work** with Mule GUIs four or five times. Only one of these was an actual customer integration, but two of them were deployed in production. I’m pretty sure you’ve even used one, maybe even in your CloudHub production VPC… Skeptical? I am of course talking about Professional service’s[net-tools API](https://github.com/mulesoft-consulting/net-tools-api)! ![net-tools Network Connectivity Tools web UI with a curl form and DNS console output](../../assets/blog/down-the-rabbit-hole-vol-2-guis-in-mulesoft-2.png) In the tried and trusted platform network testing toolkit, a client needs some connectivity tested? Use net-tools and curl to check if it’s working. Mule can’t resolve a Hostname? Use net-tools to check if the DNS server is available. > [!TIP] > Make sure to remove it once done, and don’t expose it to the internet for too long, if someone can find it they have some powerful tools available to check out your network. It offers some nice features, but what I'm more interested in is the code! ![DataWeave script serving static web files from classpath based on the requested URI path](../../assets/blog/down-the-rabbit-hole-vol-2-guis-in-mulesoft-3.png) The net-tools project has some iterations out there on the internet, but here we see the engine of this version’s ‘webserver’ any URL requested gets parsed and cross-referenced with a file in the web directory. That directory contains an index.html file and any necessary .js, .css, or other resources. The HTTP>Load Static Resource handler can do roughly the same but the result is that mule can serve a webpage on a chosen URL and port. These flows can use security filters, like net-tools is doing, or even be secured by API-Manager Policies and thus be available to any user you choose. In some of these projects, we took it a bit further, and we created specific Object stores for metadata, statistics, and even half-transformed payloads. Using Ajax in the HTML page can easily change this static display into a page that can dynamically display whatever info you choose to supply it from the endpoint. Now, we’ve discussed the HOW, now let’s look at the WHY. A practical example was a gigantic project where we had non-technical business users that wanted live insight into the tests they were doing. We’re talking about a few developers working on the APIs and dozens actively bombarding the system with tests. We could not handle the volume and logging all payloads was not feasible. The questions they had often had us look into the payload in different stages of processing. For reference, we’re talking input, JSON parsed input, two (hopefully identical) JSON responses, and a marshaled output. These messages were hundreds, sometimes thousands of lines long, and while we were not versed in reading them, they were. Had something like Splunk or ELK been available we could have used that, but it was not possible within the limitations. What we decided to do, at least during the development phase that lasted for over a year, was to store all these partially processed payloads into dedicated Object Stores with the correlation ID as their key. A five-minute job created a few endpoints that would return this data in text format. The interface from the net-tools API was *borrowed* and expanded with a few extra textboxes and UI elements. The dropdown box now used AJAX to populate the choices of correlation IDs and Ajax would GET these different payload values using the dedicated API endpoints. Construction of this interface from concept to deployment took less than a day, and enabled Functional Operations to do their work without interference from us! A few iterations of feedback later we would include some fancy CSS to mark those parts of the payload response that were not identical which was not too difficult either. Another feature was that we added a little metric stating the percentage of ‘correct’ requests which gave the entire functional team a great insight into the effectiveness and completion of the entire project. (parsing and processing 2000+ separate messages). The Business was extremely happy with this, as their work now became magnitudes easier. What started as a slight sidetrack to prevent a small annoyance is now used daily and really helps to show the total project progress! Another small example: The business was used to providing a monthly CSV file, but the FTP server they did that on had to go. A quick HTML page later that allowed them to upload that CSV file instead, to be loaded straight into the application logic solved all the requirements! (Yeah, I did not recommend this solution architecturally, but we were limited in the accepted options…) To sum up the post above: GUIs in Mule are really easy and can help you display data straight from the source. From testing while the project is in development, to easily displaying metadata or debug info to less than technical users. Even updating data or logic is possible, just make sure you properly think of the access controls and who you're providing them to. And don’t forget the most important rule: **Sanitize and verify any input!** ![My first ever Mule GUI…](../../assets/blog/down-the-rabbit-hole-vol-2-guis-in-mulesoft-4.png) --- ## DataWeave programming challenge #8: Sum all digits to get a 1-digit number Source: https://prostdev.com/post/dataweave-programming-challenge-8 | Published: Jul 11, 2023 | Category: Challenges Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F8) ## Input Consider the following input payload (can be of **txt** format): ``` 2456 2 25235 2456 6 54 58795 3 456 76544 0 ``` ## Explanation of the problem Create a DataWeave script to sum all the digits from the input payload until there's only one digit left. For example, - 123 + 456 = 579 - 579 is broken down into 5 + 7 + 9 = 21 - 21 is broken down into 2 + 1 = 3 - The final result is 3 because it's a 1-digit number ## Expected output In this case, the expected output would be: ``` 2 ``` ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 You can first sum each line from the input payload to end up with just one number. ### Clue #2 Use sizeOf() to get the length of a String. ### Clue #3 You can use reduce() to sum some of the digits. ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find some solutions we provide here so you can compare your result with us. ### Solution #1 ```dataweave %dw 2.0 import lines from dw::core::Strings output application/json fun sumNums(num, result=0) = do { var new = (num reduce $+$$) as String --- if (sizeOf(new) > 1) sumNums(new, result+new) else new } --- sumNums(sum(lines(payload))) as Number ``` ### Solution #2 ```dataweave %dw 2.0 import lines from dw::core::Strings output application/json fun sumNums(num, result=0) = do { var new = (num reduce $+$$) as String --- if (sizeOf(new) > 1) sumNums(new, result+new) else new } --- payload replace "\n" with "" then sumNums($) as Number ``` Subscribe to receive notifications as soon as new content is published ✨ --- ## We are now accepting guest writers! Source: https://prostdev.com/post/we-are-now-accepting-guest-writers | Published: Jun 28, 2023 | Category: News Hi ProstDev community! ✨ 🤗 We are delighted to announce an exciting update to our blog's article submission process! We've made significant improvements to enhance the experience for everyone involved and ensure the delivery of exceptional content to our valued readers. 📚 While our past contributions have predominantly focused on MuleSoft or DataWeave, it's important to note that our blog caters to a wide range of tech topics. Whether you have expertise in any technology, this platform welcomes your valuable insights and contributions! 🤓 We take immense pride in offering personalized 1:1 content reviews. Even if you're new to the world of blogging, rest assured that our dedicated team is here to support you every step of the way. Feel free to reach out with any inquiries or submit your article to initiate the review process. 🧐 Curious about our review process? It begins by acquainting yourself with our [guidelines](https://www.prostdev.com/post/guidelines-to-submit-your-content-in-prostdev) and [submitting](https://github.com/ProstDev/blog-posts/issues/new/choose) an article proposal. Once you've done that, our amazing team jumps in and makes some tweaks or suggestions to make your article more readable, understandable, and engaging for your audience. ⏳ Remember, practice makes perfect. No one starts out as an expert writer. We've all been there, taking those first steps. Our goal is to be your mentors and guide you along the way, helping you get comfortable with writing articles so you can rock it on your own! 📱 But wait, there's more! We don't just assist with your content. We've got a ready-to-learn fan base eager to dive into more technologies with your help! We'll take care of the article formatting, scheduling, and promotion stuff so that you can put all your focus on creating killer content. We can't wait to see what you'll [submit](https://github.com/ProstDev/blog-posts/issues/new/choose)! ✨ - The ProstDev team --- ## DataWeave programming challenge #7: Modify certain values from a JSON structure Source: https://prostdev.com/post/dataweave-programming-challenge-7 | Published: Jun 13, 2023 | Category: Challenges Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F7) > [!TIP] > Instead of trying to write the whole code yourself, browse in the documentation for any function that would do the work for you 👀 ## Input Consider the following JSON input payload: ``` { "name": "a", "object": { "name": "b", "l": [ { "other": "c", "list": [ ] }, { "thisname": "def", "list": [ { "this": "500e", "l": [ { "finalname": "f" }, { "finalname": "ghijk" } ] } ] } ] }, "array": [ { "thisname": "h" }, { "xyz": "abc123" } ] } ``` ## Explanation of the problem Create a DataWeave script that will update all the values to uppercase, **except** the ones in which the field equals **thisname**. For example, the first field **name** with the value **"a"** has to be transformed to **"A"**. However, the field **thisname** with the value of **"def"** should stay the same. ## Expected output In this case, the expected output would be: ``` { "name": "A", "object": { "name": "B", "l": [ { "other": "C", "list": [ ] }, { "thisname": "def", "list": [ { "this": "500E", "l": [ { "finalname": "F" }, { "finalname": "GHIJK" } ] } ] } ] }, "array": [ { "thisname": "h" }, { "xyz": "ABC123" } ] } ``` ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 Check out the [Tree](https://docs.mulesoft.com/dataweave/latest/dw-tree) module to see if any of those functions is useful for this case. ### Clue #2 Check out the [mapLeafValues](https://docs.mulesoft.com/dataweave/latest/dw-tree-functions-mapleafvalues) function. ### Clue #3 Use the [upper](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-upper) function to transform the strings to uppercase. ### Clue #4 Use **path[-1].selector** to retrieve the name of each value's field and compare it with a given string value. ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find here some solutions we are providing so you can compare your result with us. ### Solution #1 ```dataweave %dw 2.0 import mapLeafValues from dw::util::Tree output application/json --- payload mapLeafValues ((value, path) -> if (path[-1].selector == "thisname") value else upper(value) ) ``` ### Solution #2 ```dataweave %dw 2.0 import mapLeafValues from dw::util::Tree output application/json --- payload mapLeafValues if ($$[-1].selector == "thisname") $ else upper($) ``` Subscribe to receive notifications as soon as new content is published ✨ --- ## Part 5: CI/CD pipeline with MuleSoft and GitHub Actions - Enabling MFA through a Connected App Source: https://prostdev.com/post/part-5-ci-cd-pipeline-with-mulesoft-and-github-actions-enabling-mfa-through-a-connected-app | Published: May 16, 2023 | Category: Tutorials So far we’ve been setting up our CI/CD pipelines using our Anypoint Platform username and password. However, if you’re using an enterprise account, most likely you’re using MFA or Multi-Factor Authentication for your account. The process to create your CI/CD pipeline with this authentication method is quite different. If you haven’t been following the series or you’re not familiar with GitHub Actions, we recommend you start from the [first article](https://www.prostdev.com/post/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions) to understand how we are setting up all the configurations we need. In this post, we’ll learn how to set up our pipeline with a connected app, which is useful when you have multi-factor authentication activated in Anypoint Platform. ## Prerequisites You should already understand the basic CI/CD setup we’ve been doing in the previous articles. In summary, this is what you should already know: - How to configure and run the `build.yml` file for the pipeline under `.github/workflows`. - How to configure secrets in your GitHub repository. > [!NOTE] > The code for this article is **not** located on the **main** branch of the `github-actions` repository. Instead, a new branch was created to avoid confusion: [connected-app](https://github.com/alexandramartinez/github-actions/tree/connected-app). If you want to check the rest of the files we discuss here, like pom or build, please refer to that branch. ## Create a connected app in Anypoint Platform Because we are signing in to Anypoint Platform using MFA (through an app on your phone or an SMS code, for example), we can no longer rely on just our username and password. We have to create something called Connected App. To do this, sign in to [Anypoint Platform](https://anypoint.mulesoft.com/) and navigate to **Access Management** > **Connected Apps**. ![Access Management Connected Apps page with the Create app button and no apps yet](../../assets/blog/part-5-ci-cd-pipeline-with-mulesoft-and-github-actions-enabling-mfa-through-a-connected-app-2.png) Click on **Create app**. Add any name you want to identify this app, like `github-actions`. Select **App acts on its own behalf** and click on **Add Scopes**. Select the following scopes for your current business group and the **Sandbox** environment (since this is the one we’re using for this demo): - Design Center Developer - View Environment - View Organization - Profile - Cloudhub Organization Admin - Create Applications - Delete Applications - Download Applications - Read Applications - Read Servers ![Connected App scope selection listing Design Center, General, OpenID and Runtime Manager scopes](../../assets/blog/part-5-ci-cd-pipeline-with-mulesoft-and-github-actions-enabling-mfa-through-a-connected-app-3.png) Click on **Save**. After you create the app, make sure to copy both **ID** and **Secret**. We will use these in the pipeline for our authentication method. ![The created github-actions connected app with Copy Id and Copy Secret buttons](../../assets/blog/part-5-ci-cd-pipeline-with-mulesoft-and-github-actions-enabling-mfa-through-a-connected-app-4.png) In this case, for demonstration purposes, these are the credentials I’ll be using: ``` ID: bf51f105b644471f812b2e0c0cb8a97b Secret: 66B5ADCB18D4439D9C744236e3c590d3 ``` ## Set up your credentials on GitHub Just as we’ve done before, go to your GitHub repository and click on the **Settings** tab. Select **Secrets and variables** > **Actions** and add these two new secrets. - `CONNECTED_APP_CLIENT_ID` - `CONNECTED_APP_CLIENT_SECRET` The values should match what you previously extracted from Anypoint Platform. ## Modify your pom.xml In our `pom.xml` file, in the `org.mule.tools.maven` plugin, we used to have something like the following to authenticate via username and password. ```xml ... ${anypoint.username} ${anypoint.password} ``` We are going to replace the `username` and `password` fields with the following. ```xml ${client.id} ${client.secret} client_credentials ``` Here's the complete `pom.xml` for this example: ```xml 4.0.0 com.mycompany github-actions 1.0.0-SNAPSHOT mule-application github-actions UTF-8 UTF-8 4.4.0 3.8.0 amartinez17-github-actions Sandbox 2.3.13 org.apache.maven.plugins maven-clean-plugin 3.0.0 org.mule.tools.maven mule-maven-plugin ${mule.maven.plugin.version} true https://anypoint.mulesoft.com ${app.runtime} ${client.id} ${client.secret} client_credentials ${app.name} ${env} MICRO us-east-2 1 true ${decryption.key} mule-application com.mulesoft.munit.tools munit-maven-plugin ${munit.version} test test test coverage-report ${decryption.key} true true 90 console sonar json html org.mule.connectors mule-http-connector 1.6.0 mule-plugin org.mule.connectors mule-sockets-connector 1.2.2 mule-plugin com.mulesoft.modules mule-secure-configuration-property-module 1.2.5 mule-plugin com.mulesoft.munit munit-runner 2.3.13 mule-plugin test com.mulesoft.munit munit-tools 2.3.13 mule-plugin test org.mule.weave assertions 1.0.2 test anypoint-exchange-v3 Anypoint Exchange https://maven.anypoint.mulesoft.com/api/v3/maven default mulesoft-releases MuleSoft Releases Repository https://repository.mulesoft.org/releases/ default mulesoft-releases MuleSoft Releases Repository default https://repository.mulesoft.org/releases/ false ``` That’s it for this file! Now let’s set up the pipeline to send these new credentials. ## Modify your build.yml Open the `build.yml` file we created inside `.github/workflows`. If you haven’t created this file yet, please refer to the [first article](https://www.prostdev.com/post/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions) to learn how to set it up. Go to the `deploy` job and locate the last step: **Deploy to Sandbox**. This is what we used to have under `env`: ```yaml USERNAME: ${{ secrets.anypoint_platform_username }} PASSWORD: ${{ secrets.anypoint_platform_password }} ``` And this is what we’ll replace it with: ```yaml ID: ${{ secrets.CONNECTED_APP_CLIENT_ID }} SECRET: ${{ secrets.CONNECTED_APP_CLIENT_SECRET }} ``` We’ll change the Maven command to match these new secrets/properties. Instead of sending username and password, like this: ```bash -Danypoint.username="$USERNAME" \ -Danypoint.password="$PASSWORD" \ ``` We’ll now send the app’s ID and secret. Like this: ```bash -Dclient.id="$ID" \ -Dclient.secret="$SECRET" \ ``` Here's the complete `build.yml` for this example: ```yaml name: Build and Deploy to Sandbox on: push: branches: [ main ] jobs: test: runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - name: Set up JDK 1.8 uses: actions/setup-java@v3 with: distribution: 'zulu' java-version: 8 - name: Test with Maven env: nexus_username: ${{ secrets.nexus_username }} nexus_password: ${{ secrets.nexus_password }} KEY: ${{ secrets.decryption_key }} run: mvn test --settings .maven/settings.xml -Dsecure.key="$KEY" - name: Upload MUnit reports uses: actions/upload-artifact@v3 with: name: munit-test-reports path: target/site/munit/coverage/ build: needs: test runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - name: Set up JDK 1.8 uses: actions/setup-java@v3 with: distribution: 'zulu' java-version: 8 - name: Build with Maven run: mvn -B package --file pom.xml -DskipMunitTests - name: Stamp artifact file name with commit hash run: | artifactName1=$(ls target/*.jar | head -1) commitHash=$(git rev-parse --short "$GITHUB_SHA") artifactName2=$(ls target/*.jar | head -1 | sed "s/.jar/-$commitHash.jar/g") mv $artifactName1 $artifactName2 - name: Upload artifact uses: actions/upload-artifact@v3 with: name: artifacts path: target/*.jar deploy: needs: build runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - uses: actions/download-artifact@v3 with: name: artifacts - name: Deploy to Sandbox env: ID: ${{ secrets.CONNECTED_APP_CLIENT_ID }} SECRET: ${{ secrets.CONNECTED_APP_CLIENT_SECRET }} KEY: ${{ secrets.decryption_key }} run: | artifactName=$(ls *.jar | head -1) mvn deploy -DskipMunitTests -DmuleDeploy \ -Dmule.artifact=$artifactName \ -Dclient.id="$ID" \ -Dclient.secret="$SECRET" \ -Ddecryption.key="$KEY" ``` ## Run the pipeline That’s it! Once you’re done with the changes, simply push a new change to the **main** branch and this will trigger the pipeline. ![GitHub Actions run with test, build and deploy jobs all succeeded after the push](../../assets/blog/part-5-ci-cd-pipeline-with-mulesoft-and-github-actions-enabling-mfa-through-a-connected-app-5.png) ## More resources You can check out my [GitHub profile](https://github.com/alexandramartinez) for more CI/CD repos: - [github-actions](https://github.com/alexandramartinez/github-actions) to deploy a Mule app to CloudHub - [dataweave-utilities-library](https://github.com/alexandramartinez/dataweave-utilities-library) to publish a DataWeave library to Exchange - [api-catalog-cli-example](https://github.com/alexandramartinez/api-catalog-cli-example) to update APIs in Exchange using the API Catalog CLI I hope this was helpful! Don't forget to subscribe so you don't miss any future content. --- ## DataWeave programming challenge #6: Using tail-recursion to get the factorial of a number Source: https://prostdev.com/post/dataweave-programming-challenge-6 | Published: May 9, 2023 | Category: Challenges Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F6) ## Input Consider the following input payload (can be of **txt** format): ``` 3 5 -6 300 ``` ## Explanation of the problem - Create a [tail-recursive function](https://youtu.be/3UbTlLVNrVE) to get the [factorial](https://en.wikipedia.org/wiki/Factorial) of each **positive** number from the payload. - Sum the results. - Retrieve the digits/characters located at positions 20-25. - Make sure to return a number and not a string. - Numbers 0 or less won't be calculated. For example: - The factorial of 3 is 6 (3 x 2 x 1) - The factorial of 5 is 120 (5 x 4 x 3 x 2 x 1) - The factorial of -6 won't be calculated so the result will be 0 - And so on - Once the results are summed (6 + 120 + 0...), extract the digits at indexes 20 to 25. - Return this last 6-digit number. ## Expected output In this case, the expected output would be: ``` 537046 ``` ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 Use lines() from the Strings module or splitBy() to process every line. ### Clue #2 Make sure to use tail-recursiveness and not regular recursiveness. Otherwise, your script might not work. ### Clue #3 Make sure to create an if statement before calling the function in recursion. Otherwise, you might get stuck in an infinite loop! ### Clue #4 You can use the keyword `as` to transform to/from strings and numbers. ### Clue #5 You can extract characters from a string with the range selector [ x to y ] ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find here one of my solutions! ### Solution #1 ```dataweave %dw 2.0 import lines from dw::core::Strings output application/json fun factorial(num:Number, result:Number=1):Number = do { if (num == 0) result else if (num < 0) 0 else factorial(num - 1, result * num) } --- lines(payload) map factorial($) then sum($) as String then $[20 to 25] as Number ``` Subscribe to receive notifications as soon as new content is published ✨ --- ## Part 4: CI/CD pipeline with MuleSoft and GitHub Actions - MUnit minimum coverage percentage Source: https://prostdev.com/post/part-4-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-minimum-coverage-percentage | Published: May 2, 2023 | Category: Tutorials In the [previous article](https://www.prostdev.com/post/part-3-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-testing), we learned how to set up our Nexus credentials to be able to run the MUnits from the CI/CD pipeline. If you haven’t been following the series or you’re not familiar with GitHub Actions, we recommend you start from the [first article](https://www.prostdev.com/post/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions) to understand how we are setting up all the configurations we need. In this post, we’ll see the steps to add the minimum coverage percentage for the MUnit tests in order to pass the pipeline. ## Prerequisites You should already have all the setup in your Mule application for running the MUnits in a CI/CD pipeline. In summary, this is what you should already have: - The required dependencies, plugins, or properties in your `pom.xml` to be able to run MUnits for your Mule project. - The working MUnit tests under `src/test/munit`. - A working CI/CD pipeline. See the [previous article](https://www.prostdev.com/post/part-3-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-testing) if you still need to set this up. ## Modify your pom.xml This time there’s nothing to add/modify in the other files (`build.yml` / `settings.xml`). We’ll simply add the following configuration to our `pom.xml` under the existing `munit-maven-plugin` configuration. ```xml com.mulesoft.munit.tools munit-maven-plugin ${munit.version} test test test coverage-report ${decryption.key} true true 90 console sonar json html ``` Let’s explore each of the changes one by one. Starting with the `environmentVariables`. ```xml ${decryption.key} ``` We are adding this variable right here because it needs to be passed down to the Mule application to be able to decrypt the existing secured properties. > [!NOTE] > If you’re not sure how to do this setup or want to understand where we are using the `secure.key` or `decryption.key`, please refer to [Part 2](https://www.prostdev.com/post/part-2-ci-cd-pipeline-with-mulesoft-and-github-actions-secured-encrypted-properties) of this series. Next, we added the configuration to set the minimum coverage percentage for the application to pass the build. ```xml true 90 ``` In this case, we only set up the `requiredApplicationCoverage` option with a minimum percentage of 90. If you instead (or additional to) want to set up the percentage for the resources or the flows, you can use the other two options. To see the difference in these settings, you can refer to [this live stream](https://youtu.be/Z0ORL0yoKCo) or the [official documentation](https://docs.mulesoft.com/munit/latest/coverage-maven-concept#set-up-a-minimum-coverage-level). Finally, you can generate different coverage reports to be able to see your statistics in several formats. ```xml console sonar json html ``` You can choose to generate some of them or all of them. Simply add/remove the `format` field from this configuration. > [!DOCS] > For more information, see [Coverage Report Types](https://docs.mulesoft.com/munit/latest/coverage-maven-concept#coverage-report-types). > [!NOTE] > If you want to be able to see your generated reports from the CI/CD pipeline, you can add the following configuration under the `test` job in your `build.yml` file. Add this as the last step. You will be able to download the reports after the whole pipeline has finished running (see the next screenshot). ```yaml - name: Upload MUnit reports uses: actions/upload-artifact@v3 with: name: munit-test-reports path: target/site/munit/coverage/ ``` ## Run the pipeline That’s it! Once you’re done with the changes, simply push a new change to the **main** branch and this will trigger the pipeline. ![Successful GitHub Actions run with test, build, and deploy jobs and two artifacts](../../assets/blog/part-4-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-minimum-coverage-percentage-2.png) ## More resources You can check out my [GitHub profile](https://github.com/alexandramartinez) for more CI/CD repos: - [github-actions](https://github.com/alexandramartinez/github-actions) to deploy a Mule app to CloudHub - [dataweave-utilities-library](https://github.com/alexandramartinez/dataweave-utilities-library) to publish a DataWeave library to Exchange - [api-catalog-cli-example](https://github.com/alexandramartinez/api-catalog-cli-example) to update APIs in Exchange using the API Catalog CLI I hope this was helpful! Don't forget to subscribe so you don't miss any future content. --- ## Down the Rabbit Hole vol.1 - Using GPUs in MuleSoft Source: https://prostdev.com/post/down-the-rabbit-hole-vol-1-using-gpus-in-mulesoft | Published: Apr 25, 2023 | Category: Opinion Hello, fellow tech enthusiasts! My name is Nicky, and I’m a MuleSoft expert developer and architect with six years of experience in the field. I’m also a recognized MuleSoft mentor and a big fan of learning by doing weird stuff. Recently, I’ve been working on a pet project that I thought might interest some of you. I’ve written a Mule plugin that has the ability to use **Cuda cores for parallel asynchronous processing**. Now, I know what you’re thinking: “That’s not recommended or supported!” And you’re right. This is not something that should be done in a production environment. However, in theory, and practice, this idea works. So why did I create this plugin? Well, I’m always on the lookout for new and innovative ways to push the boundaries of what’s possible with MuleSoft. And using Cuda cores for parallel processing is certainly one way to do that. ![Logo for the Mcuda Mule GPU Acceleration plugin](../../assets/blog/down-the-rabbit-hole-vol-1-using-gpus-in-mulesoft-2.png) Think about it: with the ability to leverage Cuda cores, we can achieve incredible levels of processing speed and power. For example, we’ve been trying to train a neural network for payload validation, which so far has had some positive results. Image processing, video encoding, and other intensive tasks could also benefit from this kind of parallel processing. Of course, there are some risks involved. If not done properly, using Cuda cores could cause instability, crashes, and other issues. That’s why I want to emphasize that **this is not something that you should start using in your projects**. However, for those who are interested in exploring the possibilities of this kind of technology, it’s definitely worth a read. ## Prerequisites We won’t dive too deep into the technical details of how this plugin works, it’s important to note that it **requires a machine with an NVidia GPU**. Without this hardware, it won’t be possible to utilize Cuda cores for parallel processing. Who knows; maybe we’ll see GPU-enabled Mule Runtimes in the future? Assuming you have the necessary hardware, the next step is to **install the developer packages for Cuda**. These packages include the necessary tools and libraries to program the Java code that will interact with the GPU. Once you have the Cuda developer packages installed, you can then utilize JCuda to communicate directly with the GPU memory. JCuda is a Java binding for the Cuda runtime and driver API, which makes it possible to write Java code that can take advantage of Cuda cores. (Runtime API = Basic, pre-compiled functionality. Driver API = write your own kernel, almost like low-level C programs.) ## Mule project To get started with JCuda, you’ll need to include the JCuda JAR files in your Mule project. You can download these JAR files from the JCuda website. Once you have the JCuda JAR files included in your project, you can start writing Java code that will utilize the GPU. > [!BUTTON] > [JCuda Tutorial](http://www.jcuda.org/tutorial/TutorialIndex.html) ![Java code using JCuda to allocate managed GPU memory and run a kernel from a Mule flow](../../assets/blog/down-the-rabbit-hole-vol-1-using-gpus-in-mulesoft-3.png) In conclusion, I’m excited to see where this kind of innovation could take us in the world of MuleSoft. As always, I encourage everyone to keep pushing the limits and exploring new ideas. Who knows what we might discover next? The below image shows a proof of concept interaction of a Mule flow and the GPU. Ignore the memcpy error, that’s my fault ;-) ![Anypoint Studio hello-cuda flow and console printing Hello test from CUDA from the GPU](../../assets/blog/down-the-rabbit-hole-vol-1-using-gpus-in-mulesoft-4.png) --- ## 5 tips to have a better setup when you're live streaming from a MacOS computer Source: https://prostdev.com/post/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer | Published: Apr 19, 2023 | Category: Tutorials I've been doing a lot of live streaming lately. Either alone or with guests. And I noticed some things that could be improved (from a streamer perspective) to create a better experience for the people watching our screen. I'll give you some MacOS-specific tools, but I'm sure you can find a replacement for these for other operating systems. Let's take a look! ## 1. Adjust your screen's resolution This applies to any kind of screen, regardless of the operating system. Before starting the stream, whether you're using your own laptop/computer or an external monitor for screen-sharing, make sure you adjust the screen to a **1080 or 720 pixels resolution** (if you want Full HD or HD resolution). The reason behind this is that everything on the screen will look bigger and clearer in the live stream. A lot of people watch live streams on smaller screens. Don't assume that everyone will be able to just keep your stream on a full screen. Try to show your tools and other things as big as possible without pixelating the image and also keeping a high resolution. Here are some examples of why this setting is important. Look at the next two screens. Which one looks better for you? Can you read everything on the screen? ![Higher resolution, more space for the streamer, smaller/more difficult to see for the viewer.](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-2.png) ![Lower resolution, less space for the streamer, bigger/easier to see for the viewer.](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-3.png) Both are exactly the same screen, just with different resolution settings. The same effect will be on your live stream. It may be more comfortable for you to stream in your regular setting - which might be something similar to the first screenshot - but it will be harder for the viewers to follow/understand exactly what you're doing. You don't have to use the lowest resolution possible if it's uncomfortable for you or makes it harder to present the live coding on the screen. But try to adjust this setting where it feels better for you and also keeps a bigger image. Like the following picture. ![macOS desktop with a code editor and calculator app beside the System Settings Displays panel](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-4.png) To do this from a MacOS computer, go to your **System Settings** > **Displays**. Now, depending on whether you're using the actual MacOS computer or an external monitor for screen sharing, you will have different options. ### Option 1 - MacOS computer/laptop Change your screen resolution by selecting a larger text resolution from the thumbnails. You may have to play around with these a bit to make sure the image is good and you can still do your demo comfortably. ![macOS Displays resolution thumbnails from Larger Text to More Space, with Default selected](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-5.png) ### Option 2 - External monitor Change your screen resolution by selecting either **1080p** or **720p**. - **1080** will give you a higher resolution (Full HD or FHD) but will result in larger data (more GB for the video size). If your internet connection is not very good when live streaming, you can try changing to 720. If the data size and internet are not a problem, you can use 1080. - **720** will still be high-definition (HD) but won't be as clear as FHD. Normally these differences are not as easily caught if watched from a phone or a computer. They present more when watching from a larger device like a TV. ![macOS external-monitor resolution list with 1080p highlighted among options like 720p](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-6.png) If you want to see these differences yourself, you can try watching [this video](https://youtu.be/bsM4D-ek-T4) (or any video) in different resolutions. To do this, inside the video, go to the **settings** button > **quality**, and select either **1080** or **720**. > [!NOTE] > [That video](https://youtu.be/bsM4D-ek-T4) was recorded on an external monitor using the 1080p setting. ## 2. Zoom in on available applications There are some applications that can natively zoom in on the UI. For example, Visual Studio Code or Sublime Text. You can press **cmd +** to zoom in and **cmd -** to zoom out. In some applications, you might have to press **cmd =** to zoom in, depending on your settings and your keyword. For example, the following screenshot is how I normally use Visual Studio Code when I'm not sharing my screen. For you as the viewer, it might not be as clear to be able to follow what I'm doing. Even though I can see it as perfectly normal from my perspective. ![My normal view, without zooming in.](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-7.png) But if I press **cmd +**, I'll be able to zoom the UI in. It might look unnecessarily big from my perspective, but it's the best view for you who are just watching my shared screen. ![The UI after zooming in.](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-8.png) This looks way better for you now, right? And same as before, you don't have to make the screen incredibly big. Just find a balance where it's comfortable for you to work on it, but it's as big as possible for the people who are watching. ## 3. Use the accessibility settings Inside your **System Settings** > **Accessibility** > **Zoom**, there are several options to be able to use the Zoom tool on your screen. This is especially useful when you have some applications that don't support the 'standard' zooming options (**cmd +** or **cmd -**). ![macOS Accessibility Zoom settings with scroll-gesture zoom and Picture-in-Picture style enabled](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-9.png) The option that I always use for my streams is the **Picture-in-Picture** zoom. I have it set up to use the **command** key to be able to use the scroll from my mouse to zoom in/out. You can also click on the **Size and Location** button to set up how big/small you want the zoom to be, or click on the **Advanced** button for even more customization on the functionality. These are my Advanced settings, in case you want to set it up as I have it: ![Advanced options 1/2](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-10.png) ![Advanced options 2/2](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-11.png) ## 4. Display your clicks Your audience might not know if you're double-clicking or drag-and-dropping with your mouse. They might be able to hear through the microphone when you're clicking your mouse, but it's not good enough to let them guess what you're doing. You can use some software to show your clicks visually on-screen. The one I use is [keycastr](https://github.com/keycastr/keycastr). It shows a black line around the cursor to show if I just clicked once, twice, or if I'm pressing down on it. After you install it, make sure you select the **Display mouse clicks** checkbox to show them. Otherwise, they won't be showing. ![keycastr Display settings](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-12.png) I also use this application to display my keystrokes, which is my next and final point. ## 5. Display your keystrokes Sometimes you use keystroke combinations that the audience might not be aware of. For example, in Visual Studio Code, you have to press **cmd + shift + P** to show the palette. Your audience may not be aware of this combination and they just see you're magically opening the palette on the screen. Instead of letting your audience guess what you're doing, you can visually display what keystrokes you're pressing. There are many paid tools that are fancier and have more customization options, but I feel [keycastr](https://github.com/keycastr/keycastr) gets the job done - and it's free. After you install it, make sure you select the **Command keys only** checkbox. Otherwise, it will show ALL of your keystrokes. Including every single letter you press when writing - which can be annoying. But with this setting, it'll only show the combinations like **cmd + shift + P** or **cmd +**. ![keycastr Display settings](../../assets/blog/5-tips-to-have-a-better-setup-when-you-re-live-streaming-from-a-macos-computer-13.png) That's all for this post! I hope this is useful :D --- ## DataWeave programming challenge #5: Reverse a phrase's words, but keep the punctuation Source: https://prostdev.com/post/dataweave-programming-challenge-5 | Published: Apr 18, 2023 | Category: Challenges > [!NOTE] > This challenge is based on [Codeacademy's Reverse Words challenge](https://discuss.codecademy.com/t/challenge-reverse-words/83796) Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F5) ## Input Consider the following input payload (can be of **txt** format): ``` Hello world May the Fourth be with you Hello world! With you, be May the Fourth With you, be May the Fourth! ``` ## Explanation of the problem Create a DataWeave script that will reverse the order of each one of the words in a phrase to form a new phrase. Each phrase or sentence is separated by a new line. Each word is separated by a space. However, the punctuation signs (! and ,) will have to stay in the same place. Lower/upper cases can be kept as-is. For example: - "Hello world!" becomes "world Hello!" - "With you, be May the Fourth!" becomes "Fourth the, May be you With!" ## Expected output In this case, the expected output would be: ``` [ "world Hello", "you with be Fourth the May", "world Hello!", "Fourth the, May be you With", "Fourth the, May be you With!" ] ``` > [!NOTE] > You can keep the output as a JSON or as a text, whatever your preference is. ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 Keep the index of each punctuation sign so you can insert it there later ### Clue #2 You can use the isAlphanumeric function from the Strings module ### Clue #3 You can use the scan function to separate each phrase in an Array of Strings ### Clue #4 You can use the joinBy function to transform from Array to String ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find here my solution. I'm sure you all can make it better! :) Mine is super long 😂 ### Solution #1 ```dataweave %dw 2.0 import slice from dw::core::Arrays import lines, isAlphanumeric from dw::core::Strings output application/json --- lines(payload) map ((line) -> flatten( line scan /[^\w\d\s]|[\w]*/ ) filter (!isEmpty($))) map ((words) -> do { var wordsobj = (words map { word: $, originalIndex: $$, move: isAlphanumeric($) }) var reversed = (wordsobj filter ($.move))[-1 to 0] var spechars = wordsobj filter (not $.move) var endingStr = if (spechars[-1].originalIndex != null) (reversed[spechars[-1].originalIndex to -1].word default [] joinBy " ") else "" --- if (isEmpty(spechars)) reversed.word joinBy " " else spechars map ((charobj,coidx) -> slice( reversed, if (coidx-1 >= 0) spechars[coidx-1].originalIndex else 0, charobj.originalIndex ).word joinBy " " then trim("$($)$(charobj.word) $(endingStr)") ) joinBy " " }) ``` I also recorded myself coming up with the solution, but without explanations. Just some lo-fi music and a screen :) you can leave the video in the background while working! :D let me know if you find this useful. Subscribe to receive notifications as soon as new content is published ✨ --- ## DataWeave programming challenge #4: Solve the Tower of Hanoi mathematical puzzle Source: https://prostdev.com/post/dataweave-programming-challenge-4 | Published: Mar 21, 2023 | Category: Challenges > [!NOTE] > This challenge is based on the [Tower of Hanoi](https://en.wikipedia.org/wiki/Tower_of_Hanoi) mathematical puzzle - **this is a tough one!** Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F4) > [!IMPORTANT] > There are 3 different scenarios to test your code is working properly and to ensure there are no hardcoded values. However, only the first scenario is provided in the Playground link below. Please copy+paste the other two scenarios to make sure your solution is working properly. ## Input Consider the following JSON inputs: **SCENARIO 1** ``` { "moves": 0, "disks": 3, "targetTower": "C", "towers": { "A": [ 1, 2, 3 ], "B": [], "C": [] } } ``` **SCENARIO 2** ``` { "moves": 0, "disks": 4, "targetTower": "C", "towers": { "A": [ 1, 2, 3, 4 ], "B": [], "C": [] } } ``` **SCENARIO 3** ``` { "moves": 0, "disks": 7, "targetTower": "Y", "towers": { "X": [], "Y": [], "Z": [ 1, 2, 3, 4, 5, 6, 7 ] } } ``` ## Explanation of the problem Create a DataWeave script to solve the [Tower of Hanoi](https://en.wikipedia.org/wiki/Tower_of_Hanoi) puzzle. The input payload contains the number of moves (starting at 0), the total number of disks in the game, the target tower where all the disks have to be moved, and the 3 towers containing each disk. (You can use [this link](https://www.mathsisfun.com/games/towerofhanoi.html) to play the game online) Here are the rules of the game: - With each move, you can only move one disk at a time from its current stack to a different stack. - You can only place smaller disks on top of bigger disks. For example, you can place disk 1 on top of disk 3 but you can't place disk 5 on top of disk 2. - You can only move the disk at the top of any stack. ## Expected output For each of the three scenarios from the input, these are the expected outputs: **SCENARIO 1** ``` { "moves": 7, "disks": 3, "targetTower": "C", "towers": { "A": [ ], "B": [ ], "C": [ 1, 2, 3 ] } } ``` **SCENARIO 2** ``` { "moves": 15, "disks": 4, "targetTower": "C", "towers": { "A": [ ], "B": [ ], "C": [ 1, 2, 3, 4 ] } } ``` **SCENARIO 3** ``` { "moves": 127, "disks": 7, "targetTower": "Y", "towers": { "X": [ ], "Y": [ 1, 2, 3, 4, 5, 6, 7 ], "Z": [ ] } } ``` ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 Read the [wikipedia page](https://en.wikipedia.org/wiki/Tower_of_Hanoi) to have more context from the algorithm perspective. ### Clue #2 From the wikipedia page, take special attention to this part: For an even number of disks: make the legal move between pegs A and B (in either direction), make the legal move between pegs A and C (in either direction), make the legal move between pegs B and C (in either direction), repeat until complete. For an odd number of disks: make the legal move between pegs A and C (in either direction), make the legal move between pegs A and B (in either direction), make the legal move between pegs B and C (in either direction), repeat until complete. ### Clue #3 If the previous clue wasn't what you needed, take a look at these other rules from the wikipedia page: Number the disks 1 through n (largest to smallest). If n is odd, the first move is from peg A to peg C. If n is even, the first move is from peg A to peg B. Now, add these constraints: No odd disk may be placed directly on an odd disk. No even disk may be placed directly on an even disk. There will sometimes be two possible pegs: one will have disks, and the other will be empty. Place the disk on the non-empty peg. Never move a disk twice in succession. ### Clue #4 If you see other solutions (in Python, C++, Java, etc) remember that those don't necessarily apply to DataWeave. Try to come up with your own solution instead. ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find here my solution. I'm sure you all can make it better! :) Mine is super long 😂 ### Solution ```dataweave %dw 2.0 import update from dw::util::Values import drop, take from dw::core::Arrays output application/json var towerNames:Array = namesOf(payload.towers) var targetTower:String = payload.targetTower var initialTower:String = payload.towers pluck (if (!isEmpty($)) $$ else "") filter ($ != "") then $[0] as String var auxTower:String = (towerNames -- [targetTower, initialTower])[0] fun moveDisk(towers:Object, fromTower:String, toTower:String) = do { var from = towers[fromTower] var to = towers[toTower] --- towers update log("Moved from", fromTower) with (from drop 1) update log("to",toTower) with (from take 1)[0] >> to } fun swapDisks(towers:Object, tower1:String, tower2:String) = do { var disk1 = towers[tower1][0] default (payload.disks + 1) var disk2 = towers[tower2][0] default (payload.disks + 1) --- if (disk1 < disk2) moveDisk(towers, tower1, tower2) else moveDisk(towers, tower2, tower1) } fun play(game:Object, moveNumber:Number = 1) = do { var isEvenGame = isEven(game.disks) var isOddGame = isOdd(game.disks) @Lazy var newTowers = if ( (isEvenGame and moveNumber == 1) or (isOddGame and moveNumber == 2) ) swapDisks(game.towers, initialTower, auxTower) else if ( (isEvenGame and moveNumber == 2) or (isOddGame and moveNumber == 1) ) swapDisks(game.towers, initialTower, targetTower) else swapDisks(game.towers, auxTower, targetTower) @Lazy var newMoveNumber = if (moveNumber == 3) 1 else moveNumber + 1 var newGame = game update "towers" with newTowers update "moves" with game.moves + 1 --- if (sizeOf(game.towers[targetTower]) == game.disks) game else play(newGame, newMoveNumber) } --- play(payload) ``` I also recorded myself coming up with the solution, but without explanations. Just some lo-fi music and a screen :) you can leave the video in the background while working! :D let me know if you find this useful. Subscribe to receive notifications as soon as new content is published ✨ --- ## Part 3: CI/CD pipeline with MuleSoft and GitHub Actions - MUnit testing Source: https://prostdev.com/post/part-3-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-testing | Published: Mar 14, 2023 | Category: Tutorials In the [previous article](https://www.prostdev.com/post/part-2-ci-cd-pipeline-with-mulesoft-and-github-actions-secured-encrypted-properties), we learned how to add a decryption key to your CI/CD pipeline to be able to decrypt your encrypted properties at runtime. If you haven’t been following the series or you’re not familiar with GitHub Actions, we recommend you start from the [first article](https://www.prostdev.com/post/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions) to understand how we are setting up all the configurations we need. In this post, we’ll see the steps to create a pipeline with GitHub Actions that will run the MUnit tests you have set up in your Mule project. > [!NOTE] > We will talk about coverage percentage in the next article. We’ll only focus on creating the basic testing configuration in this post. ## Prerequisites You should already have all the setup in your Mule application for running the MUnits. In summary, this is what you should already have: - The required dependencies, plugins, or properties in your `pom.xml` to be able to run MUnits for your Mule project. - The working MUnit tests under `src/test/munit`. - Enterprise Nexus credentials to run the MUnits in the CI/CD pipeline. For more information see [this question](https://help.mulesoft.com/s/question/0D52T000064SVU4SAO/anypoint-platform-credentials-and-nexus). Note that we didn’t need these credentials before but we need them to be able to run the tests because of some dependencies. > [!IMPORTANT] > Make sure your MUnit tests work locally (from Anypoint Studio) before attempting to create the pipeline. Otherwise, you might encounter some issues. > [!NOTE] > You do not need Nexus credentials to run the MUnits from Studio, only to run them with Maven. ## Set up your credentials In your GitHub repository, go to the **Settings** tab (make sure you are signed in to see it). Now go to **Secrets and variables > Actions**. Here you will be able to set up your repository secrets. In the previous articles, we added the following actions secrets: - `ANYPOINT_PLATFORM_PASSWORD` - `ANYPOINT_PLATFORM_USERNAME` - `DECRYPTION_KEY` For this post, we’re going to add two more secrets for the nexus credentials: - `NEXUS_USERNAME` - `NEXUS_PASSWORD` Click on **New repository secret**. In the Name field, write the name of the secret. In the Secret field, write the actual value of your key (your Nexus username/password). Click **Add secret**. Please make sure your Nexus credentials are valid before continuing. To verify this, go to [this site](https://repository-master.mulesoft.org/nexus/#welcome) and sign in. ![Nexus Repository Manager sign-in dialog prompting for username and password](../../assets/blog/part-3-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-testing-2.png) You should be able to see the welcome page after logging in. ![Nexus Repository Manager welcome page shown after a successful login](../../assets/blog/part-3-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-testing-3.png) ## Set up your build.yml In the last posts, we set up and modified our `build.yml` file under `.github/workflows`. This time, we are going to be using the same base file, just adding a few modifications to include the new configuration. You can copy and paste the following code into this file. ```yaml name: Build and Deploy to Sandbox on: push: branches: [ main ] pull_request: branches: [ main ] jobs: test: runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - name: Set up JDK 1.8 uses: actions/setup-java@v3 with: distribution: 'zulu' java-version: 8 - name: Test with Maven env: nexus_username: ${{ secrets.nexus_username }} nexus_password: ${{ secrets.nexus_password }} KEY: ${{ secrets.decryption_key }} run: mvn test --settings .maven/settings.xml -Dsecure.key="$KEY" build: needs: test runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - name: Set up JDK 1.8 uses: actions/setup-java@v3 with: distribution: 'zulu' java-version: 8 - name: Build with Maven run: mvn -B package --file pom.xml -DskipMunitTests - name: Stamp artifact file name with commit hash run: | artifactName1=$(ls target/*.jar | head -1) commitHash=$(git rev-parse --short "$GITHUB_SHA") artifactName2=$(ls target/*.jar | head -1 | sed "s/.jar/-$commitHash.jar/g") mv $artifactName1 $artifactName2 - name: Upload artifact uses: actions/upload-artifact@v3 with: name: artifacts path: target/*.jar deploy: needs: build runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - uses: actions/download-artifact@v3 with: name: artifacts - name: Deploy to Sandbox env: USERNAME: ${{ secrets.anypoint_platform_username }} PASSWORD: ${{ secrets.anypoint_platform_password }} KEY: ${{ secrets.decryption_key }} run: | artifactName=$(ls *.jar | head -1) mvn deploy -DskipMunitTests -DmuleDeploy \ -Dmule.artifact=$artifactName \ -Danypoint.username="$USERNAME" \ -Danypoint.password="$PASSWORD" \ -Ddecryption.key="$KEY" ``` As you can see, we added a new `test` job that will run the MUnits before continuing with the rest of the jobs (build/deploy). We have seen the first steps before with the other two jobs, but the last step is different from what we’ve seen before. We also added the line `needs: test` under the build job so the jobs run synchronously. Finally, we added `-DskipMunitTests` to the two maven commands from build and deploy. Since we are running the tests on the first step, there’s no need to re-run them after that. The `nexus_username` and `nexus_password` are being passed as environment variables implicitly. While the `KEY` variable is being used in the maven command. Let’s take a closer look at the maven command. ```bash mvn test --settings .maven/settings.xml -Dsecure.key="$KEY" ``` Since we are using a specific configuration in our `settings.xml` to include the Nexus repository/credentials, we will be adding this file directly to the project (without hardcoding the credentials!!). We’ll see this configuration next. ⚠️ Another **important** change in this command is that this time we are sending the Mule property directly. In the previous post, we saw the decryption key property was called `secure.key`. If your property within the Mule project is named differently, please update this command appropriately. ## Set up your settings.xml As we just mentioned, we hadn’t created a `settings.xml` in the project before. Mainly because there was no need to do so. This time, since we’re using enterprise Nexus credentials to run the MUnit tests with Maven, we do need to set up specific credentials, profiles, and repositories. To start, create a new `.maven` folder at the root of your project. Inside this folder, create a `settings.xml` file. For example, this is how my folder structure looks like: ``` github-actions/ - .github/ - .maven/ - settings.xml - src/ - mule-artifact.json - pom.xml ``` Now copy and paste the following file. There’s no need to modify anything (just make sure the name is `settings.xml`). ```xml org.mule.tools MuleRepository ${nexus_username} ${nexus_password} Mule true MuleRepository MuleRepository https://repository.mulesoft.org/nexus-ee/content/repositories/releases-ee/ default true true ``` As you can see, the Nexus username and password credentials match the environment variables we are sending through the `build.yml` file. > [!DOCS] > For more information, see [Configuring Maven to Work with Mule](https://docs.mulesoft.com/mule-runtime/3.9/configuring-maven-to-work-with-mule-esb). ## Modify your pom.xml This time there’s nothing to add to your `pom.xml` file. The MUnit dependencies and needed configuration should be added automatically by MuleSoft once you add the tests to your code. However, here’s my project’s pom in case you want to compare it with yours: ```xml 4.0.0 com.mycompany github-actions 1.0.0-SNAPSHOT mule-application github-actions UTF-8 UTF-8 4.4.0 3.8.0 amartinez-github-actions Sandbox 2.3.13 org.apache.maven.plugins maven-clean-plugin 3.0.0 org.mule.tools.maven mule-maven-plugin ${mule.maven.plugin.version} true https://anypoint.mulesoft.com ${app.runtime} ${anypoint.username} ${anypoint.password} ${app.name} ${env} MICRO us-east-2 1 true ${decryption.key} mule-application com.mulesoft.munit.tools munit-maven-plugin ${munit.version} test test test coverage-report true html org.mule.connectors mule-http-connector 1.6.0 mule-plugin org.mule.connectors mule-sockets-connector 1.2.2 mule-plugin com.mulesoft.modules mule-secure-configuration-property-module 1.2.5 mule-plugin com.mulesoft.munit munit-runner 2.3.13 mule-plugin test com.mulesoft.munit munit-tools 2.3.13 mule-plugin test org.mule.weave assertions 1.0.2 test anypoint-exchange-v3 Anypoint Exchange https://maven.anypoint.mulesoft.com/api/v3/maven default mulesoft-releases MuleSoft Releases Repository https://repository.mulesoft.org/releases/ default mulesoft-releases MuleSoft Releases Repository default https://repository.mulesoft.org/releases/ false ``` ## Run the pipeline That’s it! Once you’re done with the changes, simply push a new change to the **main** branch and this will trigger the pipeline. ![GitHub Actions run summary showing test, build, and deploy jobs all succeeding](../../assets/blog/part-3-ci-cd-pipeline-with-mulesoft-and-github-actions-munit-testing-4.png) ## More resources You can check out my [GitHub profile](https://github.com/alexandramartinez) for more CI/CD repos: - [github-actions](https://github.com/alexandramartinez/github-actions) to deploy a Mule app to CloudHub - [dataweave-utilities-library](https://github.com/alexandramartinez/dataweave-utilities-library) to publish a DataWeave library to Exchange - [api-catalog-cli-example](https://github.com/alexandramartinez/api-catalog-cli-example) to update APIs in Exchange using the API Catalog CLI I hope this was helpful! Don't forget to subscribe so you don't miss any future content. --- ## DataWeave programming challenge #3: Count palindrome phrases using the Strings module Source: https://prostdev.com/post/dataweave-programming-challenge-3 | Published: Mar 7, 2023 | Category: Challenges Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F3) > [!TIP] > For this challenge, we encourage you to make use of the [Strings module](https://docs.mulesoft.com/dataweave/latest/dw-strings)'s functions. ## Input Consider the following input payload (can be of **txt** format): ``` this is not a palindrome blue Mr. Owl ate my metal worm Was it a car or a cat I saw? you're doing great! mug Rats live on no evil star what's taking so long? kayak Live on time, emit no evil Step on no pets boat Anna 2020/02/02 15/03/2001 01/02/2010 ``` ## Explanation of the problem Each line of the input payload is considered a different phrase. Review each phrase and find those that are [palindromes](https://en.wikipedia.org/wiki/Palindrome). Notice that punctuation, spaces, or special characters should not be considered to ensure a phrase is a palindrome. For example, - **Anna** in reverse is **anna** - this is a palindrome. - **2020/02/02** in reverse is **20200202** - this is a palindrome. - **Mr. Owl ate my metal worm** in reverse is **mrowlatemymetalworm** - this is a palindrome. After that, retrieve the character count of the **original** phrase. Including punctuation, spaces, and special characters. For example, - Anna is **4**. - 2020/02/02 is **10**. - Mr. Owl ate my metal worm is **25**. Return the total sum of all the palindrome's character count. ## Expected output In this case, the expected output would be: ``` 148 ``` The result for each of the phrases (whether they're palindrome or not) should be: - false - false - true - true - false - false - true - false - true - true - true - false - true - true - false - true ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 Read the documentation of the [Strings module](https://docs.mulesoft.com/dataweave/latest/dw-strings) to make yourself familiar with the existing functions. ### Clue #2 If you've been following the previous challenges, we've learned that `payload splitBy "\n"` can be replaced with the `lines()` function from the Strings module. ### Clue #3 Use the lower() function to make sure all the characters can be compared in lower-case. ### Clue #4 Instead of using map() to iterate through the string's characters, you can use mapString() from the Strings module. ### Clue #5 You don't need to use regex to find the alphanumeric characters. You can use isAlphanumeric() from the Strings module. ### Clue #6 Instead of using selectors, you can use the reverse() function from the Strings module. ### Clue #7 Use sizeOf() and sum() to retrieve the character count in an array and then sum the complete array of numbers. ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find here one of our solutions so you can compare your result with us. ### Solution #1 ```dataweave %dw 2.0 import lines, isAlphanumeric, mapString, reverse from dw::core::Strings output application/json --- sum(lines(payload) map (line) -> do { var clearLine = lower(line) mapString ( if (isAlphanumeric($)) $ else "" ) var isPalindrome = clearLine == reverse(clearLine) --- if (isPalindrome) sizeOf(line) else 0 }) ``` Subscribe to receive notifications as soon as new content is published ✨ --- ## Part 2: CI/CD pipeline with MuleSoft and GitHub Actions - secured/encrypted properties Source: https://prostdev.com/post/part-2-ci-cd-pipeline-with-mulesoft-and-github-actions-secured-encrypted-properties | Published: Mar 2, 2023 | Category: Tutorials In the [previous article](https://www.prostdev.com/post/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions), we learned the basics to get started with a simple CI/CD pipeline to deploy a Mule application to CloudHub. We learned how to create Secrets in GitHub for your Anypoint Platform credentials, how to set up the `build.yml` file to set up the workflows, how to run the pipeline and the details of each of the steps in the pipeline. If you haven’t gone through that, I highly recommend you do. The steps for this post won’t be as detailed. In this post, we’ll see the steps to create a pipeline with GitHub Actions that will decrypt your secured properties in your Mule application. If you don’t have secured properties in your application, you may not need this configuration. ## Prerequisites You should already have all the setup in your Mule application for the secured properties. In summary, this is what you should already have: - Your secure properties file(s) under `src/main/resources` - Your encrypted properties (see [Encrypt Properties Using the Secure Properties Tool](https://docs.mulesoft.com/mule-runtime/latest/secure-configuration-properties#secure_props_tool)) - Your **Secure Properties Config** in Global Elements (`secure-properties:config`) - The name of your decryption key property (i.e., `secure.key`) - We recommend you add the key name to the `mule-artifact.json` file so it appears hidden in Runtime Manager (i.e., `"secureProperties": ["secure.key"]`) > [!IMPORTANT] > Make sure your Mule application is correctly configured and works locally before attempting to create the pipeline. Otherwise, you might encounter some issues. ## Set up your credentials In your GitHub repository, go to the **Settings** tab (make sure you are signed in to see it). Now go to **Secrets and variables > Actions**. Here you will be able to set up your repository secrets. In the previous tutorial, we added two credentials: - `ANYPOINT_PLATFORM_PASSWORD` - `ANYPOINT_PLATFORM_USERNAME` For this part, we’re going to add a new one that will contain the decryption key. Click on **New repository secret**. In the Name field, write `DECRYPTION_KEY`. In the Secret field, write the actual value of your key. For example, `MyMuleSoftKey`. Click **Add secret**. ![GitHub Actions New secret form with name DECRYPTION_KEY and value MyMuleSoftKey](../../assets/blog/part-2-ci-cd-pipeline-with-mulesoft-and-github-actions-secured-encrypted-properties-2.png) You don’t have to change the name of this secret. We will keep it as-is because it will be used in the pipeline. I’ll show you later in the post where to modify the name of the property to match the one you have in your Mule application (like `secure.key`). ## Set up your repo In the last post, we learned how to set up our `build.yml` file under `.github/workflows` for the actual pipelines in GitHub. This time, we are going to be using the same base file, just adding a few modifications to include the new decryption key. You can copy and paste the following code into this file. ```yaml name: Build and Deploy to Sandbox on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - name: Set up JDK 1.8 uses: actions/setup-java@v3 with: distribution: 'zulu' java-version: 8 - name: Build with Maven run: mvn -B package --file pom.xml - name: Stamp artifact file name with commit hash run: | artifactName1=$(ls target/*.jar | head -1) commitHash=$(git rev-parse --short "$GITHUB_SHA") artifactName2=$(ls target/*.jar | head -1 | sed "s/.jar/-$commitHash.jar/g") mv $artifactName1 $artifactName2 - name: Upload artifact uses: actions/upload-artifact@v3 with: name: artifacts path: target/*.jar deploy: needs: build runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - uses: actions/download-artifact@v3 with: name: artifacts - name: Deploy to Sandbox env: USERNAME: ${{ secrets.anypoint_platform_username }} PASSWORD: ${{ secrets.anypoint_platform_password }} KEY: ${{ secrets.decryption_key }} run: | artifactName=$(ls *.jar | head -1) mvn deploy -DmuleDeploy \ -Dmule.artifact=$artifactName \ -Danypoint.username="$USERNAME" \ -Danypoint.password="$PASSWORD" \ -Ddecryption.key="$KEY" ``` As you can see, we added a new `KEY` variable that will get the decryption key from our secrets; and we are sending a `-Ddecryption.key` parameter into the maven command. > [!NOTE] > If you add this manually to the maven command, don’t forget to add a backslash (`\`) in the previous line. ## Modify your pom.xml Go to your Mule application’s `pom.xml` file and locate the `cloudHubDeployment` configuration. You will need to add the following property: > [!NOTE] > this is where the name of your actual property needs to be set. In our example, we are using `secure.key`. Modify this field to match the name of your property, but keep the `decryption.key` since we’re using that in the GitHub Action pipeline. ```xml ${decryption.key} ``` Your configuration should look something like this: ```xml org.apache.maven.plugins maven-clean-plugin 3.0.0 org.mule.tools.maven mule-maven-plugin ${mule.maven.plugin.version} true https://anypoint.mulesoft.com ${app.runtime} ${anypoint.username} ${anypoint.password} ${app.name} ${env} MICRO us-east-2 1 true ${decryption.key} mule-application ``` That’s all! Once you send these changes to the main branch, your pipeline will start running and deploying to CloudHub. ## More resources You can check out my [GitHub profile](https://github.com/alexandramartinez) for more CI/CD repos: - [github-actions](https://github.com/alexandramartinez/github-actions) to deploy a Mule app to CloudHub - [dataweave-utilities-library](https://github.com/alexandramartinez/dataweave-utilities-library) to publish a DataWeave library to Exchange - [api-catalog-cli-example](https://github.com/alexandramartinez/api-catalog-cli-example) to update APIs in Exchange using the API Catalog CLI I hope this was helpful! Don't forget to subscribe so you don't miss any future content. --- ## DataWeave programming challenge #2: Rock Paper Scissors game score system Source: https://prostdev.com/post/dataweave-programming-challenge-2 | Published: Feb 28, 2023 | Category: Challenges > [!NOTE] > This challenge is based on [Advent of Code 2022 day 2](https://adventofcode.com/2022/day/2) Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F2) ## Input Consider the following input payload (can be of **txt** format): ``` R R R P R S P R P P P S S R S P S S R R ``` ## Explanation of the problem Create a DataWeave script to keep your score on a series of Rock Paper Scissors games. The first column is what your **opponent** chose and the second column is what **you** chose. Each letter is a representation of the decision made: - R = Rock - P = Paper - S = Scissors The rules of the game are simple: - Rock defeats Scissors - Paper defeats Rock - Scissors defeat Paper As per the scoring system, you will have to keep track of whether you won, lost, or if it was a draw. Remember the second column is your choice. Here's how you will be counting the points: - 0 points if you lost the round - 3 points if the round was a draw - 6 points if you won the round For example, - The first round was **R R**, which means both your opponent and you chose Rock. This round results in a **draw**, so **3** points are added. - The second round was **R P**, which means your opponent chose Rock and you chose Paper. Paper defeats Rock, which means you win this round. **6** points are added. - The third round was **R S**, which means your opponent chose Rock and you chose Scissors. Rock defeats Scissors, which means you lose this round. **0** points are added. The final result will be the number of total points you earned in the 10 rounds. ## Expected output In this case, the expected output would be: ``` 30 ``` The result for each of the 10 rounds should be: - 3 - 6 - 0 - 0 - 3 - 6 - 6 - 0 - 3 - 3 ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 Using `splitBy "\n"` is the same as using the lines() function from the Strings module ### Clue #2 You can use reduce() to keep the score in the accumulator ### Clue #3 If you don't want to use reduce, you can use map() to get the points of each round and add the numbers at the end ### Clue #4 You can use splitBy() to get your opponent's moves and yours to compare ### Clue #5 You can create a variable with the pointing system ### Clue #6 The rules can be made of objects in order to extract the points/moves ### Clue #7 If you used map() to get the points, you can use sum() to add all the numbers ### Clue #8 You can use the `do` operator to create localized variables ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find here some solutions we are providing so you can compare your result with us. ### Solution #1 - based on Felix Schnabel's solution ```dataweave %dw 2.0 output application/json import lines from dw::core::Strings var defeats = { R: "S", P: "R", S: "P" } fun getScore(opponent:String, me:String): Number = if (opponent == me) 3 else if (defeats[me] == opponent) 6 else 0 --- sum(lines(payload) map getScore($[0],$[-1])) ``` ### Solution #2 ```dataweave %dw 2.0 output application/json var rules = { R: { R: 3, P: 6, S: 0 }, P: { R: 0, P: 3, S: 6 }, S: { R: 6, P: 0, S: 3 } } --- payload splitBy "\n" reduce ((round, score=0) -> do { var arr = round splitBy " " var opponent = arr[0] var me = arr[-1] --- score + rules[opponent][me] }) ``` ### Solution #3 ```dataweave %dw 2.0 output application/json import lines from dw::core::Strings var rules = { R: { R: 3, P: 6, S: 0 }, P: { R: 0, P: 3, S: 6 }, S: { R: 6, P: 0, S: 3 } } --- lines(payload) map do { var arr = $ splitBy " " --- rules[arr[0]][arr[1]] } then sum($) ``` Subscribe to receive notifications as soon as new content is published ✨ --- ## How to develop a Battlesnake using a MuleSoft API and the DataWeave language Source: https://prostdev.com/post/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language | Published: Feb 22, 2023 | Category: Tutorials If you’re not familiar with the modern Battlesnake game, you must at least remember the snake game from those old cell phones. It’s basically the same, but developing APIs and deploying them to the cloud...And fighting other snakes 😂 In this post, I’m going to guide you through all the steps to generate your own MuleSoft starter project in GitHub and be ready to start modifying your DataWeave code to move the snake. ## Create your own GitHub repository The first step is to get your own repo based on the Mule starter project that the great Manik created for us. To do this, you first have to go to the [official list of Starter Projects](https://docs.battlesnake.com/starter-projects) in the Battlesnake documentation. Scroll down until you see the **Mule and DataWeave Battlesnake Quickstart** project created by Manik Magar and click on it. ![Battlesnake starter projects grid with the Mule and DataWeave Quickstart highlighted](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-2.png) Once you open Manik’s GitHub repo, click on the **Use this template** button and select **Create a new repository**. ![GitHub repo with the Use this template menu open on Create a new repository](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-3.png) Add any name you choose for your repo and click **Create repository from template**. ![GitHub "Create a new repository from template" form with repo name my-snake](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-4.png) This will generate your own GitHub repo. You’re now ready to start modifying your code! ## Personalize your snake You can personalize your own snake. For example, changing its head/tail design or changing its color. You can modify this data at `src/main/resources/application.yaml`. This is what Manik added to the starter project: ```yaml battlesnake: config: "apiversion": "1" "author": "mmsonline" "color" : "#888888" "head" : "default" "tail" : "default" "version" : "0.0.1-beta" ``` And this is what I have for my snake: ```yaml battlesnake: config: "apiversion": "1" "author": "alexandramartinez" "color" : "#6c25be" "head" : "caffeine" "tail" : "nr-booster" "version" : "0.0.1-beta" ``` You can read more about the customizations in the [Battlesnake docs](https://docs.battlesnake.com/guides/customizations), or you can go ahead and search for the available [head/tail customizations](https://play.battlesnake.com/customizations). You can also leave the defaults as they are and your snake is still going to work. But it’s nice to be able to customize stuff :-) ## Find the DataWeave logic The API is not the problem here. Manik generated a pretty good starter project. The thing is that you need to create all the DataWeave code on your own to tell the snake where to move to. Make sure you read the [Battlesnake docs](https://docs.battlesnake.com/guides) so you can understand how everything works. But I’ll give you a quick summary. You will receive a [POST /move](https://docs.battlesnake.com/api/requests/move) request in your API containing the current turn. This will trigger the DataWeave code (located at `src/main/resources/dw/move-snake.dwl`) and return a response of where to move from here. All your DataWeave code will return is a JSON object telling the game where you’re gonna move next. For example: ```json { "move": "up", "shout": "Moving up!" } ``` The trick here is to know exactly where to move based on a series of rules. You will receive a payload with a general structure that looks something like this: ```json { "game": { "id": "...", "ruleset": { "name": "standard", "settings": {...} } }, ... }, "turn": 42, "board": { "height": 11, "width": 11, "snakes": [ { "id": "...", "name": "...", "body": [...], "head": {...}, ... } "food": [...], "hazards": [] }, "you": { "body": [...], "head": {...}, "length": 3, ... } } ``` > To see a full example, please refer to the docs [here](https://docs.battlesnake.com/api/example-move#move-api-request). So, your code will have to make decisions based on where the other snakes are, where the walls are, where the food is, etc. This is the challenging part! Manik gives you a pretty good way to start coding the logic. But the starter snake only knows to not collide with its own neck. It doesn’t know how to not hit walls, not hit itself, not hit others, or find food. You will need to code that on your own. ## Deploy the Mule app Whenever you feel ready to try out your snake’s logic, you can deploy your Mule application directly to **CloudHub** and take the URL from there. It should look like this: ``` http://amartinez-mule-battlesnake.us-e2.cloudhub.io ``` > For information on how to deploy your application, see [Deploy to CloudHub](https://docs.mulesoft.com/cloudhub-1/deploying-to-cloudhub). > [!TIP] > If your Anypoint Platform account expires, you can create a new account using the same details as before (name, email, phone, etc.) and just changing your username. However, deploying to CloudHub can take a long time. I’ve personally been using **ngrok** to do local deployments and be able to expose my local URL through the public internet. You can follow the next steps to use ngrok: - Install [ngrok](https://ngrok.com/) - Run your Mule application locally - Run `ngrok http 8081` - Retrieve the public URL This URL would look something similar to this: ``` https://c0cb-2001-1970-5425-e000-00-7e86.ngrok.io ``` Now you’re all ready to create your actual snake on Battlesnake! ## Create your Battlesnake First of all, you need to create an account at [play.battlesnake.com](https://play.battlesnake.com/). Once you have it, you can go to the **Battlesnakes** tab, click on **New Battlesnake**, and fill out the details. Make sure you add `/api` to the end of the URL (either CloudHub or ngrok). Here’s my snake if you want to take it as an example: ![Battlesnake settings for "Maxine the Mule" with the CloudHub API URL and tags](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-5.png) After you click on **Save**, it will attempt to call the URL. If the app was successfully deployed, there shouldn’t be any issue with saving it. Now you’re ready to test your snake! ## Create a new game Go to the **Create Game** tab. I recommend you leave the default options selected. Scroll down if needed and click **Add to Game** next to your snake. ![Battlesnake Create Game screen with Standard map and My Battlesnakes listed](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-6.png) You can also click on the **Official Bots** tab to select more snakes for the game or click on **Public Battlesnakes** to add your friends’ snakes. ![Create Game screen with Official Bots added alongside Maxine the Mule, ready to start](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-7.png) Once you feel the game is ready, simply click on **Start Game** and then press the play (▶️) button. ![Animated Battlesnake game board with snakes moving toward food pellets](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-8.gif) Look at my Maxine go! ## Add your snake to the leaderboard Once you have a good and stable snake, you can go to the [leaderboards](https://play.battlesnake.com/leaderboards) and add your Battlesnake so it can start earning points and get a place! These are my stats so far: ![Alex Martinez's Battlesnake profile showing Standard and Duels ratings and ranks](../../assets/blog/how-to-develop-a-battlesnake-using-a-mulesoft-api-and-the-dataweave-language-9.png) I work on my snake at least once per week during my live streams. You can watch me live on [Twitch](https://www.twitch.tv/devalexmartinez) or watch my previous streams on [YouTube](https://www.youtube.com/playlist?list=PLb61lESgk6hi60IazebMZ7pmZBZeIcEKt). I hope this was helpful! Feel free to play around with Maxine the Mule if you want :) just search for/select her from the **Public Battlesnakes**. Send me any good games you have with her! I’d love to see them playing :D --- ## DataWeave programming challenge #1: Add numbers separated by paragraphs and get the max number Source: https://prostdev.com/post/dataweave-programming-challenge-1 | Published: Feb 21, 2023 | Category: Challenges > [!NOTE] > This challenge is based on [Advent of Code 2022 day 1](https://adventofcode.com/2022/day/1) Try to solve this challenge on your own to maximize learning. We recommend you refer to the [DataWeave documentation](https://docs.mulesoft.com/dataweave/latest/) **only**. Try to avoid using Google or asking others so you can learn on your own and become a DataWeave expert! > [!PLAYGROUND] > [Solve on the Playground](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-challenges&path=challenges%2F1) ## Input Consider the following input payload (can be of **txt** format): ``` 1 2 3 4 5 0 15 20 5 1 3 25 6 4 4 ``` ## Explanation of the problem Create a DataWeave script to add all the numbers separated by a paragraph. For example, - the first number would be **3** (2 + 1), - the second number would be **12** (3 + 4 + 5 + 0), - the third number would be **15**, and so on. After that, retrieve the highest number. *(Using code, not manually* 😉) ## Expected output In this case, the expected output would be: ``` 29 ``` ## Clues If you're stuck with your solution, feel free to check out some of these clues to give you ideas on how to solve it! ### Clue #1 Use the splitBy() function ### Clue #2 Instead of splitting by "\n" try splitting by "\n\n" at first - to separate by paragraph ### Clue #3 Use map() to go through each paragraph ### Clue #4 Use splitBy() again to separate each of the paragraph's lines ### Clue #5 Use sum() to add all the numbers in each paragraph ### Clue #6 Use max() to return the highest number from the array ### Clue #7 If some of the numbers are string instead of numbers, use `as` to coerce them to Numbers -- this is a bug and will hopefully be fixed soon! ## Answer If you haven't solved this challenge yet, we encourage you to keep trying! It's ok if it's taking longer than you thought. We all have to start somewhere ✨ Check out the clues and read the docs before giving up. You got this!! 💙 There are many ways to solve this challenge, but you can find here some solutions we are providing so you can compare your result with us. ### Solution #1 ```dataweave %dw 2.0 output application/json --- (payload splitBy "\n\n") map (sum($ splitBy "\n") as Number) then max($) ``` ### Solution #2 ```dataweave %dw 2.0 output application/json --- (payload splitBy "\n\n") map ((item) -> ( sum(item splitBy "\n") as Number )) then max($) ``` ### Solution #3 ```dataweave %dw 2.0 output application/json --- (payload splitBy "\n\n") map ((item,index) -> do { var newItem = item splitBy "\n" --- sum(newItem) as Number }) then max($) ``` ### Solution #4 ```dataweave %dw 2.0 output application/json --- max((payload splitBy "\n\n") map (sum($ splitBy "\n") as Number)) ``` Did you like this challenge?? Let us know what you think!! 👇 Subscribe to receive notifications as soon as new content is published ✨ --- ## DataWeave 2.0 scopes for local variables: 'using' vs. 'do' operators Source: https://prostdev.com/post/dataweave-2-0-scopes-for-local-variables-using-vs-do-operators | Published: Feb 14, 2023 | Category: Guides I had previously written a post with [my top 5 DataWeave tips to make your life easier](https://www.prostdev.com/post/my-top-5-dataweave-tips-to-make-your-life-easier) where I talked about the *do* operator. However, I’ve seen a lot of people asking about the *using* operator since it was inherited from DataWeave 1.0. In this post, I’ll go through some of the main differences between these two operators so you decide which one to use in your DataWeave scripts! > [!NOTE] > By this point, I’m assuming you already have the basic DataWeave understanding. If this is not the case, I recommend you read [these tutorials first](https://developer.mulesoft.com/tutorials-and-howtos/dataweave/what-is-dataweave-getting-started-tutorial/). ## What are scopes? If you come from a different programming paradigm, this concept might be easy to understand. To keep it simple, let’s just say that scopes are these imaginary boxes where you can keep variables, functions, and so on. These variables (or functions) are not available to anyone else outside of these boxes, only inside. We will see some examples in a bit. In DataWeave, you can have **global** and **local** variables (and functions, annotations, etc.). The global variables are the ones that appear over the three dashes (---) that separate your script. For example ```dataweave %dw 2.0 output application/json var globalVar = "this is my global variable" --- globalVar ``` These variables are accessible from anywhere in your script. They don’t have to exist inside a specific scope to be used. Let’s talk about how to create local variables with the *using* or *do* operators so we can put together the meaning of *scopes*. ## Creating scopes and local variables with the *do* operator Try out the following example. What age do you think will be outputted from this script? ```dataweave %dw 2.0 output application/json var age = 21 --- { person: do { var user = "Robin" var age = 5 --- { name: user, age: age } } } ``` The answer is 5. We used the *do* operator in line 6 to create a local scope. Notice how we created the variables *user* and *age* inside this scope. This new scope will work very similarly to a regular script in the sense that it can have its variables, functions, etc. Notice how we have to add three dashes (---) in line 9 to separate the variable declaration from the localized script. This is because when you use the *do* operator, you can use this syntax for your new local scope. Below is a graphical representation of how these two scopes are separated. ![DataWeave script with the global scope outlined in green and the do-operator local scope in red](../../assets/blog/dataweave-2-0-scopes-for-local-variables-using-vs-do-operators-2.png) The age in the output is 5 because it will take the “closest” variable. In other words, the more local the variable, the more precedence it takes. What do you think will be the output if you use the following code? Try to figure it out! The answer will be at the end of the post 🙂 ```dataweave %dw 2.0 output application/json var age = 21 --- { person: do { var user = "Robin" var age = 5 --- { name: user, age: age + do { var age = 10 --- age } } } } ``` You can create as many local scopes as you want, but I wouldn’t recommend having more than 1 unless completely necessary. Mainly to avoid spaghetti code. You can read more about *do* in this post: [My top 5 DataWeave tips to make your life easier](https://www.prostdev.com/post/my-top-5-dataweave-tips-to-make-your-life-easier). ## Creating local variables with the *using* operator Now, I have to be completely honest here. I haven’t used the *using* operator. I did work on DataWeave 1.0 for a while, but it was confusing for me to understand how this syntax worked, so I never really used it. Here is an awesome post if you’re still developing with DataWeave 1.0: [How to use the “Using” Operator in DataWeave, the MuleSoft Mapping Tool](https://bitsinglass.com/mulesoft-dataweave-operator-tips/). Otherwise, I’ll explain how to create local variables in DataWeave 2.0 here. Let’s follow the previous example to demonstrate. ```dataweave %dw 2.0 output application/json var age = 21 --- { person: using (user = "Robin", age = "5") { name: user, age: age } } ``` We will have the same functionality as before but see how this syntax is different than *do*. We can also create variables with lambda expressions to use them as functions. Like in the following example. ```dataweave %dw 2.0 output application/json var age = 21 --- { person: using ( user = "Robin", age = "5", echo = (t) -> t ) { name: user, age: age, test: echo(123) } } ``` However, this operator is considered deprecated since DataWeave 2.0. I personally don’t like the way the syntax works. It makes more sense for me to use *do* instead. But you can still use *using* if you’re really into it 😀- although you might get a huge warning in your scripts telling you to stop using this operator! ![Anypoint Studio warning that the using operator is deprecated, suggesting replacing it with do](../../assets/blog/dataweave-2-0-scopes-for-local-variables-using-vs-do-operators-3.png) ## My advice for you The *using* operator is deprecated for a reason. The fact that the team didn’t remove it from DataWeave 2.0 doesn’t mean that it’s the best way to create local variables. Yes, you can still use it if that’s the way you’ve been doing it since DataWeave 1.0, but who knows when they’ll remove it? It might be risky to keep it that way. I also feel it’s more intuitive to use *do* because of the three-dashes syntax that we already know how to use. We can easily declare local variables, functions, annotations, or namespaces the same way we do global ones. However, it can be tricky to use at first because of the curly brackets ( { } ). But I swear, the more you use it, the less time you take to do it automatically! You can also check out the docs to read more about these two: - (DataWeave 2.0) [Scope and Flow Control Operators](https://docs.mulesoft.com/dataweave/latest/dw-operators#scope-and-flow-control-operators) - (DataWeave 2.0) [Example: Local DataWeave Variables](https://docs.mulesoft.com/dataweave/latest/dataweave-variables#example_local_variable) - (DataWeave 1.0) [Scoped Variables](https://docs.mulesoft.com/dataweave/1.2/dataweave-reference-documentation#scoped-variables) Oh! And the answer to the question is this: ```json { "person": { "name": "Robin", "age": 15 } } ``` Try it out if you don’t believe me :-) Let me know if you have any questions! -alex --- ## How to generate shareable link examples from GitHub to open in the DataWeave Playground Source: https://prostdev.com/post/how-to-generate-shareable-link-examples-from-github-to-open-in-the-dataweave-playground | Published: Feb 7, 2023 | Category: Tutorials Some months ago, the DataWeave engineering team created this beautiful functionality to be able to open DataWeave examples (including inputs and scripts) directly in the Playground with a single link. [Here’s an example.](https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-scripts&path=functions%2FmaskFields) In this post, we’re going to learn how to create these shareable links from a GitHub repository to open directly in the online DataWeave Playground. ## The link's anatomy Let’s use as an example the link I shared at the beginning of the post. This is the complete URL: ``` https://dataweave.mulesoft.com/learn/playground?projectMethod=GHRepo&repo=alexandramartinez%2Fdataweave-scripts&path=functions%2FmaskFields ``` Now let’s separate this to make it more readable: | URL part | Description | |---|---| | https://dataweave.mulesoft.com/learn/playground | The link should always start with this. This is the URL to open the Playground. | | ? | Marks the start of the query parameters in this URI. | | & | Marks the concatenation of more query parameters. | | `projectMethod`=GHRepo | This query parameter should always stay the same. | | `repo`=alexandramartinez%2Fdataweave-scripts | This query parameter indicates where the GitHub repository is located.
This has to be a **public** repo.
In this case, it’s referencing this repo: [github.com/alexandramartinez/dataweave-scripts](https://github.com/alexandramartinez/dataweave-scripts) | | `path`=functions%2FmaskFields | This query parameter indicates the path inside the repo where the inputs and scripts are located.
In this case, it’s referencing this folder: [alexandramartinez/dataweave-scripts/tree/main/functions/maskFields](http://github.com/alexandramartinez/dataweave-scripts/tree/main/functions/maskFields) | You may have noticed that there are some weird characters in the query parameters’ values (%2F). This is just the representation of a slash (/) to make it safe to pass it down in a URL. To make sure your values for the *repo* and *path* query parameters are URL-safe, you can use this tool: [urlencoder.org](https://www.urlencoder.org/). Simply input your value at the top and click Encode. The encoded value will be returned under the button. Now that we have a better understanding of how the link is built, let’s now make sure our folder structure inside our repo (the *maskFields* folder from the previous example) is correctly set. ## The folder structure Since we are only passing the main folder to the Playground, we have to make sure everything is where it’s supposed to be. For that, we have to set the correct folder structure. The folder structure from the previous [maskFields](https://github.com/alexandramartinez/dataweave-scripts/tree/main/functions/maskFields)[folder](https://github.com/alexandramartinez/dataweave-scripts/tree/main/functions/maskFields) example would look like this: ``` maskFields/ - inputs/ - payload.xml - transform.dwl ``` As you can see, this is a simple example where there’s only one payload and only one main script (`transform.dwl`). Let’s see a more detailed explanation of what this all means. | **Folder/file** | **Description** | [Playground section](https://developer.mulesoft.com/tutorials-and-howtos/dataweave/learn-dataweave-with-the-dataweave-playground-getting-started/) | |---|---|---| | inputs/ | This folder should always be named ‘inputs’. This is where you can add several inputs if needed.
Make sure the files have the appropriate extension (.xml, .json, .txt) for the Playground to know which data format it is.
If you’re adding several payloads (for example, one JSON and one XML), make sure they are named differently. You reference these payloads from your scripts using the file name without the extension. | Input Explorer | | `transform.dwl` | This is the main file where your DataWeave script is located. The file name should always be `transform.dwl`. | Script Explorer / Script | | Additional modules | If you want to add custom DataWeave modules to the repo, you should do it at the same level as the `transform.dwl` file. These modules can have different names but they should all have the .dwl extension. | Script Explorer / Script | Once you have everything set up and committed/pushed to your **public** GitHub repository, you will be able to use the link to share your examples :) > [!NOTE] > It may take up to 30 minutes for the Playground to be updated after you make any changes to the repository. So, if you do not see your changes being reflected immediately, just wait a bit. Feel free to contact me for any questions or troubleshooting steps regarding this. I’m happy to help! -alex --- ## Simplified try-catch strategy in DataWeave with the default keyword (instead of try/orElse) Source: https://prostdev.com/post/simplified-try-catch-strategy-in-dataweave-with-the-default-keyword-instead-of-try-orelse | Published: Jan 31, 2023 | Category: Guides The other day I was reading some StackOverflow DataWeave questions (as one does) and I got to this very interesting thread: [Why the “default” keyword acts like “try + catch / orElse” in some cases](https://stackoverflow.com/questions/73785223/why-the-default-keyword-acts-like-try-catch-orelse-in-some-cases). I wanted to take a moment to create an article about my own explanation of what I see in that thread. I already created a video about it, so, I might as well create a blog post too :) P.S. You can try out the scripts in the [DataWeave Playground](https://dataweave.mulesoft.com/learn/playground) or in [Visual Studio Code](https://www.prostdev.com/post/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code). ## The question In summary, [Harshank Bansal](https://stackoverflow.com/users/10946202/harshank-bansal) notices that if you try to run the following script, you will receive an error message stating that *ABC cannot be coerced to Number* (which is expected). Script: ```dataweave %dw 2.0 output application/json --- "ABC" as Number ``` Output: ``` Cannot coerce String (ABC) to Number 4| "ABC" as Number ^^^^^^^^^^^^^^^ Trace: at main::main (line: 4, column: 1) ``` Maybe your first instinct to try to fix this code - if you’re familiar with the [try](https://docs.mulesoft.com/dataweave/latest/dw-runtime-functions-try) function - would be to do a script similar to the one Harshank mentions in the question: ```dataweave %dw 2.0 import * from dw::Runtime output application/json --- try(() -> ("ABC" as Number)) orElse "Invalid number" ``` This correctly catches the error and outputs the “Invalid number” string instead. However, Harshank found an alternative for this specific use case which is using the [default](https://docs.mulesoft.com/dataweave/latest/dataweave-cookbook-defaults) keyword. Like so: ```dataweave %dw 2.0 output application/json --- ("ABC" as Number) default "Invalid number" ``` This previous script outputs the exact same “Invalid number” string as we did with the *try* function. However, this seems a bit weird since the *default* keyword is mainly used for *null* values and this is an error. Now let’s talk about the answer/explanation that Mariano de Achaval wrote on why this behavior happens. ## The answer If you don’t know who [Mariano](https://stackoverflow.com/users/1472690/machaval) is, he’s one of the architects that started developing DataWeave years ago. That’s why his answer is a big deal :) Mariano mentions that the *default* keyword can be used for two things: - When the left-side value is *null*, the right-side value is returned. - To handle **some** exceptions like the coercion one we saw at the beginning of the post. Notice how I wrote **some** in bold there, that’s because Mariano and Harshank both mention it doesn’t work for all possible exceptions. It only works when: - You call a function that doesn’t support *null*. - You try to coerce a value to a data type (like the example we saw). Now, it is worth mentioning that this behavior is available to use but it is not recommended (as Mariano wrote). It is better to use the *try* function instead. ## My takeaway Even though it’s not recommended to use, it doesn’t hurt to know it exists :) Sometimes you need to do a quick script where you need to ensure the type coercion doesn’t return an exception (like Harshank’s example) and the code isn’t really too complex to add the *try* function. What is **your** takeaway from this?? -alex --- ## Part 1: How to set up a CI/CD pipeline to deploy your MuleSoft apps to CloudHub using GitHub Actions Source: https://prostdev.com/post/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions | Published: Jan 24, 2023 | Category: Tutorials Creating CI/CD (Continuous Integration, Continuous Delivery) pipelines for your code has become a standard practice. Instead of worrying about deployments and keeping your environments up to date, you can set up this automated pipeline to do the deployments for you. You can focus on developing your code and let the pipeline take care of the rest. In this post, we’ll learn how to create a simple CI/CD pipeline to deploy Mule applications from GitHub to CloudHub using GitHub Actions. We’ll be using [this GitHub repo](https://github.com/alexandramartinez/github-actions) for this example. ## Set up your repo We won’t go into the details of creating a GitHub repo with your Mule application. You can take a look at the [example repo](https://github.com/alexandramartinez/github-actions) we’ll be using throughout the post so we can focus on explaining the CI/CD setup. Once you have your Mule application in a GitHub repo, create a folder called `.github`, and inside it, create another folder called `workflows`. Here, create a new `build.yml` file and paste the following contents into it. ```yaml name: Build and Deploy to Sandbox on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - name: Set up JDK 1.8 uses: actions/setup-java@v3 with: distribution: 'zulu' java-version: 8 - name: Build with Maven run: mvn -B package --file pom.xml - name: Stamp artifact file name with commit hash run: | artifactName1=$(ls target/*.jar | head -1) commitHash=$(git rev-parse --short "$GITHUB_SHA") artifactName2=$(ls target/*.jar | head -1 | sed "s/.jar/-$commitHash.jar/g") mv $artifactName1 $artifactName2 - name: Upload artifact uses: actions/upload-artifact@v3 with: name: artifacts path: target/*.jar deploy: needs: build runs-on: ubuntu-latest steps: - name: Checkout this repo uses: actions/checkout@v3 - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- - uses: actions/download-artifact@v3 with: name: artifacts - name: Deploy to Sandbox env: USERNAME: ${{ secrets.anypoint_platform_username }} PASSWORD: ${{ secrets.anypoint_platform_password }} run: | artifactName=$(ls *.jar | head -1) mvn deploy -DmuleDeploy \ -Dmule.artifact=$artifactName \ -Danypoint.username="$USERNAME" \ -Danypoint.password="$PASSWORD" ``` > [!NOTE] > You can name this YAML file however you want. It doesn’t have to be `build`. ![GitHub viewing the build.yml workflow file at .github/workflows on the main branch.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-2.png) This file is where the CI/CD pipeline is set up. Before we get into the details of this file, let’s first set up the Actions secrets with your Anypoint Platform credentials. ## Set up your credentials In your GitHub repository, go to the **Settings** tab (make sure you are signed in to see it). Now go to **Secrets and variables** **>** **Actions**. Here you will be able to set up your repository secrets. ![GitHub repo Actions secrets page listing the ANYPOINT_PLATFORM_PASSWORD and ANYPOINT_PLATFORM_USERNAME secrets.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-3.png) Click on **New repository secret**. In the **Name** field, write `ANYPOINT_PLATFORM_PASSWORD`. In the **Secret** field, write the actual Anypoint Platform password you use to sign in to Anypoint Platform. Click **Add secret**. ![New secret form with Name ANYPOINT_PLATFORM_PASSWORD and a password value, with the Add secret button.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-4.png) Repeat the same process for the `ANYPOINT_PLATFORM_USERNAME` secret. Add here the username you use to sign in to Anypoint Platform. ![New secret form with Name ANYPOINT_PLATFORM_USERNAME and a username value, with the Add secret button.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-5.png) That’s it for the basic setup! Let’s take a closer look into the `build.yml` file to make sure everything is set up properly for your case. ## Inside the build.yml file Let’s go through the contents of the `build.yml` file so you can personalize it for your own needs. If you want to skip this explanation, you can go straight to Run the pipeline. ### name The first thing we see is a `name` field. This is the name of the Action that will appear in the UI under the **Actions** tab. You can name this anything you want. ```yaml name: Build and Deploy to Sandbox ``` ![GitHub Actions tab showing the "Build and Deploy to Sandbox" workflow with its list of runs.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-6.png) ### on Next, we have the `on` field. This tells you when the Action will run. In this case, we have it set up to run anytime there’s a push to the **main** branch, or when a Pull Request has been merged into the **main** branch. You can also set it up to run with a schedule, but we won’t go through that for this example. You can see a schedule example [here](https://github.com/alexandramartinez/alexandramartinez/blob/main/.github/workflows/blog-post-workflow.yml). ```yaml on: push: branches: [ main ] pull_request: branches: [ main ] ``` ### jobs Next, we have the `jobs` field. Technically, you can just create a huge job and run all the steps inside it, but it will be overwhelming for you if you need to troubleshoot a step. It is best to split it up into different jobs when possible. It’s worth noting that jobs run async to each other. We’ll learn how to create async jobs when we see the `deploy` job in a moment. In our case, we have two jobs: `build` and `deploy`. ```yaml jobs: build: [...] deploy: [...] ``` ![Successful workflow run summary with the build and deploy jobs both passing in 3m 9s.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-7.png) ### jobs: build Inside the `build` job, we start by stating the operating system in which it will run. In this case, ubuntu. ```yaml runs-on: ubuntu-latest ``` Then, we start listing the `steps` for the job. The first two steps are repeated in both jobs. First, we have to check out the current repository in the ubuntu machine where the pipeline is running. ```yaml - name: Checkout this repo uses: actions/checkout@v3 ``` After that, we set up a cache for the Maven dependencies in the ubuntu machine’s `.m2` folder. ```yaml - name: Cache dependencies uses: actions/cache@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- ``` Now, for this `build` job, we need to set up Java 8 in order to run the Maven command to build the application into a jar file. ```yaml - name: Set up JDK 1.8 uses: actions/setup-java@v3 with: distribution: 'zulu' java-version: 8 ``` After setting up Java 8, we can now build this Mule application with Maven. ```yaml - name: Build with Maven run: mvn -B package --file pom.xml ``` Next, for troubleshooting purposes, we’ll change the name of the generated jar file to include the last commit’s hash. This file name will be visible in CloudHub so you can make sure the application deployed is the correct version. ```yaml - name: Stamp artifact file name with commit hash run: | artifactName1=$(ls target/*.jar | head -1) commitHash=$(git rev-parse --short "$GITHUB_SHA") artifactName2=$(ls target/*.jar | head -1 | sed "s/.jar/-$commitHash.jar/g") mv $artifactName1 $artifactName2 ``` Finally, we will upload this artifact (the jar file) so it becomes available for the next job. ```yaml - name: Upload artifact uses: actions/upload-artifact@v3 with: name: artifacts path: target/*.jar ``` ### jobs: deploy We had mentioned earlier that jobs run asynchronously to each other but there’s a way to configure them to run synchronously. In this case, we need to run the `build` job before the `deploy` job. Otherwise, we won’t have any jar file to deploy to CloudHub. To do this, we use the `needs` field inside the `deploy` job to make sure the `build` job runs before this one. ```yaml needs: build ``` After that, the `runs-on` and the first two steps are the same as the previous job. The third step, however, is not to set up Java 8 because we no longer need to build the Mule application. Now, we need to download the artifact (jar file) we uploaded in the last step of the previous job. ```yaml - uses: actions/download-artifact@v3 with: name: artifacts ``` After we download it, we can now deploy it to CloudHub using Maven. This next step will take the two Action secrets we had previously set up in the Set up your credentials section: `ANYPOINT_PLATFORM_PASSWORD` and `ANYPOINT_PLATFORM_USERNAME`. ```yaml - name: Deploy to Sandbox env: USERNAME: ${{ secrets.anypoint_platform_username }} PASSWORD: ${{ secrets.anypoint_platform_password }} #DECRYPTION_KEY: ${{ secrets.decryption_key }} run: | artifactName=$(ls *.jar | head -1) mvn deploy -DmuleDeploy \ -Dmule.artifact=$artifactName \ -Danypoint.username="$USERNAME" \ -Danypoint.password="$PASSWORD" # -Ddecryption.key="$DECRYPTION_KEY" ``` You will notice that there are two commented-out lines here. If your Mule application uses any sort of key to decrypt secured properties, this is where you can add it. Just make sure you create a `DECRYPTION_KEY` secret too. ## Run the pipeline Since we set up our pipeline to run every time there’s a new commit into the **main** branch, we can simply make a change to the current Mule application and commit it into the branch. Once this is done, the pipeline will be triggered. You can go into the **Actions** tab to see the workflows. ![GitHub Actions tab showing the "Build and Deploy to Sandbox" workflow with its list of runs.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-6.png) If you want to see a specific run, you can click on the title of the commit (for example, **Update `README.md`**). This will show the specifics of the pipeline run, like the jobs it ran and whether they were successful or not. ![Successful workflow run summary with the build and deploy jobs both passing in 3m 9s.](../../assets/blog/how-to-set-up-a-ci-cd-pipeline-to-deploy-your-mulesoft-apps-to-cloudhub-using-github-actions-7.png) You can also click on each specific job to get a closer look at the steps and the full logs of each step. This is helpful to troubleshoot when something goes wrong in the run. ## More resources You can check out my [GitHub profile](https://github.com/alexandramartinez) for more CI/CD repos: - [github-actions](https://github.com/alexandramartinez/github-actions) to deploy a Mule app to CloudHub - [dataweave-utilities-library](https://github.com/alexandramartinez/dataweave-utilities-library) to publish a DataWeave library to Exchange - [api-catalog-cli-example](https://github.com/alexandramartinez/api-catalog-cli-example) to update APIs in Exchange using the API Catalog CLI If you want more details about CI/CD and how I got to create this repo with these settings, you can check out the [CI/CD collection](https://www.twitch.tv/collections/uMqro6u7JRcMWQ) from our Twitch channel. > [!NOTE] > The initial versions of the pipeline are based on the following repository created by Archana Patel: [arch-jn/github-actions-mule-cicd-demo](https://github.com/arch-jn/github-actions-mule-cicd-demo). I hope this was helpful! Don't forget to subscribe so you don't miss any future content. -alex --- ## How to move your code from the DataWeave Playground to Visual Studio Code Source: https://prostdev.com/post/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code | Published: Jan 17, 2023 | Category: Tutorials ![Photo by Safar Safarov on Unsplash.](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-1.png) If you’re not familiar with the online DataWeave Playground yet, this is an online tool that you can access from your browser to develop DataWeave code without having to open Anypoint Studio. It is mainly recommended for training purposes because of its limitations - since it’s running from your browser, it’s limited to the amount of memory allocated in your browser. There is a workaround to run it locally with the Docker image (see [How to run locally the DataWeave Playground Docker Image](https://www.prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image)) but this image might stop being supported in the future. That’s why it’s best to start using Visual Studio Code if you want to develop more complex code and need more performance for bigger payloads. Here’s a video to complement your learnings from this article: ## Advantages Here are a few advantages of using the DataWeave extension for Visual Studio Code instead of the DataWeave Playground (there are more, but these seemed more important for now): - Handle bigger payloads - Get code suggestions - Create unit tests for your code - Create documentation for your code - Debug your scripts ## Install the extension on VSCode Before starting, make sure your extension on VSCode has been correctly installed. You can follow the next steps or see a detailed guide here: [Getting started with the DataWeave extension for Visual Studio Code](https://developer.mulesoft.com/tutorials-and-howtos/dataweave/dataweave-extension-vscode-getting-started/). - Install [Visual Studio Code](https://code.visualstudio.com/Download). - Install [Java 8](https://www.oracle.com/ca-en/java/technologies/javase/javase8-archive-downloads.html) and [Maven](https://maven.apache.org/download.cgi) version 3.6.3 or higher. - Install the [DataWeave extension](https://marketplace.visualstudio.com/items?itemName=MuleSoftInc.dataweave) for Visual Studio Code. And that’s it! ## Export the code from the Playground We are assuming you already have some code going on in the DataWeave Playground. If this is not the case, you can simply [open the Playground](https://dataweave.mulesoft.com/learn/playground) and use the sample code that is automatically generated. 1. From the DataWeave Playground, click on the **Export** button at the top. ![DataWeave Playground Export and Import buttons](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-2.png) 2. This will download a `.zip` file to your computer. 3. Extract the contents of the `.zip` file. This root folder is the one you’ll refer to from VSCode. It contains a `pom.xml` file and a `src` folder. 4. Open VSCode and select **File > Open Folder** ![VS Code File menu with Open Folder highlighted](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-3.png) 5. Select the root folder from step 3. 6. You should now have a structure like the following one. ![VS Code Explorer tree of the exported project showing main.dwl, payload.json, and pom.xml](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-4.png) ## Configure your layout 1. You can open both files (`payload.json` and `main.dwl`) and drag-and-drop them to where you want them to be. This will give you a similar layout as the Playground. ![VS Code with payload.json and main.dwl arranged side by side like the Playground layout](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-5.png) 2. You can click on the **Explorer** button on the far left if you want to minimize that section. ![Same VS Code layout with the Explorer panel collapsed for more editing space](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-6.png) 3. Click on the `main.dwl` window and click on the **Run Preview** button to see the output. ![main.dwl editor with the DataWeave Run Preview button highlighted in the toolbar](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-7.png) 4. Success! ![VS Code showing payload, script, and a Preview Output panel rendering "Hello world!"](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-8.png) > [!NOTE] > If you don’t get the correct output, make sure you installed Java 8 and the appropriate version of Maven, as well as the latest version of VSCode. ## Enable AutoPreview If you don’t change the AutoPreview settings, you’ll have to click on the **Run Preview** button every time you make a change to the script or the payload. This is useful if you want to save resources and only run the code when you need to update the preview. If you want to enable AutoPreview so it updates automatically when you make changes to the script (as it currently does in the DataWeave Playground), simply: - Open the `main.dwl` file - Right-click on the window - Select **DataWeave: Enable AutoPreview** ![Right-click menu in main.dwl with DataWeave: Enable AutoPreview selected](../../assets/blog/how-to-move-your-code-from-the-dataweave-playground-to-visual-studio-code-9.png) That’s it! You can disable it again if you want using the **Disable** button :) ## More resources As mentioned before, there are a lot more advantages to using the VSCode extension that we didn’t list here. Check out the following resources for more information: - [DataWeave landing page](https://dataweave.mulesoft.com/) - [Developer tutorial] [Getting started with the DataWeave extension for Visual Studio Code](https://developer.mulesoft.com/tutorials-and-howtos/dataweave/dataweave-extension-vscode-getting-started/) - [Developer tutorial] [Getting started with DataWeave libraries in Anypoint Exchange](https://developer.mulesoft.com/tutorials-and-howtos/dataweave/dataweave-libraries-in-exchange-getting-started/) - [Short video tutorial] [How to export a script from the DataWeave Playground to Visual Studio Code](https://youtu.be/_8kCFBdgX5A) - [Twitch live stream] [Exploring products: DataWeave extension for VSCode](https://www.twitch.tv/videos/1525009818) --- ## MuleSoft's Anypoint Flex Gateway logs integration with Grafana Loki Source: https://prostdev.com/post/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki | Published: Aug 30, 2022 | Category: Tutorials I’ve been writing about the MuleSoft Flex gateway for the past months, and you can find it here: [https://medium.com/@rcarrascosps](https://medium.com/@rcarrascosps) But one of the most common things to take care of is about consolidating logs and making them part of a greater initiative where you need to have a single pane of glass where you can get information about the health of your system, but also about your services, APIs, microservices, etc. The Flex gateway is going to be part of a greater solution, it will be very improbable that it will be seating just by itself. If you are using it is because you are managing multiple APIs in multiple places. And it is also very probable that your transactions are a mix of APIs calls that goes from one API to another which makes it complicated to trace, diagnose, to monitor. MuleSoft Flex gateway is a very lightweight piece of software that you can configure in both connected mode and local mode. Where the connected mode has all the advantages of managing your APIs from Anypoint Platform, which makes it very appealing. ![Anypoint API Manager dashboard with Total Requests, Policy Violations, Errors and Response Time charts](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-2.png) And the local mode is where you do not manage your APIs from Anypoint Platform, but directly in the command line of the system (Linux, Kubernetes, docket) where you’ve installed the gateway. To understand more about those modes you can take a look at my articles or the official documentation: [https://docs.mulesoft.com/gateway/1.1/flex-gateway-overview](https://docs.mulesoft.com/gateway/1.1/flex-gateway-overview) This series of articles (this is the first one) is targeted to integrate the Flex Gateway with Grafana Labs. In particular with Loki and Tempo. And this first article is related to Loki. Grafana Loki is part of the different components that Grafana Labs offer for log consolidation. It can be integrated with a lot of popular open-source and enterprise solutions such as Elasticsearch (logstash), Splunk, etc. You can learn from Grafana Loki here: [https://grafana.com/oss/loki/](https://grafana.com/oss/loki/) What we will learn in this short article is how to send the Flex Gateway logs (at least the agent log) into Grafana Loki and then analyze them, search them, and explore them from the Grafana console. ## Prerequisites The prerequisites for following this article are - Anypoint Platform instance - Install a Flex Gateway either in connected or local mode - Have access to the Flex Gateway logs - Grafana Cloud free-tier account - Promptail agent to point it to Flex Gateway - And that’s it Let’s go step by step and first install a Flex Gateway into a Linux System. ## Install a Flex Gateway instance Go to [https://anypoint.mulesoft.com](https://anypoint.mulesoft.com/), head into Runtime Manager, and click on Flex Gateways. ![Runtime Manager left menu with the Flex Gateways item highlighted](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-3.png) Then add a gateway and choose Linux (I will use that option for now). ![Add a Flex Gateway OS picker with Linux, Docker and Kubernetes options, Linux selected](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-4.png) And then follow the instructions listed there, which are the following. > [!NOTE] > these commands apply to Flex Gateway version 1.1.0. If you want to work on a different version, the instructions might be different. ```bash curl -XGET https://flex-packages.anypoint.mulesoft.com/ubuntu/pubkey.gpg | sudo apt-key add - echo "deb https://flex-packages.anypoint.mulesoft.com/ubuntu $(lsb_release -cs) main" \ | sudo tee /etc/apt/sources.list.d/mulesoft.list sudo apt update sudo apt install -y flex-gateway ``` Then execute this and change <gateway-name> with any name you choose. The token and organization IDs are going to be different than the ones listed here because they have your own credentials: ```bash sudo flexctl register \ --token=ffb21bee-a718-77d56750e88 \ --organization=f7-4c08-96a3-c9013a01d871 \ --connected=true \ --output-directory=/usr/local/share/mulesoft/flex-gateway/conf.d ``` And finally, start it and check its status: ```bash sudo systemctl start flex-gateway systemctl list-units flex-gateway* ``` Pretty much that should take less than 4 minutes. Just that simple. ## Get access to the Flex Gateway logs Now that the gateway is installed, you can check from the Linux system the log files of it. Let’s focus on one specific which is the gateway log file. And in the next articles, I will focus on the specific APIs logs. Every API can be configured to have its log file, and we will see it in our next article. But to keep things simple we will use the Gateway log. To get access to it, create a directory. For example: ```bash mkdir $HOME/flex-gateway/logs ``` Inside this directory, execute: ```bash journalctl -u flex-gateway-* > flexlog.log ``` You will have the log file in $HOME/flex-gateway/logs/flexlog.out And you will get something like this: ![Flex Gateway agent log lines captured from journalctl into the flexlog.log file](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-5.png) Now let’s configure Grafana Loki. ## Grafana Loki configuration You need a Grafana free tier account. You can get it here: [https://grafana.com/products/cloud/](https://grafana.com/products/cloud/) Once you get your account, go to the cloud console (for example [https://grafana.com/orgs/](https://grafana.com/orgs/)<YourOrganizationName>) And there, click on this: ![Grafana Cloud Stack console with Grafana Launch and a highlighted Loki Send Logs button](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-6.png) Click on the Send Logs blue button and follow these instructions: ![Grafana Promtail setup instructions showing the config.yaml and the docker run command](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-7.png) What we’ve just done is configure the promtail agent to send logs into Grafana Loki. That is a very lightweight agent that in this case is running inside docker. The configuration is very simple, we just need to identify the directory of the log files that we want to send to Loki, and that’s it: ```bash docker run --name promtail --volume "$PWD/promtail:/etc/promtail" --volume "$HOME/flex-gateway/logs:/var/log" grafana/promtail:master -config.file=/etc/promtail/config.yaml ``` The output for that is: ![Promtail container startup logs showing it tailing /var/log/flexlog.log](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-8.png) And if we go to Grafana Explorer, and select the Loki logs data source: ![Grafana Explore query builder with the Loki logs data source and a filename label filter](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-9.png) Write that simple query in the builder mode or the coding mode: ![LogQL query in the Log browser filtering on filename equals /var/log/flexlog.log](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-10.png) And you will get this: ![Grafana Explore showing the Flex Gateway log lines and volume graph returned by the query](../../assets/blog/mulesoft-s-anypoint-flex-gateway-logs-integration-with-grafana-loki-11.png) And now you can start to imagine how much you can do, once you have it in Grafana. As I’ve mentioned, this is a very simple example. The true value will be when we start reading the APIs’ specific log files. That will be in the second article of this series. --- ## Spring Module Integration in a Mule Application (Mule 4) Source: https://prostdev.com/post/spring-module-integration-in-a-mule-application-mule-4 | Published: Jul 26, 2022 | Category: Tutorials The Spring module enables Mule apps to use the Spring framework. In this article, we will use a database datasource that is created by spring beans. Also, we will invoke one bean’s method from a Mule flow. ## Step by step 1 - **Create Mule Project**: Go to Anypoint Studio and create a new Mule project. 2 - **Add spring module**: In Anypoint Studio, the Spring module is provided in the default configuration. In the Mule Palette, click on add modules, search for Spring and add this module to your project. ![Mule Palette Add Modules view with Spring listed among the available modules](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-2.png) 3 - **Spring config**: Go to spring config, add name as `Spring_Config` and provide files as `beans.xml` ![Spring Config named Spring_Config with beans.xml set as its configuration file](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-3.png) 4 - **Spring Beans**: Create a `beans.xml` file under `src/main/resources`. Add the below configuration in this file. ```xml ``` 5 - **Spring JDBC**: Add spring JDBC and PostgreSQL dependency and shared library in `pom.xml` ```xml org.springframework spring-jdbc 5.3.2 org.postgresql postgresql 42.3.4 ``` Add below shared library in the shared libraries tag. ```xml org.springframework spring-jdbc org.postgresql postgresql ``` 6 - **Spring JDBC Beans**: Add spring Datasource bean configuration in `beans.xml`. ```xml ``` 7 - **Application Properties**: Add below application properties for JDBC connection in `src/main/resources`. You can replace the correct username, password, and database name in the below properties. ```properties spring.datasource.url=jdbc:postgresql://localhost:5432/spring spring.datasource.username=username spring.datasource.password=password ``` 8 - **Database Config**: Go to Mule Palette and drag the database connector. Configure the database configuration in `global.xml` with the datasource and select the PostgreSQL jar for the JDBC driver. ![Database Config using a Data Source Reference Connection pointing to the dataSource bean](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-4.png) Test the configuration, this should return a successful connection. 9 - **HTTP Listener for accounts**: Add an HTTP listener and configure it with the default settings. Add a Select DB configuration and a Select query for the accounts table. Make sure the account table is created in your database and there will be at least one entry for the account table. Add a transformer to give the result in JSON format. ![Mule flow with Listener, a database Select, and Transform Message returning JSON](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-5.png) 10 - **Invoke**: Run the mule application and invoke the endpoint to see the result. This will return records from the accounts table. ![Postman GET to /accounts returning 200 OK with a JSON account record from the database](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-6.png) 11 - **Create Spring Beans**: Create User POJO class, UserService interface, and UserServiceImpl service implementation class. ```java title="User.java" package com.shyam.model; public class User { private String firstName; private String lastName; private String email; public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } public String getEmail() { return email; } public void setEmail(String email) { this.email = email; } } ``` ```java title="UserService.java" package com.shyam.service; public interface UserService { public String sayHello(); } ``` ```java title="UserServiceImpl.java" package com.shyam.service; import com.shyam.model.User; public class UserServiceImpl implements UserService { private User user; @Override public String sayHello() { return "Hello from " + user.getFirstName(); } public User getUser() { return user; } public void setUser(User user) { this.user = user; } } ``` 12 - **Spring Bean Configuration**: Add below spring bean configuration for User and UserServiceImpl class in `beans.xml`. ```xml ``` 13 - **Mule flow**: From a Mule flow it's very simple to access one of the previously created beans. It just uses an Invoke component to call the Spring bean’s function. ![Mule flow with Listener, a Java Invoke component, and a Logger](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-7.png) 14 - **JAVA Invoke**: In the Invoke component in the Mule application, we are simply calling the sayHello() method of the UserServiceImpl. Here is the screenshot of the configuration of the Invoke component: ![Invoke component config calling sayHello() on the userService instance of UserService](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-8.png) 15 - **Invoke**: Invoke the endpoint and see this will return Hello from Shyam, which we have configured the first name in beans property. ![Postman GET to /spring returning 200 OK with the body "Hello from Shyam"](../../assets/blog/spring-module-integration-in-a-mule-application-mule-4-9.png) ## GitHub repository - [https://github.com/shyamrajprasad/mule-spring-integration](https://github.com/shyamrajprasad/mule-spring-integration) ## References - [https://docs.mulesoft.com/spring-module/1.3/](https://docs.mulesoft.com/spring-module/1.3/) - [https://dzone.com/articles/using-spring-beans-in-mule](https://dzone.com/articles/using-spring-beans-in-mule) - [https://shyamrajprasad.medium.com/spring-module-integration-in-mule-application-8de5a08303e6](https://shyamrajprasad.medium.com/spring-module-integration-in-mule-application-8de5a08303e6) --- ## JWT token creation using DataWeave Source: https://prostdev.com/post/jwt-token-creation-using-dataweave | Published: Jul 19, 2022 | Category: Tutorials Security implementations have been revolutionary through OAuth 2.0, OpenID Connect, SAML, etc. OAuth 2.0 and OpenID connect mostly use JWT as a token format. JWT is a very familiar term for the API fraternity. There are instances where we need to create a [JWT token](https://jwt.io/introduction) to authorize our APIs to a service or authorize any client if we are using any custom solutions for authentication/authorization. Let’s take a deeper look at JWT tokens. ## Sample JWT token ``` ewogICJhbGciOiAiSFMyNTYiLAogICJ0eXAiOiAiSldUIiwKICAia2lkIjogIjEwMSIKfQ.ewogICJpc3MiOiAiR1RBIiwKICAiaWF0IjogMTY1NjQyMTQ0NiwKICAicmVxdWVzdGVkU2NvcGUiOiBbCiAgICAieG90cCIKICBdCn0.l2slJ86T7J3at9UG5esKMi5B9h02WjcpIuMZm_5mxzM ``` Let's see the basic structure of this token. (Visit [https://jwt.io/#debugger-io](https://jwt.io/#debugger-io)) ![jwt.io debugger decoding the token into its header, payload and signature parts](../../assets/blog/jwt-token-creation-using-dataweave-2.png) As you can see in the image, the token is decoded into three parts, 1. **Header** ``` ewogICJhbGciOiAiSFMyNTYiLAogICJ0eXAiOiAiSldUIiwKICAia2lkIjogIjEwMSIKfQ ``` 2. **Payload** ``` ewogICJpc3MiOiAiR1RBIiwKICAiaWF0IjogMTY1NjQyMTQ0NiwKICAicmVxdWVzdGVkU2NvcGUiOiBbCiAgICAieG90cCIKICBdCn0 ``` 3. **Signature** ``` l2slJ86T7J3at9UG5esKMi5B9h02WjcpIuMZm_5mxzM ``` We will see in the following topic how to create this token. ## How to create JWT using Java There are multiple libraries to do this in JAVA: I always try to use the simplest code. ```java import java.nio.charset.StandardCharsets; import java.security.InvalidKeyException; import java.security.NoSuchAlgorithmException; import java.util.Base64; import javax.crypto.Mac; import javax.crypto.spec.SecretKeySpec; import org.json.JSONException; import org.json.JSONObject; public class JWT { public static String encodeString(byte[] bytes) { return Base64.getUrlEncoder().withoutPadding().encodeToString(bytes); } public static String encodeJSON(JSONObject obj) { return encodeString(obj.toString().getBytes(StandardCharsets.UTF_8)); } public static String createJWT(String data, String secret) throws NoSuchAlgorithmException { try { byte[] hash = secret.getBytes(StandardCharsets.UTF_8); Mac sha256Hmac = Mac.getInstance("HmacSHA256"); SecretKeySpec secretKey = new SecretKeySpec(hash, "HmacSHA256"); sha256Hmac.init(secretKey); byte[] signedBytes = sha256Hmac.doFinal(data.getBytes(StandardCharsets.UTF_8)); return encodeString(signedBytes); } catch (InvalidKeyException ex) { System.out.print("FAILED to sign"); return null; } } public static String JWTtoken(String[] args) throws JSONException, NoSuchAlgorithmException { String jwtHeader = "{\n \"alg\": \"HS256\",\n \"typ\": \"JWT\",\n \"kid\": \"101\"\n}"; String jwtPayload = "{\n \"iss\": \"GTA\",\n \"iat\": 1656422976,\n \"requestedScope\": [\n \"xotp\"\n ]\n}"; String secret = "1A3B4A5CE86E0BF3AF6FF575C93438BH"; String signedData = createJWT( encodeJSON(new JSONObject(jwtHeader)) + "." + encodeJSON(new JSONObject(jwtPayload)), secret); String jwtToken = encodeJSON(new JSONObject(jwtHeader)) + "." + encodeJSON(new JSONObject(jwtPayload)) + '.' + signedData; return jwtToken; } public static void main(String args[]) { try { JWTtoken(null); } catch (NoSuchAlgorithmException e) { e.printStackTrace(); } catch (JSONException e) { e.printStackTrace(); } } } ``` The token generated: ``` eyJraWQiOiIxMDEiLCJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJHVEEiLCJyZXF1ZXN0ZWRTY29wZSI6WyJ4b3RwIl0sImlhdCI6MTY1NjQyMjk3Nn0.3kM0_DUuJCIig2OlyqxqnjODUxRvTA93d9Tgu-ZhOgw ``` Let’s decode this token. ![jwt.io decoding the Java-generated token, showing the GTA issuer and xotp scope payload](../../assets/blog/jwt-token-creation-using-dataweave-3.png) ## How to create JWT using DataWeave JAVA seems to be very technical. Let’s switch to our magical language. ```dataweave %dw 2.0 //Imports import * from dw::Crypto import * from dw::core::Binaries import * from dw::core::URL //Variables var secret = "1A3B4A5CE86E0BF3AF6FF575C93438BH" //256 bit key var tokenHeader = { "alg": "HS256", // The algorithm to sign "typ": "JWT", // Type of token "kid": "101" // Key Id } var tokenPayload = { "iss": "GTA", // Issuer of JWT "iat": 1656422976, // now() as Number {unit: "seconds"}, // Time in seconds since epoch, hardcoded to match with JAVA code block "requestedScope": ["xotp"] // Request scope } // Function to encode data into base64 // Note:Web Tokens use Base64Url instead of the typical Base64. They are basically the same except Base64Url are safe to pass in a URL because they use – instead of + and _ instead of / and they omit the = padding characters at the end of the string. You can do string replacement to properly convert the token then call fromBase64. // To read more on encoding/decoding base64 formats visit - https://docs.mulesoft.com/dataweave/2.4/dw-binaries-functions-tobase64 fun base64encodeURL(data) = toBase64(data) replace "/" with ("_") replace "+" with ("-") replace "=" with "" //Variables var header = base64encodeURL(write(tokenHeader, "application/json")) // Convert to Stringified JSON var payload = base64encodeURL(write(tokenPayload, "application/json")) // Convert to Stringified JSON var signature = base64encodeURL(HMACBinary(secret as Binary, (base64encodeURL(header) ++ "." ++ base64encodeURL(payload)) as Binary, "HmacSHA256")) output application/json --- header ++ "." ++ payload ++ "." ++ signature ``` The token generated: ``` ewogICJhbGciOiAiSFMyNTYiLAogICJ0eXAiOiAiSldUIiwKICAia2lkIjogIjEwMSIKfQ.ewogICJpc3MiOiAiR1RBIiwKICAiaWF0IjogMTY1NjQyMjk3NiwKICAicmVxdWVzdGVkU2NvcGUiOiBbCiAgICAieG90cCIKICBdCn0._RUefLwi3UBP7jbxE7VHB-t-aMmCNdvFSr7frgW7wNY ``` Let’s decode this token: ![jwt.io decoding the DataWeave-generated token to the same header and payload data](../../assets/blog/jwt-token-creation-using-dataweave-4.png) Voila, you are done. TBH, it's way more fun to code in DataWeave. The tokens generated from the JAVA class and DataWeave get decoded to the same data. ## Summary JWT is vital in today’s API world. It's not enough to just know the code; we need to focus on the security of API from all perspectives. ## References - [https://jwt.io/introduction](https://jwt.io/introduction) - [https://docs.mulesoft.com/dataweave/2.4/dw-crypto](https://docs.mulesoft.com/dataweave/2.4/dw-crypto) --- ## How to Integrate AWS Lambda with MuleSoft Source: https://prostdev.com/post/how-to-integrate-aws-lambda-with-mulesoft | Published: Jul 12, 2022 | Category: Tutorials In this blog post, I will demonstrate how to Integrate AWS Lambda with MuleSoft. The GitHub repository with the Mule Project can be found at the end of the post. AWS Lambda is an on-demand cloud computing resource offered as a function-as-a-service by AWS. AWS Lambda allows you to add custom logic to AWS resources such as Amazon S3 buckets and Amazon DynamoDB tables, so you can easily apply it to compute data as it enters or moves through the cloud. AWS Lambda was designed for use cases such as image or object uploads to [Amazon S3](https://en.wikipedia.org/wiki/Amazon_S3), updates to [DynamoDB](https://en.wikipedia.org/wiki/DynamoDB) tables, responding to website clicks, or reacting to sensor readings from an [IoT](https://en.wikipedia.org/wiki/Internet_of_Things)-connected device. AWS Lambda can also be used to automatically provision back-end services triggered by custom [HTTP requests](https://en.wikipedia.org/wiki/HTTP_request), and "spin down" such services when not in use, to save resources. Anypoint Connector for Amazon Lambda (Amazon Lambda Connector) enables you to execute AWS Lambda operations within your Mule flows. You can then use the response of the Lambda operation to process another Mule flow as needed. In this article, we will see how we can use the Anypoint Lambda Connector to Integrate. For more information on Amazon Lambda Connector 1.0 - Mule 4, refer to this [link](https://docs.mulesoft.com/amazon-lambda-connector/1.0/). ## 1. Create an AWS Free tier account Open the link [https://aws.amazon.com/console/](https://aws.amazon.com/console/). The below page opens up: ![AWS Management Console landing page with a Sign In to the Console button](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-2.png) Click **Sign In to the Console** and then create a new AWS account. ![AWS sign-in page with the Create a new AWS account button highlighted](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-3.png) Provide the required details and click on **Verify email address.** ![Sign up for AWS form asking for root user email address and AWS account name](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-4.png) A verification code will be sent to your email. Enter the code and click on **Verify**. ![AWS "Verify your email address" email containing a verification code](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-5.png) ![AWS sign-up "Confirm you are you" screen with a field to enter the verification code](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-6.png) Once the verification is successful, provide the password and click on **continue (step 1 of 5)**. ![AWS sign-up Create your password step after the email was successfully verified](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-7.png) Provide all the contact information and click on **continue (step 2 of 5)**. ![AWS sign-up Contact Information form for full name, phone number, and address](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-8.png) It asks for credit card information. You can safely provide as you won't be charged unless you exceed the free tier limit. For more information on this please watch this video on YouTube ([AWS Free Tier Overview](https://www.youtube.com/watch?v=Qou3oKcvyww&t=3s)). After this step click on **Create Account**. After this sign in to the AWS console by selecting the Root user. ![AWS sign-in page with the Root user option selected](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-9.png) You will now be logged in to the AWS Console as shown below: ![AWS Console Home showing recently visited services including IAM, Lambda, and S3](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-10.png) ## 2. Manage Access to AWS resources First, select the region of your choice from the list given below: ![AWS region dropdown open, listing US and Asia Pacific regions](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-11.png) Click on **Services** -> **Security, Identity & Compliance** -> **IAM.** ![AWS Services menu open on Security, Identity and Compliance with IAM listed](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-12.png) Click on **Users** and on **Add Users.** ![IAM Users list showing one user with an Add users button](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-13.png) Provide the User Name and Select AWS Credential type as shown below: ![IAM Add user step 1 setting a user name and selecting programmatic access key](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-14.png) Click on **Next: Permissions** and on **Attach existing policies directly.** ![IAM permissions step with Attach existing policies directly and AWSLambda search](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-15.png) Click on **AWSLambda_FullAccess** and **Next: Tags.** ![Policy list with AWSLambda_FullAccess checkbox selected](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-16.png) Click on **Review** and click on **Create User.** ![IAM Review screen summarizing the user with AWSLambda_FullAccess attached](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-17.png) The User is created successfully. Make a note of the Access Key ID and Secret access key as you cannot retrieve them again. ![IAM success screen showing the new user's Access key ID and Secret access key](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-18.png) ![IAM user detail page showing AWSLambda_FullAccess attached directly](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-19.png) ## 3. Create a Lambda Function in AWS In the AWS console search Lambda and click on the search result Lambda. ![AWS console search for Lambda showing the Lambda service result](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-20.png) Click on **Create Function.** ![AWS Lambda Functions page with a Create function button](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-21.png) I used a simple “hello world” blueprint. ![Create function screen with Use a blueprint and hello-world blueprint selected](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-22.png) Provide all the basic information and click on **Create Function**. ![Lambda Basic information form setting a function name and execution role](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-23.png) ![Lambda Functions list showing the created test-hello-world-mule-poc function on Node.js 12.x](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-24.png) ![Lambda function overview page showing the function ARN and Add trigger options](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-25.png) The Source Code of the function can be seen below: ![Lambda Code source editor showing the index.js hello world handler](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-26.png) You can execute the script by clicking on **Test** and the execution result is as shown below: ![Lambda test execution result showing status Succeeded with function logs](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-27.png) ## 4. Configure Amazon Lambda Connector in MuleSoft First, we need to create a new project in Anypoint Studio and manage dependencies as shown below: ![Anypoint Studio project right-click menu opening Manage Dependencies then Manage Modules](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-28.png) Add the **Amazon Lambda Connector** dependency -> **Finish** -> **Apply and close.** ![Add Dependencies dialog with Amazon Lambda Connector 1.0.4 added to selected modules](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-29.png) Now, this dependency will be added to the `pom.xml`. ![pom.xml showing the mule4-amazon-lambda-connector dependency added](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-30.png) You will now be able to see all the Amazon Lambda Connectors available in the Mule Palette. ![Mule Palette search for Lambda listing Get Function, Invoke, Invoke Async and List Functions](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-31.png) ## 5. How to use the Invoke, Get Function, and List Functions Lambda Connectors More information on these connectors can be found at [https://docs.mulesoft.com/amazon-lambda-connector/1.0/amazon-lambda-connector-reference](https://docs.mulesoft.com/amazon-lambda-connector/1.0/amazon-lambda-connector-reference) ### Invoke Connector "Invokes a Lambda function. You can invoke a function synchronously (and wait for the response), or asynchronously. To invoke a function asynchronously, set InvocationType to Event. For synchronous invocation, details about the function response, including errors, are included in the response body and headers. For either invocation type, you can find more information in the execution log and trace." More information at [this link](https://docs.mulesoft.com/amazon-lambda-connector/1.0/amazon-lambda-connector-reference#Create20150331FunctionsInvocationsByFunctionName). ![Mule flow with Invoke operation configured with function name and request-response invocation type](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-32.png) The access key and the secret key are obtained as shown in the “Manage Access to AWS resources” step. ![Amazon Lambda Connector global config with Access Key, Secret Key and us-east-1 region](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-33.png) Now let us test this using Postman. ![Postman calling lambdaInvoke with a JSON body returning value1 and a 200 OK status](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-34.png) ### Get Function Connector "Returns information about the function or function version, with a link to download the deployment package that’s valid for 10 minutes. If you specify a function version, only details that are specific to that version are returned." More information at [this link](https://docs.mulesoft.com/amazon-lambda-connector/1.0/amazon-lambda-connector-reference#Get20150331FunctionsByFunctionName). ![Mule flow with the Get Function operation configured with the function name](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-35.png) The response from the Get Function is as shown below. ![Mule Debugger showing the Get Function JSON response with function configuration details](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-36.png) ![Postman calling lambdaGet returning the function Configuration JSON](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-37.png) ### List Functions Connector "Returns a list of Lambda functions, with the version-specific configuration of each function. Lambda returns up to 50 functions per call. Set FunctionVersion to ALL to include all published versions of each function in addition to the unpublished version. The ListFunctions action returns a subset of the FunctionConfiguration fields." More information at [this link](https://docs.mulesoft.com/amazon-lambda-connector/1.0/amazon-lambda-connector-reference#Get20150331Functions). ![Mule flow with the List Functions operation configured with empty optional fields](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-38.png) ![Postman calling lambdaList returning a Functions array of Lambda functions](../../assets/blog/how-to-integrate-aws-lambda-with-mulesoft-39.png) ## Conclusion This is how we can easily create functions in AWS Lambda and Integrate with MuleSoft using the **Amazon Lambda Connectors**. ## Best practices For the purpose of simplification, I have mentioned the configuration details directly. But it is always recommended to externalize these details and encrypt the sensitive data. ## GitHub repository - [https://github.com/ProstDev/aws-lambda-mulesoft](https://github.com/ProstDev/aws-lambda-mulesoft) Thanks for reading my article and I hope it will be of some use! See you in the next post! -Pravallika --- ## Tracing Module in Mule 4 Source: https://prostdev.com/post/tracing-module-in-mule-4 | Published: Jul 5, 2022 | Category: Tutorials The tracing module enables you to enhance your logs by adding, removing, and clearing all variables from the logging context for a given Mule event. It also enables you to modify the correlation ID during flow execution. ## Uses Of Tracing Module - Clear all the logging variables from the event logging context. - Remove a logging variable from the logging context. - Set logging variables to logging context. - Modify the correlation ID during flow execution ![Mule Palette listing Tracing module operations: Clear, Remove, Set logging variable, and With CorrelationID](../../assets/blog/tracing-module-in-mule-4-2.png) ## Tracing Module Application 1 - **Create Mule application**: Go to Anypoint studio and create a mule project with the name `mulesoft-tracing-module-integration`. 2 - **Add Tracing module**: In the mule palette view, click on **+ Add Modules** and search for tracing modules. Drag the Tracing module to your modules. 3 - **HTTP Listener**: Create an HTTP listener with the default configuration and specify the `/trace` base path. 4 - **MDC Logging**: Mapped Diagnostic Context (MDC) enriches logging and improves tracking by providing more context or information in the logs for the current Mule event. By default, Mule logs two MDC entries: `processor`, which shows the location of the current event; and `event`, which shows the correlation ID of the event. Open the `log4j.xml` file and replace `[processor: %X{processorPath}; event: %X{correlationId}]` with `[%MDC]`. 5 - **Create set variable logging**: Create a set logging variable for `customerId` and request path as below. In the logger, we will print the payload. ![Mule flow with Listener, Set logging variable for customerId and requestPath, then a Logger](../../assets/blog/tracing-module-in-mule-4-3.png) ```xml ``` 6 - **Run application**: Run the application and test it out with the below command. ```bash curl --location --request POST 'http://localhost:8081/trace' \ --header 'Content-Type: application/json' \ --data-raw '{ "orderId": 54810284, "customerId": "CUST-12934", "items": [ "P-123", "P-452" ] }' ``` 7 - **Verify logs**: Verify the logs `customerId` and request path will be printed in the context path logs. The below log will be printed in the file location mentioned in `log4j.xml`. ```text INFO 2022-05-08 21:46:29,898 [[MuleRuntime].uber.07: [mulesoft-tracing-module-integration].mulesoft-tracing-module-integrationFlow.CPU_LITE @69de6b8b] [{correlationId=37e4bf90-ceea-11ec-9c4a-f84d89960c47, customerId=ARG-12934, processorPath=mulesoft-tracing-module-integrationFlow/processors/2, requestPath=POST:/trace}] logging variables: { "orderId": 548102842, "customerId": "ARG-12934", "items": [ "CP-123", "CP-452" ] } ``` > [!NOTE] > You will see different logs in the console with `processor` and `event`. 8 - **Console log4j changes**: Open `log4j.xml`, add Console logging in Appenders tag. Also, add this reference to the AsyncRoot tag. ```xml ``` 9 - **Run application**: Re-run the Mule application and verify the logger. The same logging will be printed in the console. 10 - **Remove logging variable**: After the logger, let’s remove one of the logging variables i.e request path. Also after this, remove the logging variable, and add the logger for payload. ```xml ``` ![Mule flow extended with a Remove logging variable for requestPath followed by another Logger](../../assets/blog/tracing-module-in-mule-4-4.png) 11 - **Run application and verify**: Re-run the application, hit the endpoint, and verify the logs, you will see the request path will be removed from the context path. When you will hit the application multiple times, you will see different correlation IDs. ```text INFO 2022-05-08 22:08:11,518 [[MuleRuntime].uber.09: [mulesoft-tracing-module-integration].mulesoft-tracing-module-integrationFlow.CPU_LITE @11ace7f9] [{correlationId=3fd22371-ceed-11ec-9c4a-f84d89960c47, customerId=ARG-12934, processorPath=mulesoft-tracing-module-integrationFlow/processors/4}] logging without request path: { "orderId": 548102842, "customerId": "ARG-12934", "items": [ "CP-123", "CP-452" ] } ``` 12 - **Clear logging variable**: After the payload logger, let's add a clear logging variable, it will remove all the logging variables which were created during the flow. ![Mule flow with a Clear logging variables component added before the final Logger](../../assets/blog/tracing-module-in-mule-4-5.png) 13 - **Re-run application and verify**: Run the application again and hit the endpoint and verify the logs. There will be no `customerId` logging variable. ```text INFO 2022-05-08 22:08:11,520 [[MuleRuntime].uber.09: [mulesoft-tracing-module-integration].mulesoft-tracing-module-integrationFlow.CPU_LITE @11ace7f9] [{correlationId=3fd22371-ceed-11ec-9c4a-f84d89960c47, processorPath=mulesoft-tracing-module-integrationFlow/processors/6}] no logging variable :: { "orderId": 548102842, "customerId": "ARG-12934", "items": [ "CP-123", "CP-452" ] } ``` 14 - **With Correlation ID**: Let’s add with correlation id after payload logger, and change correlation Id. I have changed the correlation id to append my name. Add the payload logger into this too. ![With CorrelationID config setting Correlation id to the expression correlationId ++ "shyam"](../../assets/blog/tracing-module-in-mule-4-6.png) 15 - **Run the application**: Re-run the application, hit the URL again, and verify the logs with the correlation logger. This updated correlation id will be visible only for the logger available under the correlation ID. ```text INFO 2022-05-08 22:20:33,690 [[MuleRuntime].uber.02: [mulesoft-tracing-module-integration].mulesoft-tracing-module-integrationFlow.CPU_LITE @3b8d37c5] [{correlationId=fa14acc0-ceee-11ec-b249-f84d89960c47: shyam, processorPath=mulesoft-tracing-module-integrationFlow/processors/7/processors/0}] with changed correlation id:: { "orderId": 548102842, "customerId": "ARG-12934", "items": [ "CP-123", "CP-452" ] } ``` 16 - **Add payload logger**: Add payload logger at the last, you can re-run the application and the updated correlation id will not be available to this logger. ```text INFO 2022-05-08 22:20:33,696 [[MuleRuntime].uber.04: [mulesoft-tracing-module-integration].mulesoft-tracing-module-integrationFlow.CPU_LITE @3b8d37c5] [{correlationId=fa14acc0-ceee-11ec-b249-f84d89960c47, processorPath=mulesoft-tracing-module-integrationFlow/processors/8}] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "orderId": 548102842, "customerId": "ARG-12934", "items": [ "CP-123", "CP-452" ] } ``` ## GitHub repository - [https://github.com/shyamrajprasad/mulesoft-tracing-module-integration](https://github.com/shyamrajprasad/mulesoft-tracing-module-integration) ## References - [https://docs.mulesoft.com/tracing-module/1.0/](https://docs.mulesoft.com/tracing-module/1.0/) --- ## How to Invert a Binary Tree with DataWeave Source: https://prostdev.com/post/how-to-invert-a-binary-tree-with-dataweave | Published: Jun 7, 2022 | Category: Tutorials The [DataWeave language](https://docs.mulesoft.com/mule-runtime/latest/dataweave) is a simple, powerful tool used to query and transform data inside of Mule. But you can also use it to solve algorithmic problems. How to invert a Binary Tree is a popular coding question to test your skills. Let’s try to solve this with DataWeave. ## Definition Ok, but what is a binary tree? Yes, sure! Firstly we define a **tree** as a set of elements (nodes), with the following properties: - Each node can be connected to many children - Each node has one parent, except the root node that has no parent A **binary tree** is a tree where each node has at most 2 children: “left” and “right”. We can define the DataWeave **binary tree** structure in the following way: ```dataweave type Tree= { value: String, right?: Tree, left?: Tree } ``` ## Problem In order to invert a binary tree, we need to swap left and right for every level of the tree, as you can see in the following example: ![Diagram of a 5-node binary tree mirrored so left and right children swap at each level](../../assets/blog/how-to-invert-a-binary-tree-with-dataweave-2.png) ## Solution It's easier than it looks. Thanks to Mr. Recursion! We can solve the problem for the first children by swapping them and then recursively applying the same solution for the left and right sub-tree. **Input** ```json { "value": "2", "left": { "value": "1" }, "right": { "value": "4", "left": { "value": "3" }, "right": { "value": "5" } } } ``` **DataWeave Solution** ```dataweave %dw 2.0 output application/json type Tree= { value: String, left?: Tree, right?: Tree } fun invertTree(node ) : Tree = { value: node.value , ( left: invertTree(node.right )) if (node.right?), ( right: invertTree(node.left )) if (node.left?) } --- invertTree(payload) ``` **Output** ```json { "value": "2", "left": { "value": "4", "left": { "value": "5" }, "right": { "value": "3" } }, "right": { "value": "1" } } ``` > [!NOTE] > This solution is using regular recursive functions. If the recursion level exceeds 255 iterations, you will receive a Stack Overflow error. To avoid this, you can also use tail-recursive functions. To learn more about this, see [this article](https://blogs.mulesoft.com/dev-guides/how-to-tutorials/untie-multilevel-structures-dataweave-recursive-calls/). ## Conclusion Now we know how to use Mr. Recursion in DataWeave in order to solve the invert binary tree problem. Thanks for reading my post and I hope it will help understand this wonderful language. That’s it for now! See you in the next post! -Stefano --- ## Universal API Management, Anypoint Flex Gateway, and API Governance Source: https://prostdev.com/post/universal-api-management-anypoint-flex-gateway-and-api-governance | Published: May 31, 2022 | Category: Guides As you are aware that recently MuleSoft has introduced Universal API Management capabilities via Flex Gateway and API Governance components as part of Anypoint Platform. These two components are very important as a part of API lifecycle management and those will help to manage any APIs using a single platform and ensure that whatever API specs that you are designing are with best practices and all security aspects have been taken care of during API design. ## Universal API Management Universal API Management allows you to manage, govern, or secure the APIs within a single control plane, it doesn’t matter whether they are Mule or non-Mule APIs or where they are located (on-premise, cloud, or anywhere). - It will allow the organizations or enterprises to control, manage, and secure the APIs under a single umbrella. - Adapt any architecture with a lightweight and flexible API Gateway to manage and secure the APIs. - Govern all APIs under a single platform. ## Anypoint Flex Gateway Flex Gateway is ultrafast and manages the APIs running anywhere. - Secure and Manage APIs located anywhere. - Extend Anypoint Platform capabilities to Mule as well as non-Mule APIs. - Achieve consistent security and governance across every API operating in any environment. - Flex Gateway can be used in local or connected mode. **What are the benefits of Flex Gateway?** - Flex gateway is an ultra-fast gateway that can be used for any APIs (MuleSoft or Non-MuleSoft APIs), deployed anywhere (Cloud, Docker, Kubernetes, Customer Hosted, etc.). - Easily manage all the APIs within your organization from a single platform and under a single umbrella. - Extend the Anypoint Platform Capabilities to Mule and non-Mule APIs. - Secure, discover, govern, or engage the APIs (Mule and non-Mule APIs). - Set up the Flex Gateway easily in 2 modes (Local and Connected). - Adapt any architecture with a lightweight and flexible API Gateway to manage and secure the APIs. **Can we apply any out-of-the-box, as well as custom API policies to APIs, published to Flex Gateway?** Yes, you can apply any out-of-the-box as well as custom policies to APIs published to Flex Gateway. You can apply API Manager alerts and view APIs metrics. **Is Flex Gateway part of a 30-day free trial Anypoint Platform account?** Yes, you can find Flex Gateway as part of a 30-day free trial Anypoint Platform account. **Where to find MuleSoft documentation for Flex Gateway overview?** Here is the link: [https://docs.mulesoft.com/gateway/flex-gateway-overview](https://docs.mulesoft.com/gateway/flex-gateway-overview) **How to upgrade Flex Gateway?** Here is the link explaining how to upgrade Flex Gateway: [https://docs.mulesoft.com/gateway/flex-gateway-upgrade](https://docs.mulesoft.com/gateway/flex-gateway-upgrade) **How to uninstall Flex Gateway?** Here is the link explaining how to uninstall Flex Gateway: [https://docs.mulesoft.com/gateway/flex-gateway-uninstall](https://docs.mulesoft.com/gateway/flex-gateway-uninstall) **What is the shared responsibility for Flex Gateway between MuleSoft and you?** Here is the MuleSoft document explaining the shared responsibility model for Flex Gateway: [https://docs.mulesoft.com/gateway/flex-shared-responsibility](https://docs.mulesoft.com/gateway/flex-shared-responsibility) **What authentication mechanism is supported for installing Flex Gateway?** There are three types of authentication mechanisms supported for installing Flex Gateway: - Anypoint Username and Password - Auth Token - Connected App **Where can Flex Gateway be set up?** Flex Gateway can be set up on three different operating systems: - Install Flex Gateway as a Linux Service. - Install Flex Gateway as a Docker Container. - Install Flex Gateway as a Kubernetes Ingress Controller. **What are the different steps for setting up Flex Gateway?** Here is the MuleSoft documentation, showing what commands can be used to set up the Flex Gateway. - [Review Prerequisites](https://docs.mulesoft.com/gateway/flex-review-prerequisites) - [Install Flex Gateway](https://docs.mulesoft.com/gateway/flex-install) - [Run Flex Gateway (Connected Mode)](https://docs.mulesoft.com/gateway/flex-conn-reg-run) - [Run Flex Gateway (Local Mode)](https://docs.mulesoft.com/gateway/flex-local-reg-run) - [Add Replicas (Connected Mode)](https://docs.mulesoft.com/gateway/flex-conn-rep-run) - [Add Replicas (Local Mode)](https://docs.mulesoft.com/gateway/flex-local-rep-run) - [Manage APIs (Connected Mode)](https://docs.mulesoft.com/gateway/flex-conn-manage) - [Manage APIs (Local Mode)](https://docs.mulesoft.com/gateway/flex-local-manage) **Can you use a single Flex Gateway for multiple APIs agnostic of the technology and location where it is deployed?** Yes, we can use a single Flex Gateway for multiple APIs, it doesn’t matter where they are running and in which technologies they have been implemented. Here is a sample architecture showing a single Flex Gateway for multiple APIs. In the above Flex Gateway architecture, we are using a single Flex Gateway in Docker container with multiple replicas to connect multiple APIs implemented in any technology. Flex Gateway replicas actually run in the Docker container in the above architecture and it is registered in Anypoint Platform. **Can you use multiple Flex Gateways for multiple APIs?** Yes, we can use multiple Flex Gateways for multiple APIs deployed anywhere. Here is a sample architecture showing a multiple Flex Gateway for multiple APIs. In the above Flex Gateway architecture, we are using multiple Flex Gateways in Docker containers with multiple replicas to connect multiple APIs implemented in any technology. We are using one Flex Gateway for each API in the above architecture and this can be grouped according to your need and requirements. Flex Gateway replicas actually run in the Docker container in the above architecture and it is registered in Anypoint Platform. Here is the list of videos that will explain how to set up the Flex Gateway in both Connected and Local mode: - [Anypoint Universal API Management and Flex Gateway - Part I | MuleSoft](https://youtu.be/ZG9Si5w-hUA) - [Anypoint Universal API Management and Flex Gateway - Part II | Flex Gateway in Connected Mode](https://youtu.be/t_oevGDU3Ro) - [Anypoint Universal API Management and Flex Gateway - Part III | Manage, Secure API in Connected Mode](https://youtu.be/2dPfR7riEQI) - [Anypoint Universal API Management and Flex Gateway - Part IV | Flex Gateway in Local Mode](https://youtu.be/Ouzt2zyZNkM) - [Anypoint Universal API Management and Flex Gateway - Part V | Manage, Secure API in Local Mode](https://youtu.be/OY2EVsS3SPI) - [Anypoint Universal API Management and Flex Gateway - Part VI | Manage Multiple API in Local Mode](https://youtu.be/2meUTtANR_o) - [Surat MuleSoft Meetup#41 (Flex Gateway and API Governance)](https://youtu.be/ocaeFlBD9dA) ## API Governance MuleSoft has recently introduced API Governance as a part of Anypoint Platform that enables you to apply governance ruleset to your APIs that ensures API Consistency and provides default several rulesets such as a Top 10 OWASP API Security, Anypoint API Best Practices, OpenAPI Best Practices governance rulesets, etc. API Governance will ensure the API designs across the enterprises are consistent and are designed with API best practices and guidelines. This will ensure the security of the API and improve the quality of the APIs. **What are the benefits of API Governance?** - Enable developers to apply governance rulesets at design time. - Produce consistent API specs across the enterprises. - Improved API Quality and Security. - API design with Anypoint best practices and OpenAPI best practices. - Ensure Design-Time conformance. - Reduce Top 10 OWASP security risks. **Are there any default rulesets for API Governance?** API Governance comes with the following default rulesets: - [Anypoint Best Practices](https://anypoint.mulesoft.com/exchange/68ef9520-24e9-4cf2-b2f5-620025690913/anypoint-best-practices/minor/1.0/) - [Authentication Security Best Practices](https://anypoint.mulesoft.com/exchange/68ef9520-24e9-4cf2-b2f5-620025690913/authentication-security-best-practices/minor/1.0/) - [HTTPS Enforcement](https://anypoint.mulesoft.com/exchange/68ef9520-24e9-4cf2-b2f5-620025690913/https-enforcement/minor/1.0/) - [OpenAPI Best Practices](https://anypoint.mulesoft.com/exchange/68ef9520-24e9-4cf2-b2f5-620025690913/openapi-best-practices/minor/1.0/) - [OWASP API Security Top 10 2019 Checklist](https://anypoint.mulesoft.com/exchange/68ef9520-24e9-4cf2-b2f5-620025690913/owasp-api-security/minor/1.0/) - [Required Examples](https://anypoint.mulesoft.com/exchange/68ef9520-24e9-4cf2-b2f5-620025690913/required-examples/minor/1.0/) **How to implement API Governance for the APIs?** The first step for creating API Governance is to create the profile in Anypoint Platform’s API Governance and select what are the rulesets that you need to enable for that profile and also you can add filters and notifications. Filters will ensure which APIs need to be scanned against the profile that we have created. Notifications will generate emails to the users in case the APIs haven't been designed according to rulesets associated with the profile and they will be marked as **Non-Conformant**. There are three statuses maintained for your APIs as part of the API Governance: - **Not Validated** - API is not validated against the API Governance profile. - **Conformant** - API has satisfied the rulesets that were associated with the profile. - **Non-Conformant** - API has not satisfied the rulesets that were associated with the profile. Below is the report generated for your APIs and it will show the status of the APIs with what are the rulesets failed and what are the violations in your APIs. Here is a video showing how to implement Anypoint API Governance: [Anypoint API Governance With MuleSoft](https://youtu.be/BZXbRWJ5Tro). ## Conclusion As we have seen in the above article, how Flex Gateway can be used to manage any APIs agnostic of technologies and platforms. API Governance will ensure that the design of your APIs is consistent, secure, and defined with Anypoint API and OpenAPI best practices and Top 10 OWASP security has been taken care of. These are two important features that have been released recently as a part of Anypoint Platform. Next, you can explore the Flex Gateway and API Governance capabilities within Anypoint Platform. --- ## 3 scenarios to consume an Apache Kafka Topic with MuleSoft Source: https://prostdev.com/post/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft | Published: May 10, 2022 | Category: Guides Apache Kafka is one of the best options in the market for data streaming. Developed by LinkedIn engineers and then opened up for the open-source community, it has been very well received (and used) by many organizations. If you need to move/produce streams of data from one point to another, Apache Kafka is the right option for you. Apache Kafka has simple architecture in terms of how the information is streamed: - Topics - Partitions - Replicas - Brokers Those four concepts are the basis for understanding where and how a producer and/or a consumer can produce/read information from Kafka. ![Kafka architecture diagram: producers feeding brokers with topics and partitions, read by consumer groups, with zookeeper](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-2.png) The architecture itself, as we’ve mentioned, is not complex, but the zookeeper dependency may be one of the challenges when deploying this on your own. That is why [Confluent](https://www.confluent.io/) is a good option for customers who want to avoid the configuration and maintenance challenges that Apache Kafka may bring to the table. Producers are any type of application and/or platform that needs to produce information into Kafka. For example: - Log files of an application that you would like to send to Kafka for further analysis and/or consolidation - Geolocation of your fleet of buses that needs to be processed and analyzed in real-time - Information from your Point-of-sale (PoS) that you would like to process in your central office. Imagine a retailer that needs to send information from every branch to the central. - Tweets analysis. Imagine you need to get information from different Twitter channels - Database tables - Etc. As we’ve depicted in the diagram, the minimum unit of configuration is a partition that belongs to a Topic, and Topics belong to Brokers. Topics can be replicated along with the Brokers. And Brokers can act in a cluster fashion. ## Architecture description Messages are published to Topics’ partitions and marked with an offset, which will be very useful for consumers at the time they start to read from the stream. ![Source: https://kafka.apache.org/documentation.html](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-3.png) Kafka is a very powerful platform, and part of that is that it works pretty smart when dealing with the idea of multiple partitions for a Topic. One of the main Kafka capabilities is the resiliency of the platform as well as the flexibility to read the messages even when one of the brokers may be experiencing an issue, or a consumer is having a problem. As you can see -in the previous image- a message is published into a Topic Partition (as a producer you can decide where to point to) and an offset is generated to mark the messages, which will help us when we want to read the messages. You can have single or multiple producers, producing messages on the same topic and the same or different partitions. It will depend on your design. But let’s not just leave it as a generic idea of “depends on your design,” which is true but this type of article has the objective to help you decide on how to use things, and that’s what we are going to do. This idea of having multiple producers producing messages on the same Topic but under different partitions can be to divide how the consumers will read the information. Let’s get back to the retail scenario where different branches need to send information to the central. Every branch has a group of PoS (Point-of-Sales). The Topic may represent the branch and partitions can represent every PoS. Then, from the central, we can have multiple consumers reading from the same Topic (branch) but pointing to different partitions (PoS), and processing the information individually. Maybe we need to differentiate every single PoS, and therefore we can assign consumers to read from that particular partition. Another idea is that a Topic represents a group of branches (regions for example), and every partition represents a single branch; you may have a specific consumer reading the information of a specific branch. As we’ve mentioned, it will depend on how you design it and that is fully related to your use case. ## Getting deep into the consumers' mechanics Let’s think about it from the consumer perspective and let’s continue elaborating on the retail scenario. If we think of a large chain of supermarkets with thousands of branches around the country, and the need is to process every single ticket that is generated by every single branch, and ultimately send it to the central, then the volume of messages will be something relevant in this system, and also the rhythm of consuming them. With a single consumer reading the messages from a single Topic and partition, may not be the most scalable system. But if we can have multiple consumers reading from different partitions, then we can have parallel multiple consumers. Kafka also offers a way for the consumers to be rebalanced automatically. That is very useful functionality, where we have a Topic with different partitions that are being fed by multiple producers, but from the consumer standpoint, we can have a group that could be pointing just to the Topic name, and Kafka will assign which partitions the different consumer are going to read from. And even better, in case a consumer goes down, Kafka will automatically rebalance the partitions across the available consumers. Ain’t that cool? ![Source: https://www.confluent.io/blog/cooperative-rebalancing-in-kafka-streams-consumer-ksqldb/](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-4.png) That is one of the main differences between Kafka and other messaging systems. For example, the good old JMS does offer different options for consumers and producers to consume and generate -respectively- messages. It can be a queue or a topic. For a queue, just one single consumer can be subscribed to it. For a topic, many consumers can read from the same Topic but will read the same messages. > [!NOTE] > We are not saying nor comparing Kafka with JMS right here, but what we are doing is highlighting the functionalities that Kafka brings to the table. But you may be asking yourself: ain’t this a MuleSoft article? Where is MuleSoft within all this contextual Kafka explanation? Well, it happens that MuleSoft offers a connector that can produce and consume messages from Topics. This article is not to explain Kafka but to talk about the usage of MuleSoft Apache Connector ([https://docs.mulesoft.com/kafka-connector/4.6/](https://docs.mulesoft.com/kafka-connector/4.6/)) and how you can have different options to read information from Kafka Topics. Apache Kafka offers different SDKs for your application to reduce the complexity to produce/reading information into a Topic/Partition. For example Scala, JAVA, .NET, Go, Python, etc. The connector is implementing those very same SDKs, so you can do most of the things that you can achieve through the SDKs, with the MuleSoft connector. ## Scenarios of MuleSoft and Kafka getting together If you are looking to use MuleSoft to read from Kafka, my suggestions are to first understand Kafka's mechanics before trying to configure the connector. The connector is very straightforward to use; but if you do not have a good design and understanding of Kafka, and you just configure a MuleSoft flow to consume messages and for example, just point it to a specific partition, you have a scenario where thousands of messages need to be consumed and the result of that is that you are consuming message on a good rhythm but not in the one you expect, then it is because you may not have all the context of Kafka. Or imagine that you have a cluster of MuleSoft Runtimes running on-premise and you have the need to connect to Kafka and you did a similar configuration as explained in the previous paragraph. You deploy your MuleSoft application into the cluster, and you realize that the message is being duplicated (you have a two-node MuleSoft cluster). Why is that happening? Is that what I wanted? Why is MuleSoft duplicating messages? Then you start doubting that your MuleSoft application is not working as you were expecting. You even go further and raise a ticket with Support. But you know why? The problem is not MuleSoft, it is a lack of understanding of how a MuleSoft cluster works and also how Kafka works. This article is going to explain the mechanisms that you can use within a MuleSoft application to read messages from a Topic and partitions, and which are the expected behaviors and consequences. We will go through these scenarios: - Using a configuration where you can point to different topics and/or partitions. Here you explicitly configure the adapter to do so. - Using a configuration where you just point to the Topic name without specifying a partition number. - Using the configuration within a MuleSoft cluster (runtime on-premise, hybrid mode). In all those cases, MuleSoft is behaving like a consumer. And maybe it is obvious, but we need to understand that the connector is going to behave like a normal Kafka consumer. Is nothing extra, as we’ve mentioned, it is built on top of the official Kafka SDKs, it is nothing more than that. That is why we need to understand Kafka to configure it properly and understand how to read the messages. Let’s go scenario by scenario. ## Scenario [#1](https://www.prostdev.com/blog/hashtags/1) - Connect to a specific Topic and Partition In this scenario, we will connect Kafka directly to a partition number and Topic. Why would we do that? Your use case may be one where you need to have specific MuleSoft applications pointing to specific combinations of Topics and Partitions. Let’s say it in another way: deliberately you have a MuleSoft application reading from a specific partition, period. You understand that in this scenario you will have a single application and a single flow reading from the partition, you may not want to have more than one application and/or flow connected to the same partition unless you want to duplicate your messages and that is not a problem for you. A very simple MuleSoft flow will be something like this: ![Anypoint Studio flow: a Kafka Message listener connected to a Logger](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-5.png) In this case, the configuration from the Kafka connector perspective is as follows: ![Kafka Consumer config with Assignments set to Edit inline and TopicA partition 1 specified](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-6.png) In this scenario, our MuleSoft connector configuration in the Topics section needs to be Assignments (orange box), if you select that one the next section (red box) will let you decide which Topic and Partition you want to read from. You need to specify those two values, there is no way to continue with the rest of the configuration unless you input the Topic name and partition number. What happens if I add another flow or even more, another MuleSoft application with the same configuration? Like this: ![Two Kafka consumer flows with identical Message listeners reading the same topic partition](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-7.png) Here we are telling Kafka that we have two consumers wanting to read whatever messages are in the Partition 1 of Topic TopicA. The result: **both flows are going to get triggered and will process (duplicating) all messages**. As we’ve mentioned it can be another flow or another application, but that configuration will produce that effect in terms of duplicating the messages. If your use case is just to read the message once and process it once, then you need just one single application and/or flow consuming from Kafka. If your use case is to read the same message multiple times and process them distinctively, then this configuration is also going to work for you. Maybe you want to perform different transformations to the same message and process it differently, then you can have a separate flow doing that. Another possible question you may have is: what happens with this type of configuration but deploying the application on a MuleSoft cluster or in Cloudhub with multiple replicas? That question will be answered in the Scenario [#3](https://www.prostdev.com/blog/hashtags/3) section. ## Scenario [#2](https://www.prostdev.com/blog/hashtags/2) - Connect to a Topic but without specifying the Partition Contrary to the previous scenario, in this case, we want to point just to the Topic and let Kafka balance the consumers to read from the multiple partitions. This is what we explained at the beginning of the article. Let’s think that we have a Topic with 8 partitions, and hundreds of thousands of messages are getting produced and ready to be consumed by a group of consumers (MuleSoft applications). Those messages are distributed across all the 8 partitions. If we used the strategy of the previous scenario -all consumers pointing to the same partition- we will not read all the messages in the first place, and also the consumption rhythm to get all those hundreds of thousand messages with a single consumer will not be the best idea. Let’s also think that in the future those messages may increase; if we think in the Retail scenario that we’ve been describing in previous sections of this article, a new branch can be introduced to the system and this will increase the number of messages as well as partitions. That can cause us to increase the number of consumers; but this needs to be as smooth as possible and as simple as incrementing them, without the need to modify our code of change configurations. For example, in CloudHub, we can simply increase the number of replicas of our application and that will increase the number of consumers. Kafka will see that a new consumer is registered and will assign it a partition on the fly. To use this type of configuration, we need to use the following: ![Kafka Consumer config using Topic Subscription Patterns set to TopicA with no partition](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-8.png) The orange box has the Topic subscription configuration, in this case: **Topic Subscription Patterns**. And the red box will appear after selecting this option and will ask you for a Topic Name, in our case: TopicA. You do not need to specify the partition; in this case, Kafka will balance the partition across the different MuleSoft applications that are acting as consumers. In this case, if we have something like the following: ![Two Kafka consumer flows with topic-subscription Message listeners that each read different partitions](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-9.png) We have two different MuleSoft flows with the same Message Listener configuration (the one we described in previous paragraphs), in this case, both flows will consume different messages from different partitions. Or it can also be the case of having different multiple MuleSoft applications with the same Message Listener configuration, and every application will consume different messages. If one of our applications goes down for any reason, Kafka will rebalance the partitions through the rest of the consumers. Ain’t that cool? This scenario is very powerful, you can mix Kafka’s capabilities with MuleSoft’s. Kafka with the consumer balancing, and MuleSoft with its scalability. Now let’s move to the 3rd scenario where we will see how this configuration and the previous one (Scenario [#1](https://www.prostdev.com/blog/hashtags/1)) are affected within a MuleSoft cluster. ## Scenario [#3](https://www.prostdev.com/blog/hashtags/3) - MuleSoft acting as a cluster and reading from Kafka Both of the previous scenarios will respond to questions regarding your use cases. Is not that one scenario is better than the other. There will be some consequences if we have a use case that does not fit with the way you configure the connector, and we do not want that to happen. We want this type of article, to give you suggestions and ideas on how to determine the best way to use MuleSoft and avoid surprises. Let’s imagine we have a four-node MuleSoft cluster. In this case; hybrid mode. Within a MuleSoft cluster, certain dynamics happen behind the scenes for us, but that we need to understand to avoid misunderstandings. For example: - If you have a MuleSoft application that contains a scheduler, the scheduler will only run on the master node, to avoid that it will trigger in all the nodes. - If you have a MuleSoft application that is using the Object Store, this object store is replicated along with the cluster nodes. This is pretty good since all members have access to the same information - VM Queues. They are also distributed through the different nodes, and in case of a failure, if you have it configured with persistency, the information in the VM Queue will resist node failures and will continue working on the rest of the nodes - If you are subscribed to a JMS queue, for example, and you are consuming from Queue, you will need that only the master node can consume messages, and in case of a failure, the next eligible master node will continue consuming them But in our case with Kafka? As we’ve described in the previous two scenarios, sometimes we need to connect to a specific partition or sometimes we just want to connect to a Topic and have Kafka distribute the partitions for us. In the first scenario, again, if the main purpose is to point to a specific partition. If you deploy that on a cluster without checking the following box: ![Message listener Advanced settings with the Primary node only checkbox highlighted](../../assets/blog/3-scenarios-to-consume-an-apache-kafka-topic-with-mulesoft-10.png) The expected behavior is that every single node that is part of the cluster and that has our application deployed is going to consume the same message. If we don’t want duplicated messages, then check the Primary node only box, and that’s it. Consequences of this: only the master node will read from the partition, and the rhythm of consumption will be determined by the power of that node and application. Also, in case of a failure of the master node, and once a new master node is elected, it will be the one responsible for consuming the messages. Scenario [#2](https://www.prostdev.com/blog/hashtags/2) is a different story. Here we **do not need** to use this checkbox, since the main point of using this configuration model is to have multiple consumers reading messages. If you keep the Primary node only checkbox selected, then you will have just the master node reading messages. The consequences of this are: - You will not read messages for all partitions, since the worker node will be pointing to the partition that Kafka determined when it was subscribed. - You lost horsepower since you will have the rest of the nodes without reading messages. The conclusion: for use cases that are more aligned to Scenario [#2](https://www.prostdev.com/blog/hashtags/2), do not use the Primary node only option, keep it disabled ## Conclusions MuleSoft and Kafka working together is a very powerful solution. Implementing them together, as in any other technology, imply that you need to know how both work and which are the different alternatives to mix them. In this article, we just talked about consumption, which in our opinion is the scenario with the most alternatives. Before going directly to Anypoint Studio to create your applications, we suggest you first analyze and study the solution as a whole, to get the most out of MuleSoft and the applications/platforms that it will integrate. --- ## Amazon DynamoDB Connector Operations in Mule 4 (Part 1) Source: https://prostdev.com/post/amazon-dynamodb-connector-operations-in-mule-4-part-1 | Published: Mar 29, 2022 | Category: Tutorials This blog post provides quick examples of how to use the Amazon DynamoDB Connector in Mule 4 using Anypoint Studio. There is a lot of documentation available on this connector’s usage, but none could be found that specifically shows how to structure the JSON requests, which are required for all DynamoDB connector operations. This article explains all the connector operations along with their JSON requests, with simple use case scenarios. We have considered all available DynamoDB data types for this use case, so you will have a very good understanding and usage of different DynamoDB data types. ## Operations Below is the list of connector operations considered for the first part of this series. - Create Table - Describe Table - List Tables - Put Item - Batch Put Item - Single Table - Batch Put Item - Multiple Tables - Get Item - Batch Get Item - Single Table - Batch Get Item - Multiple Tables - Update Item - Query - Scan - Delete Item - Batch Delete Item - Single Table - Batch Delete Item - Multiple Tables ## Use Case To better understand the DynamoDB operations, we have considered a use case of the *continents* & their *countries*. All this data is segregated based on types of information & stored into two DynamoDB tables. Each *continent* consists of multiple *countries*, & each *country* will have its own specific information like unique id, available city name, specialties, available zip codes, year-wise populations, etc. All this information is stored in the table `CONTINENT-COUNTRY-INFO-TABLE`. Along with the above information, countries also have some classified information, like available agents in-country and intelligence agency information. All this classified information is stored in the table `COUNTRY-CLASIFIED-INFO-TABLE`. This use case has taken into consideration *(almost)* all available DynamoDB connector operations and their data types for clear understanding. **CONTINENT-COUNTRY-INFO-TABLE** | **Column Name** | **Data Type** | **Attributes** | |---|---|---| | continent_name | String | primary-partition-key | | country_unique_id | Number | primary-sort-key | | country_name | String | | | isActive | Boolean | | | country_specialities | StringSet | | | country_available_pincodes | NumberSet | | | country_yearwise_population | Map | | | country_city_name | List | | | country_geo_value | null | | **COUNTRY-CLASIFIED-INFO-TABLE** | **Column Name** | **Data Type** | **Attributes** | |---|---|---| | country_unique_id | Number | primary-partition-key | | country_agents | Number | | | country_intelligence_agency | String | | ## 1. Create Table ![Mule flow: Listener, Create table DynamoDB operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-1.png) Let's start with our first operation. This operation will create a new table with the provided configurations. > [!NOTE] > Execute this connector operation separately for both tables, with their own configurations. ![Create table config with continent_name HASH and country_unique_id RANGE keys and 100 read/write units.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-2.png) Below are the provided parameters: - **Table name:** The name for the table. - **AttributeDefinitions:** This value is an array of attributes that describe the key schema for the table and indexes. We have configured a composite primary key, which is a combination of primary key and sort key. - `continent_name`: Primary Partition Key - `country_unique_id`: Primary Sort key A composite primary key gives you additional flexibility when querying data. For example, if you provide only the value for `continent_name`, DynamoDB retrieves all of the `country_unique_id` associated with it. - **KeySchema**: This value specifies the attributes that make up the primary key for a table or an index. - **Read Capacity Units**: 100 – The maximum number of strongly consistent reads per second. - **Write Capacity Units**: 100 – The maximum number of writes consumed per second. **Response** Once you execute this connector, the DynamoDB tables will be created and you will receive a SUCCESS Response. ## 2. Describe Table ![Mule flow: Listener, Describe table DynamoDB operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-3.png) This operation is used to get information about the table, including the current status of the table, when it was created, the primary key schema, and any indexes on the table. ![Describe table config with Table name set to CONTINENT-COUNTRY-INFO-TABLE.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-4.png) The only parameter we configure for this operation is the *Table name*. **Response** Once executed, the connector will return a response in the below format. ```json { "globalSecondaryIndexes": null, "latestStreamLabel": null, "latestStreamArn": null, "keySchema": [ { "attributeName": "continent_name", "keyType": "HASH" }, { "attributeName": "country_unique_id", "keyType": "RANGE" } ], "provisionedThroughput": { "writeCapacityUnits": 100, "readCapacityUnits": 100, "numberOfDecreasesToday": 0, "lastIncreaseDateTime": null, "lastDecreaseDateTime": null }, "tableStatus": "ACTIVE", "itemCount": 0, "tableArn": "arn:aws:dynamodb:xx-xxxx-x:xxxxxxxxxxxxxxxxx:table/CONTINENT-COUNTRY-INFO-TABLE", "tableSizeBytes": 0, "tableName": "CONTINENT-COUNTRY-INFO-TABLE", "streamSpecification": null, "attributeDefinitions": [ { "attributeType": "STRING", "attributeName": "continent_name" }, { "attributeType": "NUMBER", "attributeName": "country_unique_id" } ], "localSecondaryIndexes": null, "creationDateTime": "2022-03-07T11:13:56.413" } ``` ## 3. List Tables ![Mule flow: Listener, List tables DynamoDB operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-5.png) This operation will return all the table names associated with your current AWS account and endpoint. ![List tables config with an Exclusive start table name and a Limit of 100.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-6.png) Below parameters are configured for this operation: - **Exclusive start table name**: The first table name that this operation will evaluate. If no table names are provided, it will list all tables. - **Limit**: A maximum number of table names to return. If this parameter is not specified, the limit is 100. **Response** Once executed, the connector will return a response in the below format. ```json { “tableNames” : [ “table-1”, “table-2”, “table-3” ], “lastEvaluatedTableName” : “table-4” } ``` ## 4. Put Item ![Mule flow: Listener, Transform Message for DynamoDB structure, Put item operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-7.png) Use this operation when the need is to create a new item or replace an existing item in the DB. If an item that has the same primary key as a new item already exists in a specified table, the new item replaces an existing item. ![Put item config with Table name CONTINENT-COUNTRY-INFO-TABLE and Item set to payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-8.png) **Request** Let's now create our first country “India” with other required information. ```dataweave %dw 2.0 output application/json --- { "continent_name": { "S": "ASIA" }, "country_unique_id": { "N": 1 }, "country_name": { "S": "India" }, "isActive": { "Bool": true }, "country_specialities": { "ss": ["India is democratic country", "India has 121 languages and 270 mother tongues", "Jana-gana-mana is a national song of india"] }, "country_available_pincodes": { "ns": [110001, 110002, 110003] }, "country_yearwise_population": { "M": { "2020": { "N": "1380004385" }, "2021": { "N": "1393409038" }, "2022": { "N": "1406631776" } } }, "country_city_name": { "L": [ { "S": "Mumbai" }, { "S": "Delhi" }, { "S": "Pune" }, { "S": "Kolkata" } ] } } ``` **Response** Once you execute the above flow, you will see 1 record is created in the table `CONTINENT-COUNTRY-INFO-TABLE`. ## 5. Batch Put Item - Single Table ![Mule flow: Listener, Transform Message, Batch put item operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-9.png) This operation inserts/puts multiple records or items in one table. **Request** Let's now create 2 country records “Singapore” and “Japan” in the same table `CONTINENT-COUNTRY-INFO-TABLE`. ```dataweave { "CONTINENT-COUNTRY-INFO-TABLE": [{ //Table1 "PutRequest": { // Item 1 "continent_name": { "S": "ASIA" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": 2 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_name": { "S": "Singapore" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "isActive": { "Bool": true } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_specialities": { "ss": [ "Singapore is Renowned for having some of the cleanest streets in the world", "Singapore has a Reputation as the Garden City", "Singapore has Good Street Food"] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_available_pincodes": { "ns": [1,2,3] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_yearwise_population": { "M": { "2020": { "N": 5850342 }, "2021": { "N": 5896686 }, "2022": { "N": 5943546 } }} as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_city_name": { "L": [{ "S": "Central Region" }, { "S": "East Region" }, { "S": "North Region" }, { "S": "North-East Region" }, { "S": "West Region" }] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class:"org.mule.extension.dynamodb.api.model.WriteRequest" }, { "PutRequest": { // Item 2 "continent_name": { "S": "ASIA" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": 3 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_name": { "S": "Japan" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "isActive": { "Bool": true } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_specialities": { "ss": [ "Mount Fuji is the highest volcano in Japan", "Tokyo Skytree is the Tallest tower not just in Japan, but in the entire world", "Japan has the Fastest bullet train", "Geisha are a big part of Japanese tradition and culture"] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_available_pincodes": { "ns": [1200000,1900100,1120000] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_yearwise_population": { "M": { "2020": { "N": 126476461 }, "2021": { "N": 126050804 }, "2022": { "N": 125584838 } } } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_city_name": { "L": [{ "S": "Tokyo" }, { "S": "Osaka" }, { "S": "Kawasaki" }, { "S": "Chiba" }] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class:"org.mule.extension.dynamodb.api.model.WriteRequest" }] } ``` **Response** Once you execute the above flow, you will see both these records are created in table `CONTINENT-COUNTRY-INFO-TABLE`. ## 6. Batch Put Item - Multiple Tables ![Mule flow: Listener, Transform Message, Batch put item - multiple table operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-10.png) This operation Inserts/Puts multiple records or items in one or more tables. ![Batch put item config with Request put items set to the Expression #[payload].](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-11.png) **Request** Let's now create one more country “Switzerland” in table `CONTINENT-COUNTRY-INFO-TABLE` and all 4 countries classified information (India, Singapore, Japan, Switzerland) in table `COUNTRY-CLASIFIED-INFO-TABLE`. ```dataweave { "CONTINENT-COUNTRY-INFO-TABLE": [{ //Table1 "PutRequest": { // Item 1 "continent_name": { "S": "Europe" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": 4 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_name": { "S": "Switzerland" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "isActive": { "Bool": true } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_specialities": { "ss": [ "Switzerland is famous for their watches", "Switzerland is a beautiful, tourist-attracting country", "Switzerland is famous for its mesmerizing alpine scenery, luxury branded watches, and deliciously milky chocolate"] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_available_pincodes": { "ns": [1715,1732,1673] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_yearwise_population": { "M": { "2020": { "N": 8654622 }, "2021": { "N": 8715494 }, "2022": { "N": 8773637 } }} as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_city_name": { "L": [{ "S": "Zurich" }, { "S": "Lucerne" }, { "S": "Basel" }, { "S": "Bern" }, { "S": "Geneva" }] } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class:"org.mule.extension.dynamodb.api.model.WriteRequest" }], "COUNTRY-CLASIFIED-INFO-TABLE": [{ "PutRequest": { "country_unique_id": { "N": 1 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_agents": { "N": 500000 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_intelligence_agency": { "S": "Research and Analysis Wing (RAW)" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class:"org.mule.extension.dynamodb.api.model.WriteRequest" },{ "PutRequest": { "country_unique_id": { "N": 2 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_agents": { "N": 120000 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_intelligence_agency": { "S": "The Security and Intelligence Division (SID)" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class:"org.mule.extension.dynamodb.api.model.WriteRequest" },{ "PutRequest": { "country_unique_id": { "N": 3 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_agents": { "N": 90000 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_intelligence_agency": { "S": "The Public Security Intelligence Agency" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class:"org.mule.extension.dynamodb.api.model.WriteRequest" }, { "PutRequest": { "country_unique_id": { "N": 4 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_agents": { "N": 50000 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_intelligence_agency": { "S": "The Swiss Intelligence Agency" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class:"org.mule.extension.dynamodb.api.model.WriteRequest" } ] } ``` **Response** Once you execute the above flow, you will see all these records are created in the respective table. ## 7. Get Item ![Get item flow with a Transform Message building the key with continent_name ASIA and country_unique_id 2.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-12.png) The GetItem operation returns a set of attributes for the item with the given primary key. If there is no matching item GetItem does not return any data. ![Get item config with Table name CONTINENT-COUNTRY-INFO-TABLE, Key set to payload, and Consistent read False.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-13.png) **Request** Let's now fetch the record from table `CONTINENT-COUNTRY-INFO-TABLE`, having continent = ‘ASIA’ & country unique id = ‘2’. ```dataweave %dw 2.0 output application/json --- { "continent_name" : {"S" : "ASIA"}, "country_unique_id" : {"N" : 2} } ``` **Response** Once you execute the above flow, you will see the respective records in response. ## 8. Batch Get Item - Single Table ![Mule flow: Listener, Transform Message, Batch get item - single table operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-14.png) The Batch Get Item operation returns the attributes of one or more items from one table. You identify the requested item by primary key and sort key combination. ![Batch get item config with Request items set to the Expression #[payload] and Return consumed capacity TOTAL.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-15.png) **Request** Let’s now fetch two records from table `CONTINENT-COUNTRY-INFO-TABLE` , having continent = ‘ASIA’ & ‘Europe’ & country unique id = ‘1’ & ‘4’. ```dataweave { "CONTINENT-COUNTRY-INFO-TABLE" : { "Keys": [ { "continent_name" : {"S":"ASIA"} as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"}, "country_unique_id":{"N": 1} as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"} }, { "continent_name" : {"S":"Europe"} as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"}, "country_unique_id":{"N": 4} as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"} } ] } as Object {class:"org.mule.extension.dynamodb.api.model.KeysAndAttributes"} } ``` **Response** Once you execute the above flow, you will see both these respective records in response. ## 9. Batch Get Item - Multiple Tables ![Mule flow: Listener, Transform Message, Batch get item - multiple table operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-16.png) The Batch Get Item operation returns the attributes of one or more items from one or more tables. You identify the requested item by primary key. ![Batch get item config for multiple tables with Request items #[payload] and Return consumed capacity TOTAL.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-17.png) **Request** Let's now fetch two records each from table `CONTINENT-COUNTRY-INFO-TABLE` & `COUNTRY-CLASIFIED-INFO-TABLE`. ```dataweave { "CONTINENT-COUNTRY-INFO-TABLE": { "Keys": [{ "continent_name": { "S": "ASIA" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": 1 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } }, { "continent_name": { "S": "ASIA" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": 2 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } }] } as Object { class:"org.mule.extension.dynamodb.api.model.KeysAndAttributes" }, "COUNTRY-CLASIFIED-INFO-TABLE": { "Keys": [{ "country_unique_id": { "N": 1 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } }, { "country_unique_id": { "N": 2 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } }] } as Object { class:"org.mule.extension.dynamodb.api.model.KeysAndAttributes" } } ``` **Response** Once you execute the above flow, you will see both these respective records from each table, in the response. ## 10. Update Item ![Update item flow with a Transform Message building the key for continent_name ASIA and country_unique_id 1.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-18.png) This operation edits an existing item's attributes or adds a new item to the table if it does not already exist. ![Update item config with a SET update expression and inline attribute values for isActive and countryName.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-19.png) **Request** Let's now update the record with country name "India" with: - `isActive = 'false'` - `country_name = 'India/Bharat'` ```dataweave %dw 2.0 output application/json --- { "continent_name": {"S": "ASIA"} as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": {"N": 1} as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } ``` **Response** Once the above request is served, both these columns (`isActive` & `country_name`) will be updated respectively with the new values. ## 11. Query ![Query flow with a Transform Message building criteria continent_name ASIA and isActive true.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-20.png) The Query operation finds items based on primary key values. You can query any table that has or secondary index that has a composite primary key (a partition key and a sort key). ![Query config with the key condition expression continent_name = :continent_name on the partition key.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-21.png) ![Query parameters mapping #isActive and the :continent_name and :isActive attribute values from the payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-22.png) ![Query config with filter expression isActive = :isActive and a Limit of 10.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-23.png) **Request** Now, Let’s query the table `CONTINENT-COUNTRY-INFO-TABLE` and fetch all the records that satisfy the following criteria: - `continent_name = 'ASIA'` - `isActive = 'true'` ```dataweave %dw 2.0 output application/json --- { "continent_name" : {"S" : "ASIA"}, "isActive" : {"Bool" : true} } ``` **Response** Once you execute the above flow, you will find 3 records in the response. ## 12. Scan ![Scan flow with a Transform Message building criteria continent_name Europe and isActive true.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-24.png) The scan operation returns one or more items or item attributes By accessing every item in a table or secondary index. To have DynamoDB return fewer items, you can provide a FilterExpression operation. ![Scan config with Segment 0, Total segments 1, and continent_name and isActive attribute values.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-25.png) ![Scan config with two filter expressions on continent_name and isActive and a Limit of 100.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-26.png) **Request** Let's scan the table `CONTINENT-COUNTRY-INFO-TABLE` and fetch all the records that satisfy the following criteria: - `continent_name = 'Europe'` - `isActive = 'true'` ```dataweave %dw 2.0 output application/json --- { "continent_name" : {"S" : "Europe"}, "isActive" : {"Bool" : true} } ``` **Response** Once you execute the above flow, you will find only 1 record in response. ## 13. Delete Item ![Mule flow: Listener, Transform Message single delete request, Delete item operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-27.png) This operation deletes a single item in a table. Items are identified using primary key and sort key combination. ![Delete item flow with a Transform Message building the key continent_name ASIA and country_unique_id 3.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-28.png) **Request** Let's now delete the record with country = ‘Japan’ from table `CONTINENT-COUNTRY-INFO-TABLE` ```dataweave %dw 2.0 output application/json --- { "continent_name": { "S": "ASIA" } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": 3 } as Object { class:"org.mule.extension.dynamodb.api.model.AttributeValue" } } ``` **Response** Once executed, the record will be deleted from the table. ## 14. Batch Delete Item - Single Table ![Mule flow: Listener, Transform Message, Batch delete item - single table operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-29.png) This operation deletes multiple items in one table. ![Batch delete item config with Request delete items set to the payload expression.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-30.png) **Request** We’ll now delete 2 records in a single request. Let’s delete both countries “Singapore” & “Switzerland” from the table `CONTINENT-COUNTRY-INFO-TABLE`. ```dataweave { "CONTINENT-COUNTRY-INFO-TABLE": [{ "DeleteRequest": { "continent_name": { "S": "ASIA" } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": "2" } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class : "org.mule.extension.dynamodb.api.model.WriteRequest" }, { "DeleteRequest": { "continent_name": { "S": "Europe" } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": "4" } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class : "org.mule.extension.dynamodb.api.model.WriteRequest" }] } ``` **Response** Once executed, both these records will have been deleted from the table. ## 15. Batch Delete Item - Multiple Tables ![Mule flow: Listener, Transform Message, Batch delete item - multiple table operation, then Set Payload.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-31.png) This operation deletes multiple items from multiple tables. ![Batch delete item config for multiple tables with Request delete items set to the payload expression.](../../assets/blog/amazon-dynamodb-connector-operations-in-mule-4-part-1-32.png) **Request** We’ll delete country ‘India’ from table `CONTINENT-COUNTRY-INFO-TABLE` and its classified information from the table `COUNTRY-CLASIFIED-INFO-TABLE`. ```dataweave { "CONTINENT-COUNTRY-INFO-TABLE": [{ "DeleteRequest": { "continent_name": { "S": "ASIA" } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" }, "country_unique_id": { "N": 1 } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class : "org.mule.extension.dynamodb.api.model.WriteRequest" } ], "COUNTRY-CLASIFIED-INFO-TABLE": [{ "DeleteRequest": { "country_unique_id": { "N": 1 } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class : "org.mule.extension.dynamodb.api.model.WriteRequest" }, { "DeleteRequest": { "country_unique_id": { "N": 3 } as Object { class: "org.mule.extension.dynamodb.api.model.AttributeValue" } } } as Object { class : "org.mule.extension.dynamodb.api.model.WriteRequest" }] } ``` **Response** Once executed, all 3 records will have been deleted from both tables. ## Conclusion We have seen the working examples of 15 operations of DynamoDB connectors. Stay tuned for the next part. Thank you and happy learning!! --- ## Guidelines to submit your content in ProstDev Source: https://prostdev.com/post/guidelines-to-submit-your-content-in-prostdev | Published: Mar 22, 2022 | Category: News Thank you for your interest in creating a post with us! We've been getting more articles lately but our bandwidth to review them is a bit decreased at the moment. Because of this, the time we spend reviewing the posts will need to be reduced. We want to keep serving our readers with great quality content, so our quality standards will have to be enforced by you **before** sending your application. Once you've followed the guidelines, we can start the review process. Your application might be **delayed** or **rejected** if your contribution does not comply with the standards. - Your article needs to be created in Google Docs before submitting. - Please make sure your document has the appropriate editing access for us to make editions to your file. - The title of your article must be 100 characters or less. - The contents of your article must be between 500 and 1,500 words. Note that specific content within the article (like tables or code blocks) might make the article longer than it is. If this is the case, please let us know when you submit your application or create gist files for the code (next point). - Any code blocks with more than 5 lines of code must be replaced with [public gist files](https://docs.github.com/en/get-started/writing-on-github/editing-and-sharing-content-with-gists/creating-gists). Please read this [Article Formatting section](https://www.prostdev.com/post/prostdev-s-new-blog-post-template#viewer-eg9k8) for more information. - If you copy/paste quotes, pictures, or code from other sites, you need to provide the link where you got it from. We don't accept any form of plagiarism. - We will reject content that has not been proofread before submission. To proofread your content, you can use services or systems that check your grammar, like [Grammarly](https://grammarly.com/) or [Hemingway](https://hemingwayapp.com/); or ask a friend to review your content first. - Once you've taken care of all these, you can submit your content [here](https://github.com/ProstDev/blog-posts/issues/new/choose). Please note that you will need a GitHub Account. Once you submit your content, we might take between 1-2 weeks to get back to you, depending on our current content queue. Any missed guideline/standard from the previous list will result in rejection or further delays in your review process. If your content is reviewed on time and follows the appropriate standards, we can publish your content as soon as 2 weeks. If further editions are required from your side, the publishing date will be extended. Any round of editions might add an additional week to the publishing date depending on our queue. Please be patient. Thank you! --- ## Anypoint Platform Single Sign-on (SSO) SAML Configuration with Oracle IDCS - PART 2 Source: https://prostdev.com/post/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2 | Published: Mar 15, 2022 | Category: Tutorials Our[last article](https://www.prostdev.com/post/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1) was about how to integrate MuleSoft Anypoint Platform with Oracle Identity Cloud Services. With this, you can use Oracle IDCS as your Identity Provider. This is very useful for Single Sign-On purposes, and if you already have Oracle Cloud Infrastructure and you are using MuleSoft to integrate it with Oracle SaaS apps, this alternative for SSO should be a good fit for you. In our last article, we mentioned that we would have a second part where we explain how to map attributes, such as - Email - First name and Last name - Telephone Number - Groups - Etc Your users are part of Oracle Identity Cloud Services. Some of those users need to enter to MuleSoft Anypoint Platform to perform different activities, like - Design APIs - Browse Exchange - Deploy Applications - Manage your organization But you don’t want to assign those roles directly on MuleSoft, you want to have that information coming from your Identity Provider; at the end that is the place where you create your users and assign groups. So, it is natural to think that the user is already assigned to groups that can be mapped to your MuleSoft roles. Once the users log in to MuleSoft, through the Identity Provider and it Authenticates/Authorizes them, and finally are able to get into MuleSoft, the expectation is that the users are already assigned to their roles and start working with their duties. Well, that is something that you can configure between MuleSoft and Oracle IDCS. Let’s get back to the IDCS console and create a group: ![Oracle IDCS Groups list with the Add button highlighted to create a new group](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-1.png) We just need to click on the Add button and create a group: ![Add Group step 1 naming the group CloudhubAdminSbx with a description](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-2.png) We are naming the user: CloudhubAdminSbx, and giving a brief description. Then click on Next and let’s assign a user: ![Add Group step 2 assigning the user Rolando Carrasco to the new group](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-3.png) In this case, it is me: [rcarrascogb2@me.com](mailto:rcarrascogb2@me.com). Then just click on Finish. What we have done is to create a new group (take a note of the name because we will use it later to map it inside MuleSoft Anypoint Platform), and we are assigning a user to it. Now let’s get back to our Application inside IDCS (the one we configured in our previous article), to map attributes and groups. Let’s do it. ![IDCS Mule application SSO Configuration page with Attribute Configuration highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-4.png) In the Attribute configuration, click on it and you will see something like this: ![IDCS Attribute Configuration table mapping email, firstname, lastname, and Groups](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-5.png) Your table may be empty and you will need to add the needed attributes, in my case: - email - firstname - lastname - Groups Those are the attributes that will be sent from IDCS to MuleSoft inside the SAML assertion and MuleSoft will map into its attributes. Take a look at the names, and get back to MuleSoft Anypoint Platform console to the Access Management menu: ![Anypoint Access Management Identity Providers list showing Anypoint and Oracle IDCS](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-6.png) We created this configuration in our previous [article](https://www.prostdev.com/post/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1), click on it. Go almost at the bottom of that page: ![Anypoint IdP advanced settings with first name, last name, email, and group attribute fields](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-7.png) Those names are mapped into MuleSoft and they need to match. In IDCS those fields are going to be filled with the user information. If we zoom in to the configuration at IDCS you will see this: ![IDCS attribute table with Name, Format, Type, and Value columns color-boxed for emphasis](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-8.png) In orange are the attributes that we want to return, in purple are the format in this case basic. In blue is the type of attribute and finally the value that IDCS will populate on them in red. And for the groups in specific we’ve configured it like this: ![Groups attribute set to Group Membership with the All Groups option selected](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-9.png) We are returning all the groups that the user is a member of. We have other options, like: ![Group filter dropdown open showing Equals, Starts with, and All Groups options](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-10.png) That means that our users can be assigned into multiple IDCS groups, but we can filter them to return just the ones the application needs. For this article, and to be practical, we are returning all of them. Everything is in place, our user ([rcarrascogb2@me.com](mailto:rcarrascogb2@me.com)) can try to log in to MuleSoft, and not only that will make it through, but at the same type, depending on the list of groups where the user belongs, it will map it into MuleSoft roles. But the question is: how does MuleSoft know that those group names (which by the way are arbitrary) are getting mapped with their roles? Just go to MuleSoft Anypoint Platform, and go to the Roles section: ![Anypoint Access Management Roles list including Cloudhub Admin (Sandbox)](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-11.png) Click on the group Cloud Admin (Sandbox) and you will see this: ![Cloudhub Admin (Sandbox) role page with the Set external group mapping link highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-12.png) Click on “Set external group mapping”, and you will see this: ![External IdP Groups dialog mapping group CloudhubAdminSbx to the Oracle IDCS provider](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-13.png) Type the group name that we’ve created at IDCS, choose the Identity Provider, and finally click on the add button. ![External IdP Groups dialog after the CloudhubAdminSbx Oracle IDCS mapping is added](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-14.png) Click on Save. Before testing it, let’s get back to MuleSoft Anypoint Platform, and review the user information: ![User Roles tab listing only the Cloudhub Admin (Design) role before login](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-15.png) The user has only that role assigned. But now, after his next login, it will have an additional role ([Cloudhub Admin (Sandbox)](https://anypoint.mulesoft.com/accounts/#/cs/core/roles/edit/66151060-bb95-499d-868f-bd4bebbb7c7d/permissions)). Also, as we saw in our previous article, the user did not have the attributes being mapped: ![Users list showing the Oracle IDCS user as "unknown unknown" with only an email mapped](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-16.png) No name (firstname and last name), just the email was mapped. Now the new configuration will populate that information as well as the group. Let’s see this action, let’s try to log in: ![Anypoint Platform sign-in page with a "Continue with Oracle IDCS" button](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-17.png) Click on Continue with Oracle IDCS: ![Oracle Cloud login page (in Spanish) with the user's email and password filled in](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-18.png) After we click on login, let’s capture the SAML response from IDCS: ![Captured SAML response XML with the firstname, lastname, email, and groups attributes](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-19.png) In pink, you can see that the firstname attribute was returned by Oracle IDCS. With orange (and highlighted in red), you can see the list of groups, including the one we created previously. And in blue we also see the lastname and email attributes being returned. Now, if we go to the Roles section of the user rcarrascogb2@me.com, we will see that the role has been assigned: ![User Roles tab now listing both Cloudhub Admin Sandbox and Design roles after login](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-2-20.png) With this, you can provision users into your Identity Provider, assign groups at that level and then map them into MuleSoft. I hope this is useful for you. --- ## Basic Google Big Query operations with a Salesforce sync demo in Mule 4 Source: https://prostdev.com/post/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4 | Published: Mar 1, 2022 | Category: Tutorials *GitHub repository with the Mule project can be found at the end of the post.* If we think about data storage the first thing that comes to our mind is a regular database, this can be any of the most popular ones like Mysql, SQL Server, Postgres, Vertica, etc. But I noticed not too many have interacted with one of the services Google provides with the same purpose **Google BigQuery.** And maybe it is because of the [pricing](https://cloud.google.com/bigquery/pricing?utm_source=google&utm_medium=cpc&utm_campaign=na-US-all-en-dr-bkws-all-all-trial-e-dr-1011347&utm_content=text-ad-none-any-DEV_c-CRE_573148306951-ADGP_Desk%20%7C%20BKWS%20-%20EXA%20%7C%20Txt%20~%20Data%20Analytics%20~%20BigQuery_Pricing%20Google%20Google-KWID_43700068582852990-kwd-166600832170&utm_term=KW_google%20bigquery%20pricing-ST_google%20bigquery%20pricing&gclsrc=aw.ds&gclid=Cj0KCQiAjJOQBhCkARIsAEKMtO3fCohKU2ihxQrQ21u9XixqWhXy9w7QNu8MqGaVp58zn59WTN0ekA4aAtAeEALw_wcB), but in the end, many companies are moving to cloud services and this service seems to be a great fit for them. In this post, I would like to demonstrate in a few steps how we can make a sync job that allows us to describe a Salesforce instance and use a few objects to create a full schema of those objects (tables) into a Google Big Query Dataset. Then with the schema created we should be able to push some data into BigQuery from Salesforce and see it in our Google Cloud Console project. ## Prerequisites To connect to Salesforce and Google BigQuery, there are a few prerequisites we need. **Salesforce:** - If you don’t have a salesforce instance, you can create a developer one [here](https://developer.salesforce.com/signup). - From the Salesforce side, you will need a **username, password,** and **security token** (you can follow [this process](https://help.salesforce.com/s/articleView?id=sf.user_security_token.htm&type=5) to get it). - A developer instance contains a few records, but if you need to have some more data, this will help the process to sync that information over. **GCP (Google Cloud Platform)** - You can sign up [here](https://console.cloud.google.com/freetrial/signup/tos?_ga=2.68094097.1640278748.1644510554-1516430238.1644510554&_gac=1.218077796.1644510554.Cj0KCQiAjJOQBhCkARIsAEKMtO0NyXkbcz86jMGZOta5V7HYUNkiDHCDR_6OSc4ioZFtAHlp0tw8_JUaAnI7EALw_wcB) for free. Google gives you $300 for 90 days to test the product (similar to Azure). If you already have a Google account, you can use it for this. ## Creating a new project in GCP and setting up our service account key Once you sign up for your account on GCP, you should be able to click on the New Project option and write a project name, in this example I chose **mulesoft.** ![GCP "Select a project" dialog with an arrow pointing to the New Project button](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-1.png) ![GCP New Project form with a project name field and Create button](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-2.png) Once a project is created we should be able to go to the menu on the left and we should be able to select IAM & Admin > Service Accounts option. ![GCP left menu with IAM & Admin expanded and Service Accounts highlighted](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-3.png) Now, we should be able to create our service account. “A **service account** is a special type of Google account intended to represent a non-human user that needs to authenticate and be authorized to access data in Google APIs. Typically, service accounts are used in scenarios such as Running workloads on virtual machines.” At the top of the page, you should be able to see the option to create it. Then, you just need to specify a Name and click on create and continue. ![Create service account form with a name field set to "sample"](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-4.png) The next step is to set the permissions, so for this, we need to select from the roles combo BigQuery Admin. ![Role selector dropdown with BigQuery Admin chosen for the service account](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-5.png) Once created, we should be able to select from the three-dot menu on the right the option **Manage Keys.** ![Service account three-dot Actions menu with Manage keys highlighted](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-6.png) Then we can create a new Key, in this case, one as JSON should be enough. The key will get downloaded automatically to your computer (**please keep this JSON key somewhere you can use it later**). ![Add Key dropdown showing Create new key and Upload existing key options](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-7.png) ### Dataset in Big Query Datasets are top-level containers that are used to organize and control access to your tables and views. A table or view must belong to a dataset, so you need to create at least one dataset before loading data into BigQuery. From the left menu, we can search for BigQuery and click on it. ![GCP navigation menu with BigQuery pinned under the Pinned section](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-8.png) That will take us to the BigQuery console. Now we can click on the three-dots menu and select the **Create dataset** option. ![BigQuery Explorer with the project's three-dot menu showing Create dataset](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-9.png) Now we just need to set the name as salesforce and click on “Create Dataset.” ![BigQuery Create dataset panel with Dataset ID set to "salesforce"](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-10.png) ## Setting up our Mule application Since this is a sync job, we don’t need any API specification but totally can fit some scenarios where we have another application that needs to consume specific endpoints/operations. Let’s then open our Anypoint Studio app (In my case I’m using a mac) and let’s use the default template. For this we are going to create six flows: - **Sync**. This flow just triggers the process. - **DescribeInstance.** This flow will be in charge of calling the describe operation using the Salesforce connector and provide all objects information from the Salesforce instance, also will have a loop that will allow us to process the job for the objects we are going to use. - **DescribeIndividualSalesforceObject.** Allows to describe a specific Salesforce object, this will basically will capture the fields and field types (STRING, EMAIL, ID, REFERENCE, etc.) and will be in charge to create a payload that BigQuery will recognize in order to get created in GBQ - **BigQueryCreateTable.** This flow only will be in charge of creating the table in BigQuery based on the Salesforce object name and the fields. - **QuerySalesforceObject.** This flow dynamically will query the Salesforce object and pull the data. (*For this we are limiting the output to 100 records but on a bigger scale it should be done on a batch process of course.*) - **InsertDataIntoBigQuery**. This flow will push the data over into BigQuery only. > [!NOTE] > I will go over in detail each of the flows in the next sections. Now let’s grab our JSON key generated by google and copy the file under the `src/main/resources` folder. The key will let us authenticate against our project and execute the operations. ![Studio package tree showing the JSON key under src/main/resources](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-11.png) ### Import the Google Big Query connector From Exchange, we can search “BigQuery” and we should be able to see the connector listed. ![Exchange listing for the Google BigQuery Connector for Mule 4](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-12.png) Then, we can just use the “Add to project” option and we should be able to see the operations in the Mule Palette. ![Mule Palette listing the Google BigQuery connector operations](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-13.png) ### Sync Flow As I mentioned, this is only in charge of triggering the whole application, so we only need one scheduler component and a flow reference to the DescribeInstance flow. ![Sync flow with a Scheduler connected to a DescribeInstance flow reference](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-14.png) ### DescribeInstance This flow will describe the whole Salesforce instance using the [Describe Global](https://developer.salesforce.com/docs/atlas.en-us.208.0.api_rest.meta/api_rest/resources_describeGlobal.htm) operation. The next step on this is to use a DataWeave transform to get only the Objects we are interested in. So in this case, I’m only pulling three: Accounts, Contacts, and a custom object called `Project__c`. I left in the transformation a few more attributes to only pull the objects that we can query. ```dataweave payload.sobjects filter ( $.queryable == true and $.replicateable == true and $.retrieveable == true and $.searchable == true and $.triggerable == true and ($.name == "Account" or $.name == "Contact" or $.name =="Project__c")) map $ ``` ![DescribeInstance flow: Describe global, a filter transform, then four flow references in a loop](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-15.png) Finally, you need to loop over these three objects and there’s a flow reference for this sample that will call the other flows to be able to continue the process. ### DescribeIndividualSalesforceObject This flow takes the name of the Salesforce Object and will allow it to describe it. The connector only asks for the object name, so we have a pretty interesting DW Transformation. ```dataweave %dw 2.0 input payload application/java output application/java fun validateField(field) = if ( (field == "REFERENCE") or (field == "ID") or (field == "PICKLIST") or (field == "TEXTAREA") or (field == "ADDRESS")or (field == "EMAIL")or (field == "PHONE") or (field == "URL")) "STRING" else if ( (field == "DOUBLE") or (field == "CURRENCY") ) "FLOAT" else if ((field == "INT")) "INTEGER" else field --- (payload.fields filter ($."type" != "LOCATION") map { fieldName : $.name, fieldType : validateField($."type") }) ``` Salesforce data types are not 100% the same as BigQuery, so we need to make a little trick to be able to create the schema in BigQuery seamlessly as Salesforce. In this case, I’ve created a small function to convert some fields (like ID, REFERENCE, TEXTAREA, PHONE, ADDRESS, PICKLIST, EMAIL) to be STRING. In this case, the reference or values are not really anything else than a text. For DOUBLE and CURRENCY, I’m using the value FLOAT. Finally, for INT fields are changed to be INTEGER. Because **Location** fields are a bit tricky and we are not able to make much with the API on them, I’m removing all location fields. The output of this is the actual schema we will use to create the table in Google BigQuery. ![DescribeIndividualSalesforceObject flow: Describe SObject then two Transform Message steps](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-16.png) ### BigQueryCreateTable This flow allows us to create the table in BigQuery; we only need to specify Table, Dataset, and Table Fields. ![Create Table operation config with Table from a variable and Dataset set to salesforce](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-17.png) ![BigQueryCreateTable flow with a single Create Table with Fields operation](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-18.png) ### QuerySalesforceObject This flow queries the Object in Salesforce and then maps the data dynamically to prepare the payload for BigQuery. ![Salesforce Query config with a dynamic SELECT and field/table parameters](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-19.png) The query comes from a variable “salesforceFields” the same field we collected when we described the Object using this script: ```dataweave (payload.fields filter ($."type" != "LOCATION") map { fieldName : $.name }).fieldName joinBy "," ``` And finally, I’m limiting the result to only 100 records. The next step is to map the Salesforce result data and map it dynamically using this script: ```dataweave %dw 2.0 import try, fail from dw::Runtime output application/java fun isDate(value: Any): Boolean = try(() -> value as Date).success fun getDate(value: Any): Date | Null | Any = ( if ( isDate(value) ) value as Date as String else value ) --- (payload map (item,index) ->{ (item mapObject ((value, key, index) -> { (key):(getDate(value)) } )) }) ``` Thanks so much to Alex Martinez for the insights on the utilities for DW 2.0! ([https://github.com/alexandramartinez/DataWeave-scripts/blob/main/utilities/utilities.dwl](https://github.com/alexandramartinez/DataWeave-scripts/blob/main/utilities/utilities.dwl)) This last script maps the records and uses the key as field and the value, but the value needs to be replaced as Date in this case for the Strings that are date or date-time. So I consider this the best script in this app. ![QuerySalesforceObject flow: Query, a Dynamic Field Mapping transform, then Logger](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-20.png) ### InsertDataIntoBigQuery This flow just inserts the data we prepared, so basically, we only need to specify table id, dataset id, and the Row Data. ![InsertDataIntoBigQuery flow with an Insert All operation followed by a Logger](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-21.png) ![Insert All operation config with Table id from a variable and Rows data set to payload](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-22.png) ## Setting up our Mule application Now we should be able to run our application and see the new tables and the data over Google BigQuery. On GCP, I can see the tables I selected being created: ![BigQuery Explorer showing the salesforce dataset with Account, Contact and Project__c tables](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-23.png) And if we open any of them we should look into the schema to verify all fields are there. ![BigQuery Contact table schema listing field names, types and nullable modes](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-24.png) Finally, we should be able to query the table in the console or click on the Preview option to check the data is there. ![BigQuery query results showing synced Account rows from Salesforce](../../assets/blog/basic-google-big-query-operations-with-a-salesforce-sync-demo-in-mule-4-25.png) I think this is kind of a common request we get on the integration space and many tweaks can be implemented if we are thinking of big migrations or setting some jobs that eventually will require tables to be created automatically from Salesforce to GCP. ## GitHub repository If you’d like to try it, I created this [GitHub](https://github.com/emoran/sfdc-to-bigquery) repository. I hope this was useful and I’m open to hearing any enhancement/scenario. --- ## Oracle database Stored Procedure does not time out as expected in MuleSoft Source: https://prostdev.com/post/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft | Published: Feb 15, 2022 | Category: Guides In a recent MuleSoft project, I noticed a problem when trying to run an Oracle stored procedure that took a fair amount of time to complete. ## Context Even though the query timeout parameter was set in my code, I noticed that the execution of the MuleSoft flow kept waiting for the stored procedure to finish its execution and then continued the execution of subsequent processors in the flow. Here's an example flow I built to exemplify this behavior. I have an **HTTP listener** that accepts HTTP requests in the **/stored-proc** resource. The next processor is the **call to the stored procedure** in the Oracle database. This stored procedure does nothing but wait a random amount of time (from 5 to 10 seconds) and then write a message to the log. This stored procedure was intentionally built that way to simulate long-running execution. Finally, the flow **sets a success message** to the payload, and it is returned in the **HTTP response**. ![Mule flow: HTTP Listener, stored procedure call WAIT_FOR_A_WHILE, then Set Payload success](../../assets/blog/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft-1.png) I am using a Docker container for the Oracle database. It is always much simpler to set up your environment using Docker containers to avoid installing all the software in a local environment. If you are interested in using this container, you can get it from DockerHub: [https://hub.docker.com/_/oracle-database-enterprise-edition](https://hub.docker.com/_/oracle-database-enterprise-edition) Or execute the following Docker command: ```bash docker pull store/oracle/database-enterprise:12.2.0.1 ``` ## Oracle Stored Procedure Below is the code I used to create the stored procedure in the database. As I mentioned earlier, the stored procedure only waits a few seconds before writing a message to the database log. ```sql CREATE OR REPLACE PROCEDURE ANONYMOUS.WAIT_FOR_A_WHILE IS BEGIN DBMS_LOCK.Sleep(DBMS_RANDOM.VALUE(5,10)); SYS.DBMS_OUTPUT.PUT_LINE(to_char(sysdate, 'hh24:mi:ss') || ': finished.'); END WAIT_FOR_A_WHILE; ``` ## MuleSoft flow configuration The processors that are part of the flow are configured as follows: 1. The HTTP listener listens on port 18081 and on the /stored-proc resource. ![HTTP Listener config with the path set to /stored-proc highlighted](../../assets/blog/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft-2.png) 2. The database connector executes the stored procedure and has a configured timeout of 2 seconds. My expectation is that by taking more than 2 seconds, the connector returns an error due to timeout, and the flow execution is interrupted. ![Database connector calling WAIT_FOR_A_WHILE with a 2-second query timeout configured](../../assets/blog/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft-3.png) 3. The set payload only places a message after successfully executing the stored procedure. ![Set Payload returning a JSON "Stored proc executed." success message](../../assets/blog/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft-4.png) ## Testing the MuleSoft flow However, the flow behavior is not as expected. What happens is that the database connector waits for the execution of the stored procedure to finish (regardless of how long the execution takes) and then returns the response message established in the set payload. This indicates that the timeout configuration is not working properly. ![Postman GET returning 200 OK after 7.29 s, ignoring the 2-second timeout](../../assets/blog/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft-5.png) The endpoint response takes more than 5 seconds. This is understandable (but not expected) since the stored procedure is waiting between 5 and 10 seconds before returning a response. ## Fixing the issue The solution to the problem is somewhat simple. Doing some research in the MuleSoft documentation, we can find that the origin of the problem is not our code but the Oracle driver: [https://help.mulesoft.com/s/article/Database-connector-Query-timeout-not-working-as-expected-on-oracle-database](https://help.mulesoft.com/s/article/Database-connector-Query-timeout-not-working-as-expected-on-oracle-database) The documentation indicates that if we put the parameter **-Doracle.net.disableOob=true** at runtime startup, this problem will be solved. To do this in a local environment, it is only necessary to go to the menu: Run > Run Configurations > Arguments and add the parameter in the startup configuration in Anypoint Studio and run the application again. ![Run Configurations Arguments tab adding -Doracle.net.disableOob=true to the VM arguments](../../assets/blog/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft-6.png) In the following executions, we will always get an error indicating that the user (in this case the MuleSoft application) canceled the operation with the database. This is because the stored procedure is taking between 5 and 10 seconds to respond and the database connector is waiting a maximum of 2 seconds for the operation to complete. ![Postman now returning 500 Server Error with ORA-01013 user requested cancel of current operation](../../assets/blog/oracle-database-stored-procedure-does-not-time-out-as-expected-in-mulesoft-7.png) This means that the timeout parameter is now working properly. ## Conclusion As a conclusion, we can say that if we have the following mix: **MuleSoft + Oracle Database long-running executions (queries or stored procedure calls) + Timeout configuration** It is necessary to configure the Oracle driver parameter at Runtime startup to prevent the database connector from waiting for the execution to finish. Instead, the timeout configuration would be honored. **Additional notes**: You can notice that in the second POSTMAN call we have a response time of 6.59 seconds. That is also a problem on the Oracle JDBC driver side of things. The timeout configuration is honored, but the driver takes a long time to get the control back to the MuleSoft app. This was isolated and tested with a simple Java class and it has the same behavior. So, if you want to solve that problem, you’ll need to raise a ticket with the Oracle support team to solve the whole issue. --- ## Anypoint Platform Single Sign-on (SSO) SAML Configuration with Oracle IDCS – PART 1 Source: https://prostdev.com/post/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1 | Published: Feb 1, 2022 | Category: Tutorials It is very common to have the need to login into the Anypoint Platform console ([https://anypoint.mulesoft.com](https://anypoint.mulesoft.com/)) using what the customer already has in terms of an Identity Provider and an Identity Repository (LDAP). This could be for different reasons, such as: - Promote the current Single Sign-on (SSO) functionalities, provided by the customer’s Identity Provider. - Avoid the creation of new credentials and therefore, have the user memorize a new set of user/password to connect to Anypoint Platform. It is also common for customers to use OKTA, OpenAM, AWS Cognito, Microsoft Azure, etc. as their Identity Provider. And with it, it is normal to configure Anypoint Platform to connect with them using OpenID or SAML. I have another post (in Spanish) on how to make the connection with OKTA and Anypoint Platform. You can find it [here](https://mulesoftdevelopersmx.blogspot.com/2020/10/mulesoft-anypoint-platform-openid.html). On the internet, you will find information about how to configure SSO for Anypoint Platform, but with Identity Providers like OKTA or the ones I’ve mentioned at the beginning of this article. In this post, we will talk about how to do it using Oracle Identity Cloud Services (IDCS), which is another Identity Provider alternative, and it may be an interesting one, if you are using MuleSoft to connect to Oracle SaaS Applications, for example. ## Prerequisites The prerequisites for making this configuration are: - An active Anypoint Platform Instance. It can be a 30-day trial. - An active Oracle Identity Cloud Services tenant (can be a 300 credits free trial). - Some knowledge on SAML 2.0. - Understanding the role of an Identity Provider and a Service Provider. You can read [this article](https://www.okta.com/identity-101/why-your-company-needs-an-identity-provider/) to understand their differences. In our case, Anypoint Platform is acting as the Service Provider and Oracle IDCS is working as the Identity Provider (IdP). ## IDCS Configuration The first thing we need to do is to create an application in Oracle Identity Cloud Service that will generate the metadata that later we will import into Anypoint Platform. Let’s do it! Log in into your Oracle IDCS tenant and create a new application: ![Oracle IDCS Add Application dialog with SAML Application option highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-1.png) Click on SAML Application, the following screen will appear: ![App Details form with Name and Application URL / Relay State fields highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-2.png) Fill in the parameters: - Name: This can be anything and is the name that will represent your Application. In my case: MuleSoft Anypoint Platform - Application URL/Relay State: This needs to be[https://anypoint.mulesoft.com](http://anypoint.mulesoft.com/) Then click on the Next button: ![Step wizard at Details step with the Next button highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-3.png) At the top of the next screen you will see this: ![SSO Configuration tab with Download Identity Provider Metadata button highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-4.png) Click on Download Identity Provider Metadata. That will generate an XML file with all the information from the Identity Provider (IDCS), which we will use to import at Anypoint Platform. ## Anypoint Platform Configuration Leave the IDCS screen open and log in to[https://anypoint.mulesoft.com](https://anypoint.mulesoft.com/) in another tab of your browser. At the main menu, click on Access Management: ![Anypoint Platform menu with Access Management highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-5.png) Once there, head into Identity Management: ![Access Management sidebar with Identity Providers menu item highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-6.png) You will see that there is a default Identity Provider: ![Identity Providers list showing the default Anypoint User Credentials provider](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-7.png) Which is Anypoint itself. And there is where the users are being created and maintained. But the intention of this post is to add a new Identity Provider and connect it to Oracle IDCS using SAML. Just click on the Add Identity Provider blue button and select SAML: ![Add Identity Provider dropdown open with SAML 2.0 option highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-8.png) Once you are on the SAML 2.0 configuration page, it will allow you to import the metadata XML file that we’ve downloaded from the IDCS console in previous steps: ![New Identity Provider page with Import IdP Metadata file chooser highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-9.png) Click on the choose file, browse to the location where you’ve downloaded it, and import it. You will see that it will fill almost all the parameters on the screen: ![SAML config fields auto-filled from metadata: Sign On URL, Issuer, Public Key, Audience](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-10.png) The ones that are marked with orange were filled automatically, and the blue ones are the ones where you need to make some decisions: - Name: This is an arbitrary name that will identify this configuration. - Audience: This is also an arbitrary name, but in this case, this parameter is very relevant, since it will match with the configuration at the IDCS side. Then just click on Save Changes: ![Single Sign-On Initiation set to Both with the Save Changes button](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-11.png) You will have two Identity Providers configured: ![Identity Providers list now showing both Anypoint and Oracle IDCS](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-12.png) ## Get Important Information Before we get back to the Oracle IDCS console to finalize the configuration, we need to make a couple of things: - Obtain the audience that needs to match with the Entity ID at the IDCS side - Obtain the assertion consumer URL - Obtain the certificate key from Anypoint - Get the login URL that your Anypoint users will use to log in through Oracle IDCS To get those four things, simply click on the identity provider that you have just configured (in my case Oracle IDCS): 1. Get the audience that needs to match with the Entity ID at the IDCS side: ![Public Key field and highlighted Audience value rolps.anypoint.mulesoft.com](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-13.png) 2. Assertion consumer URL is taken from here: ![Configuration tab showing the Assertion Consumer Service (ACS) URL field](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-14.png) 3. The certificate key is taken from the Anypoint Keys tab. Once there, simply click on the download button that I am marking in yellow. This will download a .pem file: ![Anypoint keys tab with the primary key's download button highlighted in yellow](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-15.png) 4. The login URL can be copied from here: ![Identity Providers list with the external login domain URL highlighted](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-16.png) ## Putting It Together Now get back to Oracle IDCS console and fill the next parameters, accordingly to what we have just explained in the previous points: ![Oracle IDCS General SSO settings: Entity ID, Assertion Consumer URL, NameID, Signing Certificate](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-17.png) - In the Entity ID, copy the value of the audience that we’ve copied from the Anypoint Platform console - In the Assertion Consumer URL, copy the value that we’ve copied from the Anypoint Platform console - In the Signing Certificate, upload the .pem file that we’ve downloaded from the previous steps We are almost done, we just need to map the email attribute that will be returned from Oracle IDCS and that Anypoint Platform will map it: ![Attribute Configuration mapping an email attribute to the Primary Email user value](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-18.png) We just need to add **email** with the configuration we are showing in the image. After that, save your Oracle IDCS application and activate it. We are ready to test. Open a new browser and type the URL that we’ve copied from previous steps. It must be something like this: ``` https://anypoint.mulesoft.com/login/domain/ ``` You will see this: ![Anypoint login page with a Continue with Oracle IDCS button above username and password fields](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-19.png) You now have two options to login to Anypoint Platform and as you can see the name of your Identity Provider Configuration appears in the button. If you click there, you will be redirected to Oracle IDCS default login form: ![Oracle Cloud sign-in form in Spanish with username and password fields](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-20.png) Use your Oracle IDCS credentials, and you will get logged in: ![Anypoint Platform dashboard after login showing Design Center and DataGraph tiles](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-21.png) You will see this in the Users list at the Anypoint Platform: ![Anypoint Users list with the Identity Provider column showing Anypoint and Oracle IDCS](../../assets/blog/anypoint-platform-single-sign-on-sso-saml-configuration-with-oracle-idcs-part-1-22.png) (I am deleting sensitive data for security reasons). But you can see that it mapped the Username with the email provided by Oracle IDCS, and the email. You can also verify that the Identity Provider for that user is Oracle IDCS. But you can be questioning yourself, what happened with the name? Why was it not mapped? **Well, we will talk about that in the next article. We will elaborate on how to map attributes and roles coming from the Identity Provider, and mapped in Anypoint Platform.** --- ## How to check if a key is present in a JSON payload in DataWeave with these 3 examples Source: https://prostdev.com/post/how-to-check-if-a-key-is-present-in-a-json-payload-in-dataweave-with-these-3-examples | Published: Jan 18, 2022 | Category: Tutorials “In DataWeave, when a field in the input payload has a value and you append a **question mark (?)** at the end of the field, it will return **true**. On the other hand, if the field doesn’t exist, then it would return **false**.” Refer to the link below for more information: [https://blogs.mulesoft.com/dev-guides/how-to-tutorials/implement-logic-handling-in-dataweave/](https://blogs.mulesoft.com/dev-guides/how-to-tutorials/implement-logic-handling-in-dataweave/) Here are 3 examples where you can see how to check if a key in the input JSON payload is present or not. ## 1. Single Object In this example, I have used a single JSON object to find if a key is present or not and the result will be a boolean value. **Input:** ```json { "id":1, "company":"abc", "Address":"USA", "Phone":"123" } ``` **DataWeave Expression:** ```dataweave %dw 2.0 output application/json --- payload.id? ``` **Output:** ``` true ``` ## 2. Multiple Objects For this example, we have multiple objects in an array and we are going to check if the key is present or not with different DataWeave Expressions which will give us different results. In **DataWeave Expression 1** the output returned is just a boolean value which helps to combine the result. But we won't be able to know which object has **id** present in it. It just checks if there’s at least one value that matches the key and gives us a boolean value. In **DataWeave Expression 2** we will iterate through each item and then we can find out which item has **id** present in it. It gives complete information about a single object and the presence of the key in it. **Input:** ```json [ { "company":"abc", "Address":"USA", "Phone":"123" }, { "id":1, "company":"def", "Address":"UK", "Phone":"345" } ] ``` **DataWeave Expression 1:** ```dataweave %dw 2.0 output application/json --- payload.id? ``` **DataWeave Expression 2:** ```dataweave %dw 2.0 output application/json --- payload map ((item, index) -> item.id?) ``` **Output 1:** ``` true ``` **Output 2:** ```json [ false, true ] ``` ## 3. Nested Objects In this example, we have objects present in another object (nested objects). In this case, we will first go to the parent object and then to the key to find out whether it is present or not. **Input:** ```json { "id":1, "company":"abc", "Address":{ "street":"xyz", "zip":"080" }, "Phone":"123" } ``` **DataWeave Expression:** ```dataweave %dw 2.0 output application/json --- payload.Address.zip? ``` **Output:** ``` true ``` We have learned how we can check if the key is present or not within a different set of inputs and also observed different results obtained. I hope this was useful to you. --- ## MuleSoft Connected with Oracle Streaming Service using the Kafka Connector Source: https://prostdev.com/post/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector | Published: Jan 4, 2022 | Category: Tutorials Apache Kafka is a very popular streaming service, and I could say that it is the leader in the market. Is part of the Apache Initiative, but vendors like Confluent offer a managed service that can be very powerful for organizations who are looking to start using the platform, without the need to control/manage the underlined infrastructure. But there are other options for streaming services that can be also useful for organizations who don’t want to get too much into the Kafka complexity. And in my case, since I am constantly working with Oracle Cloud Infrastructure, I’ve found a very interesting option for Kafka in Oracle Streaming Service. Oracle Streaming Service is a managed service running on top of Oracle Cloud. Is a serverless service, that users just need to provision and start creating streams/partitions that can be used for multiple purposes. With MuleSoft, it is common to have the need to connect to a Kafka Topic to consume a stream of messages. I’ve found that a very common scenario, for example in Retail, where they need to constantly send information from their Point-of-Sale to a central location. And if you think of a retailer who has multiple branches all over the country, then a stream makes a lot of sense. And for integration purposes, to have MuleSoft in the middle to enrich, transform and ultimately push the information to a system-of-records, then this mix of platforms offers you a very strong alternative. In this article we will show how to connect MuleSoft with Oracle Streaming Service, using the Kafka compatibility mode that this Oracle service offers. The first thing we need to do is to create a stream in Oracle Cloud: ![Oracle Cloud Streams list with the blue Create Stream button highlighted](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-1.png) That is a very straight-forward step, we just need to click on that blue button and fill in these parameters: ![Create Stream form with numbered name, compartment, stream pool, and partitions fields](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-2.png) - The name for your stream - Select the compartment where you want to create the stream - Create a New Stream Pool - Define the number of partitions And that’s it. You don’t need to worry about anything extra. Now, let’s move to MuleSoft. Imagine we need to create a MuleSoft application that consumes the stream that we have just created. MuleSoft does not have a connector to OCI Streams, but it does have a connector for KAFKA. Let’s look to a very simple application: ![Anypoint Studio Mule flow with a Kafka Message listener feeding a Logger](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-3.png) As you can see, we are using the KAFKA connector. We have the Message listener processor to listen into the stream and a simple Logger processor to print in the log file the payload/message. Now let’s see what the connector connection is going to ask us: ![Kafka Consumer config with the connection-type dropdown open on SASL/PLAIN Connection](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-4.png) The first thing to highlight is the Connection type. It offers different options: a) Consumer Plaintext Connection b) Consumer Kerberos Connection **c) Consumer SASL/PLAIN Connection** d) Consumer SASL/SCRAM Connection e) Consumer SASL/TOKEN Connection The one we need to use is **option c**. Also notice that we need to configure the bootstrap server, and as you can see, we’ve already filled in that information. We will show you where we get that info in the next paragraphs. After that, we need to define our **Group ID**. That is something you need to define, in my case: `test-consumer-group`. If we scroll down a little bit more, we will find the following parameters: ![Kafka Consumer config showing the lab-stream topic and SASL/PLAIN username and password](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-5.png) Which is: - Topics. In my case `lab-stream` - **SASL/PLAIN**. Which is the username that has access to the streaming service, and the password which is the AUTH Token that you will create in the next steps. Then, a particular step that can be tricky, is that we need to tell the connector settings that we want to use **SSL**, and for that, we need to go to this tab: ![Kafka Consumer Security tab with TLS Configuration set to Edit inline to enable SSL](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-6.png) And enable the TLS Configuration. Even though there will be no configuration at this tab, just please enable it. And that’s it, that is all we need. But the normal question will be: **but from where can I get that information from the OCI Streaming Service?** Well, let’s get back to the OCI Console and click on the Default Pool of your newly created stream: ![OCI Streams list with the stream's DefaultPool stream pool highlighted](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-7.jpg) Once there, at the left of the screen: ![DefaultPool stream pool page with the Kafka Connection Settings link highlighted](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-8.png) Click on the Kafka Connection Settings, It will show you this: ![Kafka Connection Settings showing bootstrap server, SASL connection string, and SASL_SSL](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-9.png) The bootstrap server is what we input in the connection window in MuleSoft. The username is the one that appears in the SASL Connection String box. Look to the element username, and copy the whole string, it will be something similar to this: ``` pedrito/oracleidentitycloudservice/pedrito@me.com/ocid1.streampool.oc1.iad.gz2gqeeegfljtyb2zr6cfxua ``` And for the **password**, you need to create an `AUTH_TOKEN`. For that regard, go to the user page: ![OCI user Auth Tokens page with the Generate Token button to create the password token](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-10.png) Click on the **Auth Tokens**, and generate a new one. Write it down immediately, since this is going to be the only time you will see it. Now we are ready to run our application in our Anypoint Studio and start consuming the stream. Let’s produce a message from the Streaming Service console: ![Test Stream panel producing a sample JSON glossary message to the lab-stream](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-11.png) And from the Console output in our Anypoint Studio we can see the message being consumed: ![Anypoint Studio console logging the JSON glossary message consumed from the stream](../../assets/blog/mulesoft-connected-with-oracle-streaming-service-using-the-kafka-connector-12.png) What do you think? This article is focused on highlighting MuleSoft’s KAFKA connector, to be used with OCI Streaming Services. --- ## CVE-2021-44228: Critical zero-day Log4j vulnerability (Log4Shell) discovered and MuleSoft’s solution Source: https://prostdev.com/post/cve-2021-44228-critical-zero-day-log4j-vulnerability-log4shell-discovered-and-mulesoft-s-solution | Published: Dec 15, 2021 | Category: News Over December 2021, a critical vulnerability has been unraveled pertinent to the popular Apache’s Log4j library. There is a high possibility that the code you have been working on this past few weeks has been using this component. So, let’s understand what is happening and what has happened. ## What is Apache Log4j? Apache Log4j is one of the most popular logging libraries and is a part of the Apache Logging Project. This has been one of the easiest and most effective libraries to enable logging for applications and is extensively used in Java applications. ## What is RCE (Remote Code Execution)? According to [Wikipedia](https://en.wikipedia.org/wiki/Arbitrary_code_execution): “arbitrary code execution (ACE) is an attacker's ability to run any commands or code of the attacker's choice on a target machine or in a target process. An arbitrary code execution vulnerability is a security flaw in software or hardware allowing arbitrary code execution. A program that is designed to exploit such a vulnerability is called an arbitrary code execution exploit. The ability to trigger arbitrary code execution over a network (especially via a wide-area network such as the Internet) is often referred to as remote code execution (RCE).” ## What happened? Recently, a critical zero-day vulnerability has been discovered which impacts this ever-popular logging library. All versions of Log4j2 versions from 2.0-beta9 to 2.14.1 have been affected by this vulnerability and currently, this is being exploited in the wild. This vulnerability enables an option for RCE (Remote Code Execution) on the vulnerable server and has been rated at a severity level of 10.0 by CVSS. This has been globally addressed as **CVE-2021-44228**. > [!NOTE] > To learn more about Log4j 2.x Security Vulnerabilities, visit their [official site](https://logging.apache.org/log4j/2.x/security.html). ## What made this vulnerability so dangerous? [CVE-2021-44228](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228), also known as Log4Shell, is a vulnerability that enables RCE. The fact that it takes only a few lines of code to activate this makes this so much more dangerous. The attacker only needs to force into one of the servers to potentially gain access to the entire system which can be done by simply adding just one string to the log. Post this an entire arbitrary code can be uploaded into the application. ## Why did this happen? This potentially happened due to a loophole in the JNDI (Java Naming & Directory Interface) which is an API that enables a Java application to perform lookups on any type of named Java objects. Currently, in this case, the JNDI has not been restricted against unknown look-ups. An attacker can send an artificially constructed request to an application employing a vulnerable version of Log4j. This request then can enforce an LDAP / LDAPS injection by using JNDI (which is not validating the authenticity of the incoming request) which then sends a malicious / ill-intended payload to the application from one of the attacker-controlled servers. This payload can contain a piece of arbitrary code (or a Java class) that can enable RCE. ## What can be done to fix this? There are a couple of ways which have been discovered to fix this: - One can update the Log4j package to version `2.17.0` which has been recently released by Apache. (**RECOMMENDED**) - In the case of Log4j versions from 2.10 to 2.14.1, one can update the system property `log4j2.formatMsgNoLookups` or the environment variable `LOG4J_FORMAT_MSG_NO_LOOKUPS` to `true`. - For Log4j versions before 2.10, one can simply remove the `JndiLookup` class from the classpath: `zip -q -d log4j-core – *. Jar org / apache / logging / log4j / core / lookup / JndiLookup .class`. ## How to fix it in MuleSoft **CloudHub** customers: If you are running on CloudHub, MuleSoft has released the fix for this vulnerability. The fix is scheduled to be applied to all runtimes in CloudHub starting Dec 11th, 2021 9 PM PDT to Dec 14th, 2021 10 PM PDT. **On-Premise** customers: This issue requires immediate attention. MuleSoft strongly recommends all on-premise customers take action to update their Mule runtimes as soon as possible. Mule runtime engines associated with the following products will need to be patched as well: - Anypoint Studio - Runtime Fabric (RTF) - Pivotal Cloud Foundry (PCF) - Private Cloud Edition (PCE) For details about how to fix this issue in Mule 4 or Mule 3, please follow the instructions on this MuleSoft article: [https://help.mulesoft.com/s/article/Apache-Log4j2-vulnerability-December-2021](https://help.mulesoft.com/s/article/Apache-Log4j2-vulnerability-December-2021) ## How to update log4j from Maven Go to `mvnrepository.com` and search for `log4j-core` or click on [this link](https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core). This will show you the latest version available at the top and if there are any vulnerabilities discovered for any specific version. ![mvnrepository.com log4j-core page listing versions with vulnerability counts per release](../../assets/blog/cve-2021-44228-critical-zero-day-log4j-vulnerability-log4shell-discovered-and-mulesoft-s-solution-1.png) Click on version `2.17.1` to get the code for your `pom.xml` file. This will open a new view: ![log4j-core 2.17.1 detail page showing the Maven dependency XML snippet to copy](../../assets/blog/cve-2021-44228-critical-zero-day-log4j-vulnerability-log4shell-discovered-and-mulesoft-s-solution-2.png) Simply click on the XML text under **Maven** and this will copy the code to your clipboard. Go to any of your Mule projects and open the `pom.xml` file located at the root folder of your project. Paste the copied code into the `` section. ![pom.xml dependencies section with the log4j-core 2.17.1 dependency block highlighted](../../assets/blog/cve-2021-44228-critical-zero-day-log4j-vulnerability-log4shell-discovered-and-mulesoft-s-solution-3.png) That’s it! This will update your log4j version in the `.m2` folder. To verify, you can go to your `.m2` folder and follow this path: `.m2/repository/org/apache/logging/log4j/log4j-core` - there should now be a folder called `2.17.1`. ![Terminal ls -la of the .m2 log4j-core folder listing version subfolders including 2.17.1](../../assets/blog/cve-2021-44228-critical-zero-day-log4j-vulnerability-log4shell-discovered-and-mulesoft-s-solution-4.png) After that, remove the previous versions to make sure you only have the latest version. You can remove other folders using `rm -rf ... `. For example, `rm -rf 2.13.3/ 2.14.0/`. ## Updates since this post was published - Apache Log4j version 2.16.0 is now available - Apache Log4j version 2.17.0 is now available - Added section "How to update log4j from Maven" We hope this information has been helpful in knowing what has happened and what can be done to mitigate the risk. As this issue is still being investigated, we will keep posting with any latest developments on this. Also, you can comment if you want to add anything to this article / if anything needs to be changed. Thank you!! - [Soumyajit Sinha](https://www.prostdev.com/profile/soumyajitsinha888/profile) - [Leonardo Gonzalez](https://www.prostdev.com/profile/32856a2a-f548-4d76-a415-447e4b1aa2d4/profile) - [Alex Martinez](https://www.prostdev.com/profile/alexandramartinez/profile) --- ## Anypoint Studio 7.11.0: New features overview Source: https://prostdev.com/post/anypoint-studio-7-11-0-new-features-overview | Published: Dec 14, 2021 | Category: News Are you already using the newest version of Anypoint Studio? If the answer is yes, let me guide you through the coolest features it has. If the answer is no, go ahead and download it [here](https://www.mulesoft.com/lp/dl/studio) to start playing with it! ## New features Anypoint Studio includes three new features that you will enjoy as a MuleSoft Developer: - Support for Mule Runtime 4.4. - Correlation ID Management. - Improved Logging Capabilities. - DataWeave Updates (DataWeave 2.4 features!). - Mule Tracing Module. - Feature Flags. - DataWeave 2.4. Including language improvements and new modules. - Read larger-than-memory strings automatically. - New modules, functions, types, annotations, and variables. - Helper functions for handling null values. - “Referenced by” function. Now you can see a list of all flow references to a particular flow/subflow and you also can jump there with a few clicks. If you are interested in learning more about Mule 4.4 new features, you will find all the details [here](https://docs.mulesoft.com/mule-runtime/4.4/whats-new-in-mule). Also, if you are an amazing DataWeave programmer, it’s a good idea to take a look at the [docs](https://docs.mulesoft.com/dataweave/2.4/whats-new-in-dw). As soon as you start your Anypoint Studio installation, you will see the following: ![Anypoint Studio 7.11.0 welcome screen highlighting Mule 4.4 and DataWeave 2.4 as new features](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-1.png) ## Support for Mule 4.4 In this new version of Studio, you will have available the newest version of the Mule Runtime (4.4). When you create a new project just select Mule Server 4.4.0 EE and start enjoying the new features: ![New Mule Project dialog with the Runtime list showing Mule Server 4.4.0 EE selected](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-2.png) As you can notice you are using the Mule 4.4 version. You will find it everywhere in your project (`pom.xml` file, dependencies, and so on). ![Anypoint Studio project with the pom.xml showing the app.runtime property set to 4.4.0](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-3.png) You will also find this version within the `mule-artifact.json` file: ![mule-artifact.json file with minMuleVersion set to "4.4.0"](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-4.png) The new Tracing Module is already in the Mule Palette: ![Mule Palette search for "tracing" listing the Tracing module's logging-variable operations](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-5.png) ## DataWeave 2.4 There are various new DataWeave features, modules, and functions. Let’s try some of them. ### onNull function The onNull function executes a callback function if the preceding expression returns a null value and then replaces the null value with the result of the callback. In the following DataWeave script, we are concatenating ‘Hello’ with the content of the payload. But, what happens if the payload has a null value? We’ll receive an error like this: ![DataWeave error from concatenating "Hello " with a null payload, with the warnings dialog open](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-6.png) If we fix the input payload value, then the DataWeave script will return the expected (concatenated) value: ![DataWeave concatenation returning "Hello World!" once the payload is no longer null](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-7.png) How can we avoid this type of error using DataWeave and particularly the onNull function? ![DataWeave using onNull to replace a null payload, returning "Hello, this is a null payload."](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-8.png) ### hammingDistance function Let’s try another new function that caught my attention: hammingDistance. According to Wikipedia, in information theory, the Hamming distance between two strings of equal length is the number of positions at which the corresponding symbols are different. Let me explain this with an example: We have two strings: - String 1 = 11011001 - String 2 = 10011101 Both strings have eight characters of length and they are different only at positions 2 and 5. So they have two positions at which their characters are different, then the result of the hamming distance should be 2. We can confirm this using the DataWeave function. Notice that the hamming distance function is within the Strings module, so it must be imported explicitly in your DataWeave script. ![DataWeave hammingDistance comparing two 8-bit strings and returning 2 for the two differing positions](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-9.png) Those are only two of the new functions that were included in DataWeave 2.4. Please explore the docs and enjoy all the new features, modules, and functions. ## “Referenced by” function Finally, let’s review this new feature called “referenced by”. This could be useful when you have multiple configuration files within a complex Mule application and you want to jump to the flows that are referencing a particular flow or subflow. The only thing you need to do is right-click on the flow (or subflow) you are currently working on and then select the “Referenced by” option (Ctrl+Shift+G). ![Right-click flow menu in Anypoint Studio with the "Referenced by" option highlighted](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-10.png) The result will be a list of flows that are referencing that particular flow and then you can jump to any of them. ![Mule Configuration tree listing the two flow references to the sub-flow as Ref #1 and Ref #2](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-11.png) When you double-click over the reference, automatically you will be positioned on that particular flow reference: ![Double-clicking a reference jumps the canvas to the sub-flow's Flow Reference in parent-flow](../../assets/blog/anypoint-studio-7-11-0-new-features-overview-12.png) It is cool, isn't it? Now you have explored the newest version of Anypoint Studio, please let us know what do you think? What are the new things you found on it? And… happy coding! --- ## Retry and Reprocess flow design with Anypoint MQ Source: https://prostdev.com/post/retry-and-reprocess-flow-design-with-anypoint-mq | Published: Nov 16, 2021 | Category: Tutorials *GitHub repository with the Mule project and the spring boot application can be found at the end of the post.* In any MuleSoft project, when Anypoint MQ comes into the picture there is always a requirement of retry and reprocess mechanism. Based on my experience I have prepared this blog to show how easily we can design retry and reprocess mechanisms. ## Requirement The main flow of the Mule app will subscribe to a queue and the payload has to be sent to the backend via an HTTP call. While sending the subscribed payload to the backend via HTTP. If the backed system is down (during sending the HTTP request), how are we going to define our flow to perform the retry and reprocess mechanism? ## Step 1 Put the HTTP request inside an **until-successful scope**. Which is a default scope provided by MuleSoft for retrying any event. Define the below parameters in the scope: - **maxRetries**: Number of retries the request will do to try to connect with the backend service. - **millisBetweenRetries**: After the retry performs, how much time the application should hold for the next retry. Code for backend call using until-successful: ```xml ``` ## Step 2 If we can't connect to the backend even after using the until-successful scope (retry process), we have to introduce a reprocess mechanism. To enable the reprocess mechanism, we have to call a subflow (which will perform reprocess) from the "**On Error Continue**" of the main flow. > [!IMPORTANT] > We have to acknowledge the subscribed queue in both HAPPY or ERROR scenarios. If we don't do that, the queue will hold the message and it may lead to a duplicate process. ![Mule flow with HTTP Request in a Retry to connect scope and an On Error Continue reprocess subflow](../../assets/blog/retry-and-reprocess-flow-design-with-anypoint-mq-1.png) ## Step 3 To introduce the reprocess mechanism we have to define **two** queues. **1. Dead letter queue** This queue will work as an intermediate queue. This means, after failing the retry process we will store the message in this queue for re-processing. While sending the message to DLQ, we are going to pass two values in MQ properties: - **reprocessCount**: This value is going to be used as a counter of the reprocess call. - **qname**: The main queue name from which our application is subscribing to the messages. Apart from the above two MQ properties, we have to define a property in the properties file. Which will be the maximum re-process count. - **maxUserRegistrationReprocessAttempts: "3"** - This means the reprocess will happen (Main Queue - DLQ) only three times. **2. Parking queue** This queue will be treated as the final destination of the processing message. This means if both mechanisms (retry and reprocess) fail, the message will be stored in this queue (which will reprocess again). The reason for introducing the parking queue is 100% delivery/processing of each message (at any condition). We can introduce an additional flow, which will reprocess the landed message from parking to the main queue. We can define this in many ways. One of the ways is: Once the parking queue is filled with a certain limit an email will trigger to the support team. After receiving the email, the support team can trigger an endpoint that will start pulling the messages from the parking queue and pushing them to the main queue (using parallel for each component). ## How to test the POC Deploy the Mule app in Anypoint Studio and the spring boot app in eclipse or CMD. Define the main queue, dead letter queue, and parking queue according to your requirements and update the CI/CS of the Anypoint client app (by using the MQ portal). ![Properties file AnypointMQ config with client_id, queue, deadLetter, and parking queue settings](../../assets/blog/retry-and-reprocess-flow-design-with-anypoint-mq-2.png) **Happy flow**: After the successful deployment of both Mule and the backend application, send a message as shown below. The flow will execute without any error. ![Anypoint MQ console sending a JSON user-registration message to the queue](../../assets/blog/retry-and-reprocess-flow-design-with-anypoint-mq-3.png) **Error flow**: To test the retry and reprocess flows, we have made the backend unavailable. So, we have to deploy only the Mule app (should not deploy the spring boot app) and send the message like above and observe the process in the log. After performing the retry and reprocess flows, the message will land in the parking queue as below. ![Parking queue showing a JSON message with reprocessData that landed after retries failed](../../assets/blog/retry-and-reprocess-flow-design-with-anypoint-mq-4.png) ## Conclusion This POC is just to give an idea about retry and reprocess design using Anypoint MQ. There will be a lot of scopes to enhance the POC and prepare as per your requirement. Finally, I will be more than happy to receive your valuable feedback. ## GitHub repository Kindly find the complete Mule and backed (spring boot) application [here](https://github.com/anky123/ankuran.online.mule4/tree/master/project07%20%7C%20retry-process-with-mq). --- ## How to configure Solace DMQ (Dead Message Queue) through the Mule JMS connector Source: https://prostdev.com/post/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector | Published: Nov 9, 2021 | Category: Tutorials In this post, I will be explaining how to configure Solace DMQ through the Mule JMS Connector. There can be many business scenarios where the processing of the message will be delayed by a predefined amount of time. In such scenarios, we can make use of the concept of the TTL property and the DMQ. ## Dead Message Queues By default, Guaranteed messages are removed from a durable endpoint's message pool and discarded when: - The number of redelivery attempts for a message exceeds the Max Redelivery value for the original destination endpoint; - Or, a message's Time-To-Live (TTL) value has been exceeded and the endpoint is configured to respect message TTL expiry times. If you don't want the messages to be discarded, they can be moved to a dead message queue (DMQ) assigned to the endpoint. ## How DMQs Work Any durable queue on the same Message VPN as the endpoint that the messages are being removed from can be assigned as a DMQ. Messages flagged as DMQ-eligible by the publishing client will be sent to the endpoint's DMQ rather than be discarded. However, if an endpoint's assigned DMQ doesn't exist, then the removed Guaranteed messages will still be discarded even if they are DMQ-eligible. By default, each durable endpoint is assigned a DMQ named `DEAD_MSG_QUEUE`, but this queue does not pre-exist and must be created by a management user. Now let us see the steps involved in setting up DMQ and TTL. ## 1. Enabling the "Publisher Messages DMQ Eligible" property in the JMS JNDI of the Message VPN Log in to Solace Cloud Message VPN. (Please refer to my first blog, “[How to integrate Solace PubSub+ Cloud with MuleSoft](https://www.prostdev.com/post/how-to-integrate-solace-pubsub-cloud-with-mulesoft),” which explains how to create a Solace Cloud Trial account and how to spin up a Service in Solace). Click on JMS JNDI --> Click on Connection Factories --> Click on /jms/cf/default --> Scroll down and enable the property “Publisher Messages DMQ Eligible” as highlighted in the screenshot below. ![Solace JMS connection factory settings with "Publisher Messages DMQ Eligible" toggled on](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-1.png) ## 2. Creating Queues and Topics Create the new Queues and their corresponding Topics as shown below. As part of the POC, our queues of interest are highlighted below. ![Solace Queues list highlighting test-queue-1 and test-queue-2](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-2.png) ## 3. Setting up DMQ and TTL in the “test-queue-1” queue In the advanced settings of the `test-queue-1` enable the ‘**Respect TTL’** and set the ‘**Maximum TTL**’ as 180 seconds (this can be set based on our need). Also, set the ‘**DMQ Name**’ as `test-queue-2`. As per this setting, whenever any message reaches `test-queue-1`, the message stays in this queue for 180 seconds before it’s pushed to DMQ (`test-queue-2`). ![test-queue-1 settings: DMQ Name test-queue-2, Respect TTL on, Maximum TTL 180 seconds](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-3.png) ## 4. Publishing Message to Solace Topic with SOLACE_JMS_PROP_DEAD_MSG_QUEUE_ELIGIBLE set to true In the `solace-jms-publish.xml`, I added an HTTP Listener to be able to listen to a request with the content that I want to publish into the Solace Topic. Here you can see how I have configured the JMS Publish connector: ![Anypoint Studio publish flow with HTTP Listener and JMS Publish to a Solace topic](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-4.png) Also, I have set the property `SOLACE_JMS_PROP_DEAD_MSG_QUEUE_ELIGIBLE` to `true` in the **JMS Config Provider Properties.** ![JMS Config provider properties listing SOLACE_JMS_PROP_DEAD_MSG_QUEUE_ELIGIBLE](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-5.png) ![dev.yaml properties file with the Solace JMS config and dmqEligible set to "true"](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-6.png) ## 5. Publishing the message from Postman ![Postman POST to localhost:8081/publish returning 200 OK "published the message to Solace Topic"](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-7.png) Now the message is published to the Topic with ‘**DMQ Eligible**’ set to ‘**Yes’**. This message stays in the Queue `test-queue-1` for 180 seconds before which it will be published to DMQ (`test-queue-2`) automatically for further processing. ![test-queue-1 Messages Queued tab showing one message with DMQ Eligible Yes](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-8.png) ![test-queue-2 (the DMQ) Messages Queued tab showing the message after the TTL elapsed](../../assets/blog/how-to-configure-solace-dmq-dead-message-queue-through-the-mule-jms-connector-9.png) ## Recap Now we know how a delay in the processing of the messages can be achieved in Solace using the DMQ and TTL feature. Thanks for reading my post and I hope it will help! That’s it for now! See you in the next post! -Pravallika --- ## How to Handle Single and XA (Extended Architecture) Transactions in MuleSoft Source: https://prostdev.com/post/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft | Published: Oct 12, 2021 | Category: Guides "Transaction" is a word that is used way too frequently in our daily lives. We use it whenever we are engaged in purchasing or selling any commodity. In our daily lives, a transaction signifies an instance of buying or selling something. Also, a transaction can be deemed as completed only upon the overall completion of the buying/selling activity along with the monetary affair associated with it. In the software industry, we use the word ‘transaction’ in multiple areas. Predominantly, it is used while discussing SQL queries. Transactions make up a very important component of any software development life cycle (SDLC). Let’s see how transactions are changing lives! A transaction is a group of operations where all the steps are needed to be successful to commit a result. If any one of the intermediate steps fail, the whole chain of steps fails collectively. Let’s understand it using a pictorial reference. ![Two transaction flowcharts: one committing all four steps, one rolling back after an error at step 2](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-2.png) Let’s consider a case where we have a chain of steps that represents the following: | Step | Action | Table involved | |---|---|---| | Step 1 | This is a SQL fetch operation that fetches information on a certain deal | Table A | | Step 2 | This is a SQL insert operation that inserts the new record after consolidating the data received in the previous step | Table A | | Step 3 | This is a SQL delete operation that deletes the previously available record | Table A | | Step 4 | This is a SQL update operation that updates the amount update in another table | Table B | We want all the steps to be completed collectively or else we want none of them to happen. In layman’s terms, we want the event to be committed only if all the steps are successful. If any intermediate step fails, we do not want the event to be committed and we do not want any step to be actioned upon. In the above picture, we have encapsulated these 4 steps in a **transaction**. The first diagram starts and completes without any errors and gets committed. But in the second diagram, we can see that in step 2 we encountered a certain error. Now, we don’t want the event to be committed. This is made possible by the **transaction** action. It will make sure that all the steps are rolled back to the initial state from which the event was kicked off. Now that we are clear on what transaction is and how it helps in software development, let’s indulge ourselves in bringing it to life in MuleSoft. MuleSoft supports two types of transactions: - Single resource - XA transactions (Extended Architecture) Let's talk about them in detail. ## Single Resource transactions Let's consider a Mule flow where we are dealing with a single mode of data transaction. Consider our flow is having only JDBC affairs or JMS activities or VM Queues. We have created a flow in which we are dealing with 2 tables connecting to the same database. This is how the flow looks like: ![single-resource-example flow in a Try scope: numbered Select, Insert, Delete, and Update database operations](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-3.png) Here, the following database actions are carried out: | Step | Action | Table involved | |---|---|---| | Step 1 | A JDBC select operation (**1**) fetches information on a certain publication | Table A | | Step 2 | A JDBC insert operation (**2**) inserts the new record after consolidating the data received from the user & the data received in the previous fetch operation | Table A | | Step 3 | A JDBC delete operation (**3**) deletes the previously available record in the table in which data is inserted in step 2 | Table A | | Step 4 | A JDBC update operation (**4**) updates the author details of the publication in another table in the same database | Table B | Here, we have encapsulated all the JDBC components within a **TRY** block. **Why use the TRY block?** Try block is helping us to enforce the transaction on the JDBC operations by acting as the initiator of a transaction thread. The JDBC components simply connect themselves to the transaction initiated by the try component. In a try block, you can set how you want to deal with transaction action. ![Try scope settings with the Transactional action dropdown open showing ALWAYS_BEGIN, BEGIN_OR_JOIN, INDIFFERENT](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-4.png) The try-block provides us the following transaction actions: | Transactional action | Description | |---|---| | `ALWAYS_BEGIN` | It will always initiate a new transaction | | `BEGIN_OR_JOIN` | It will initiate a new transaction if there are no preceding transactions. In case there is a transaction that has already been set up, the try block will join that transaction | | `INDIFFERENT` | It will be neutral. It will neither create a transaction nor join an existing transaction | *For more information on the Transactional Actions, please refer to the* [official documentation](https://docs.mulesoft.com/mule-runtime/4.3/transaction-management#transactional-actions)*.* Once you have defined the transaction action, you can go ahead and set the transaction type. ![Try scope settings with the Transaction type dropdown open showing LOCAL and XA options](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-5.png) The try-block provides us the following transaction actions: | Transaction type | Description | |---|---| | LOCAL | Single Resource | | XA | Extended Architecture Transactions | In our case, since we are dealing with Single Resource (only JDBC operation), we will be selecting the LOCAL transaction type and we will let the try-block initiate the transaction. Once the transaction has been kicked off by the try block, all we need to do is to instruct our JDBC components to JOIN this transaction. We can do that simply by going to the “advanced” section of the JDBC operator. ![Database operation Advanced tab with the Transactional action dropdown set to ALWAYS_JOIN](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-6.png) We will select `ALWAYS_JOIN` as the transactional action so that it joins the transaction initiated by the try block earlier. We will be selecting the same option for all our JDBC components. Once this is done, we can go ahead and deploy our code. ## Extended Architecture transactions Let’s consider a case when we need to deal with multiple data exchange sources. For example, if we need to use a JMS broker along with a JDBC operation and VM queues and we need all of these operations to be performed atomically. We want all these operations to be performed following the ACID (Atomicity, Consistency, Isolation & Durability) concept. | ACID property | Description | |---|---| | Atomicity | The entire transaction should be performed at a single go with all the components being successful | | Consistency | The results from all the operations should be consistent across all the sources/data exchanges | | Isolation | The operations should be performed separately and sequentially. There should be no overlap | | Durability | Once the transaction is committed, then under no circumstance it should be rolled back | The XA transaction follows a **2-Phase-Commit protocol**. The first phase of the 2PC is used to check whether all the actions/steps (we can consider these steps to be our JDBC/JMS components) in a transaction have been successfully completed. If all of them pass or get completed, the transaction manager commits the transaction and writes the corresponding completion logs which can be found in a file with a name ending with “tx-logs” in the .mule folder of your workspace directory. If any of the components fail execution, the transaction manager rolls back the entire transaction. We will be using **Bitronix Transaction Manager** as our transaction manager to accomplish this. We can add this in our Global Elements section in Anypoint Studio. ![Two dialogs: choosing the Bitronix Transaction Manager global type, then its properties with OK highlighted](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-7.png) To add **Bitronix Transaction Manager**, head on to the Global Elements section and search for Bitronix (left image). Once done, another window (right image) will pop up. Click on “OK” and you are all set! Let’s try this out in Anypoint Studio. This is how the flow will look like in this case: ![extended-architecture-example flow adding a fifth step to publish to a JMS topic across two databases](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-8.png) Here, the following database actions are carried out: | Step | Action | |---|---| | Step 1 | A JDBC select operation (**1**) fetches information on a certain publication from a table in **Database A** | | Step 2 | A JDBC insert operation (**2**) inserts the new record after consolidating the data received from the user & the data received in the previous fetch operation in a table in **Database A** | | Step 3 | A JDBC delete operation (**3**) deletes the previously available record in the table in which data is inserted in step 2 from a table in **Database A** | | Step 4 | A JDBC update operation (**4**) updates the author details of the publication in another table in the **Database B** | | Step 5 | Publish the author update details in a JMS Queue (**5**) | Compared to what we have done in our implementation of Single Resource transaction, we will do the same thing with the try block, JDBC & JMS components. The only difference is that we have to intimate Mule that we will be using XA transactions. To do that, we have to select XA as “transaction type” in the Try scope settings. For JDBC & JMS components, this is configurable in their Global Settings. In the Database connection config palette, we can head to the “transactions” section where we will have a tick-box to turn the XA transaction on. ![Database Config Transactions tab with the Use XA Transactions checkbox ticked](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-9.png) In the JMS connection config palette, we need to scroll down to the “Connection Factor” section. Select “Edit inline” for the “Factory configuration”. Then we will have a tick-box to turn the XA transaction on. ![JMS Config with No caching strategy and the Enable xa checkbox ticked in the connection factory](../../assets/blog/how-to-handle-single-and-xa-extended-architecture-transactions-in-mulesoft-10.png) > [!NOTE] > For JMS Connection, XA will not work when caching is activated. Please select “No caching” as the Caching Strategy and then click on “Test Connection…” to see whether the connection is successful or not. We will select `ALWAYS_JOIN` as the transactional action so that it joins the transaction initiated by the try block earlier. We will be selecting the same option for all our JDBC and JMS components. Once this is done, we can go ahead and deploy our code. So, now we have the knowledge of what a transaction is and how to set up one in MuleSoft. Apart from that, we have talked about the various kinds of transactions and how to consider one based on the requirement. Following this up will be an article in which we will take a deeper dive into other intricacies of transactions. Thank you!! --- ## How to create a README file for your GitHub profile, part 2: Markdown basics Source: https://prostdev.com/post/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics | Published: Sep 28, 2021 | Category: Tutorials *GitHub repository with the final `README.md` file can be found at the end of the post.* Having a [GitHub](https://github.com/) account in today’s world is a **must-have** for any software developer. In this series of posts, I’ll show you how to create a striking `README.md` file to attach to your GitHub profile. If you want to see an example of the final result, take a look at [my GitHub profile](https://github.com/alexandramartinez)! If you’re new to Git, GitHub, or `README.md` files, make sure to check out the previous post of this series: [How to create a README file for your GitHub profile, part 1: Intro to Git, GitHub, and README files](https://www.prostdev.com/post/how-to-create-a-readme-file-for-your-github-profile-part-1-intro-to-git-github-and-readme-files). Now, I’ll guide you through the basics of creating a Markdown file. ### What is Markdown? [Wikipedia](https://en.wikipedia.org/wiki/Markdown) states that “Markdown is a lightweight markup language for creating formatted text using a plain-text editor. ... Markdown is widely used in blogging, instant messaging, online forums, collaborative software, documentation pages, and readme files.” In summary, it’s a language used to write files using formatted text (like bold or italic characters, bulleted lists, numbered lists, headers, links, code, etc.). Markdown files end in `.md` (like `README.md`). I like to look at it as simplified [HTML](https://en.wikipedia.org/wiki/HTML). But enough theory. You’ll learn more with practice. ### Create the repository to keep the README If you don’t already have a GitHub account, please create one first. Once you have it, click on your profile picture at the top-right of the page and click on “Your repositories.” ![GitHub profile dropdown menu with Your repositories highlighted](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-1.png) Now click on the green “New” button. ![GitHub Repositories tab toolbar with the green New repository button](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-2.png) **This is important** - you have to name your new repo exactly the same as your username. If you’re not sure what yours is, just click on your profile picture again, and you will see it after the “Signed in as …”. In my case, it’s “alexandramartinez”. You don’t have to add a description. Make sure to select “Public” and the “Add a README file” checkbox like in the screenshot below. ![New repository form set to Public with Add a README file checked, and Create repository button](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-3.png) Click on “Create repository.” You now have a new repo with an (almost) empty README file. Click on the “edit” button on the right side of the `README.md` file. ![README.md preview showing the alexandramartinez heading with an edit pencil icon](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-4.png) This will bring you to the editor view. You also have a Preview tab in order to see the formatting of the file. ![GitHub README editor with Edit file and Preview tabs showing the # alexandramartinez heading](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-5.png) ### How to add headers in Markdown Ok, from this point on, I’ll mainly show you the code and the explanations of the formatting. I trust you’ll use the “Edit file” and “Preview” tabs to make sure it looks as expected. Replace your username with your own title. In my case, I wrote: ``` ## Hi, I'm Alex! ``` The hashtag character is used to specify headers. The single hashtag is the biggest one. You can use ##, ###, ####, and so on to specify what type of header you want to format (each smaller than the last one). In my README file, I’m just using the first title and the following header (H1 and H2). This is how my file looks so far, with all my headers and a brief introduction: ``` ## Hi, I'm Alex! I'm a Software Engineer who started creating content and is now a Developer Advocate! ### About me ### Latest Content ### GitHub Stats ### Connect ### MuleSoft Repos ``` ### How to add bulleted lists in Markdown Now, in the “About me” header, I am using a bulleted list to show my current positions. To create this list, you just have to use a dash (-) and leave a space between the dash and the following character. Like this: ``` - Developer Advocate at MuleSoft - Founder & Content Creator at ProstDev - Lead at Women Who Code Monterrey ``` ### How to add hyperlinks in Markdown I also want to add links to the companies I mentioned in this list. The syntax to add links in Markdown is the following: ``` [Text to display](link) ``` Here is what it looks like with the 3 links added to MuleSoft, ProstDev, and Women Who Code Monterrey: ``` - Developer Advocate at [MuleSoft](https://www.mulesoft.com/) - Founder & Content Creator at [ProstDev](https://www.prostdev.com/) - Lead at [Women Who Code Monterrey](https://www.womenwhocode.com/monterrey) ``` Finally, feel free to make your introduction a bit more fun or show who you are. In my case, I thought adding emojis was a bit more casual. Yes, you can also add emojis to Markdown! ### Final result (for now) Here’s the result we have so far (following my own README file): ``` ## Hi, I'm Alex! I'm a Software Engineer who started creating content and is now a Developer Advocate! - 🇲🇽 in 🇨🇦 - 🐱 🐶 Eris & Waffle - ❤️ Horror, Disney, Jurassic Park, Coffee, Halloween - 🎮 PS4 & Switch ### 👋 About me - Developer Advocate at [MuleSoft](https://www.mulesoft.com/) - Founder & Content Creator at [ProstDev](https://www.prostdev.com/) - Lead at [Women Who Code Monterrey](https://www.womenwhocode.com/monterrey) ### Latest Content ### GitHub Stats ### Connect ### MuleSoft Repos ``` ![Rendered README preview with the intro, emoji bullets and About me links plus section headers](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-6.png) ### Commit your changes Once you feel happy with your changes, scroll-down until you see the “Commit changes” form and click the green button. ![GitHub Commit changes form with the default Update README.md message and green Commit button](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-7.png) You can add a title to the commit, so you remember exactly which changes you did here, such as “Added intro and about me to `README.md`”. Note that if you don’t write anything, GitHub will automatically add the title “Update `README.md`,” so you don’t *have to* add a title/description to the commit, but it’s a best practice to keep in mind for your future projects. ![Commit changes form with a custom message Added intro and about me to README.md](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-8.png) You’ll notice that there’s a green square on the right side of your repo that says this is a special repository. ![GitHub special-repository banner noting the README will appear on your public profile](../../assets/blog/how-to-create-a-readme-file-for-your-github-profile-part-2-markdown-basics-9.png) You should now be able to see this README file in your GitHub profile! If you have any issues, you can refer to the [official documentation here](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-github-profile/customizing-your-profile/managing-your-profile-readme) or leave me a comment to troubleshoot! ### Recap - You need to create a repository with the same name as your GitHub username. - The hashtags in Markdown are used to specify a header or a title. “#” is the main title, and the following ones can be used as header 2 (##), header 3 (###), and so on. - Bulleted lists can be added in Markdown using the dash (-) character and leaving a space between the dash and the next character/word. - Hyperlinks can be added in Markdown using this syntax: [Text to display](link). For example, [ProstDev](https://www.prostdev.com/). - Remember to commit any changes done to the `README.md` file to your repository! Don’t miss any articles! Remember to subscribe at the bottom of the page to receive email notifications as soon as new content is published. ✨ -Alex ### GitHub repository [ProstDev - alexandramartinez](https://github.com/ProstDev/alexandramartinez) --- ## MuleSoft Runtime Fabric Deployed on Oracle Cloud Infrastructure (OCI) - Part 2: Mgmt & Operations Source: https://prostdev.com/post/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations | Published: Sep 21, 2021 | Category: Tutorials This is the second part of a series of articles focused on MuleSoft Runtime Fabric. In our [previous article](https://www.prostdev.com/post/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1), we’ve described how to deploy a full MuleSoft Runtime Fabric on top of Oracle Cloud Infrastructure. Now is the time to learn some mechanisms to manage that cluster. We are going to describe some tips about how to: - The process of deploying an application into the Runtime Fabric - The process to review the logs produced by the deployment process - Basic but useful commands to understand what is happening within the Runtime Fabric (namespaces, pods, secrets, etc) - Consolidate the log files into an external system (we are going to use Papertrail for our article) - Destroy pods - Test our applications - The usage of the rtfctl CLI Plus, a set of tips that are going to be useful for you while managing and maintaining your Runtime Fabric. Let’s start from the beginning. ## How to know if my cluster is up and running There are different ways to validate the health of your runtime fabric. Let’s elaborate on a couple of them: - Via Anypoint Platform console ([https://anypoint.mulesoft.com](https://anypoint.mulesoft.com/)) - Via the rtfctl CLI. We will get into some details with this one For the first option, you can get a very quick view by going into Runtime Manager -> Runtime Fabrics: ![Runtime Manager Runtime Fabrics view with summary and health detail blocks numbered 1, 2, 3](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-1.png) In the first block (circle number 1) you will have the option/link for Runtime Fabrics. In the second block (the purple one or number two in the image), you will get a summary of your Runtime Fabric, and finally, in the third block (the brown one or number 3) you will get a little bit of information like Application Status, Deployments, and Nodes. Let’s get a deeper look into block [#2](https://www.prostdev.com/blog/hashtags/2) (the purple one in the image): ![Runtime Fabric summary panel showing Active status, version 1.8.23, vCPU and memory](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-2.png) With this piece of information, you can validate: - That your cluster is Active. The other option is that it can be Degraded (we will elaborate more on that in the next paragraphs) or disconnected. The desired status is Active. - Then you can see the version of your Runtime Fabric, and it suggests that you upgrade to the latest one. In my case, I could upgrade to 1.10.0. But that will be part of another article in the upcoming future. - The number of vCPUs that you have for your Runtime Fabric and the Memory. - And a hyperlink to download the rtfctl CLI. If you are an operator and want to see how healthy your Runtime Fabric is, with this small section you can get it. Direct and simple. Now, sometimes things go in directions that we do not control that can make our cluster change its status. In that case in section [#3](https://www.prostdev.com/blog/hashtags/3) (the brown in the previous images) you can get some useful detail: ![Health Details panel: all systems operational with Appliance Status, Deployments and Nodes healthy](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-3.png) If something is wrong with the Application Status, your Deployments, or an event with your Nodes, you can get that detail from here. For example, the following image is from a Degraded cluster: ![Degraded cluster health detail warning that system components are degraded with failing nodes](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-4.png) As you can see, some of the errors that are causing the cluster to be in Degraded status are described in the different sections, in this case: in the Nodes section. For this error, something is wrong with our nodes, to the point that there is no status from them; they are probably down. Now let’s do something similar with RTFCTL CLI. ## RTFCTL CLI Since the very beginning of the computing era, the command-line interface (CLI) has been the common place to manage systems. But in this modern era, CLIs have become even more popular; and it is true to say that sometimes it is much more natural to use those CLIs rather than expecting a web-based console. For MuleSoft Runtime Fabric, there is an alternative for that, and that is called rtfctl. The official documentation is here: [https://docs.mulesoft.com/runtime-fabric/latest/install-rtfctl](https://docs.mulesoft.com/runtime-fabric/latest/install-rtfctl) If you want to install it, you have the following options: Windows: ```bash curl -L https://anypoint.mulesoft.com/runtimefabric/api/download/rtfctl-windows/latest -o rtfctl.exe ``` MacOS (Darwin): ```bash curl -L https://anypoint.mulesoft.com/runtimefabric/api/download/rtfctl-darwin/latest -o rtfctl ``` Linux: ```bash curl -L https://anypoint.mulesoft.com/runtimefabric/api/download/rtfctl/latest -o rtfctl ``` (Quick note: I normally install the rtfctl within the controller nodes of my Runtime Fabrics cluster.) Once you have it, make it available in your PATH (either Windows or Linux), and you can start using it. The available commands are: ![Table of rtfctl commands with descriptions, from appliance and apply to validate and version](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-5.png) (Taken from the official documentation [https://docs.mulesoft.com/runtime-fabric/latest/install-rtfctl](https://docs.mulesoft.com/runtime-fabric/latest/install-rtfctl)). As you can see, there are a lot of options, from getting to know the status of your cluster, to upgrading it. Let’s start from the most basic one: ```bash sudo rtfctl status ``` That command will return the following: ``` Using proxy configuration - (proxy "", no proxy "0.0.0.0/0,.local,0.0.0.0/0,.local,,0.0.0.0/0,.local,0.0.0.0/0,.local")" Using 'US' region transport-layer.prod.cloudhub.io:443 ✔ https://anypoint.mulesoft.com ✔ https://worker-cloud-helm-prod.s3.amazonaws.com ✔ https://exchange2-asset-manager-kprod.s3.amazonaws.com ✔ https://ecr.us-east-1.amazonaws.com ✔ https://494141260463.dkr.ecr.us-east-1.amazonaws.com ✔ https://prod-us-east-1-starport-layer-bucket.s3.amazonaws.com ✔ https://prod-us-east-1-starport-layer-bucket.s3.us-east-1.amazonaws.com ✔ https://runtime-fabric.s3.amazonaws.com ✔ configuration-resolver.prod.cloudhub.io:443 ✔ tcp://dias-ingestor-nginx.prod.cloudhub.io:5044 ✔ State: active Version: 1.1.1606942735-3f99c37 Nodes: controlrtf (11.0.0.5) : Status: healthy rtfworker1 (11.0.0.3) : Status: healthy rtfworker2 (11.0.0.4) : Status: healthy SERVICE HEALTHY agent true deployer true mule-clusterip-service true resource-cache true registry-creds true certificate-renewal true CERTIFICATE EXPIRES AT client 2022-03-12 02:08:43 +0000 UTC (199 days) rtfctl newer version 0.3.135 available; current version 0.3.102. Please use `rtfctl update` to update. ``` It is quite similar to what you get from the Anypoint Platform Web UI. Basically, it is telling us the status/state of our cluster, in this case: **Active**. And it also tells us the health of the controller and the workers, here: ``` Nodes: controlrtf (11.0.0.5) : Status: healthy rtfworker1 (11.0.0.3) : Status: healthy rtfworker2 (11.0.0.4) : Status: healthy ``` One crucial thing for this type of deployment (as described in our [previous](https://www.prostdev.com/post/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1) article) is the connectivity to the Control Plane. In a controlled environment where you have no limitations to connect to the internet, there is no problem to reach the control plane, but for enterprise deployments where you are behind a corporate firewall, you need to be sure that you can connect to all the different endpoints that the Runtime Fabric needs. Those endpoints are very well documented in the official documentation: [https://docs.mulesoft.com/runtime-fabric/1.3/install-port-reqs](https://docs.mulesoft.com/runtime-fabric/1.3/install-port-reqs) Specifically the section: Port IPs and Hostnames to Add to the Allowlist. As explained in our previous article, you must have connectivity to all those endpoints (depending on your region). But, sometimes, your cluster is not able to reach those endpoints, and one way to validate that it is running is the following command: ```bash sudo ./rtfctl test outbound-network ``` And the output is this: ``` Using proxy configuration - (proxy "", no proxy "0.0.0.0/0,.local,0.0.0.0/0,.local,,0.0.0.0/0,.local,0.0.0.0/0,.local")" Using 'US' region transport-layer.prod.cloudhub.io:443 ✔ https://anypoint.mulesoft.com ✔ https://worker-cloud-helm-prod.s3.amazonaws.com ✔ https://exchange2-asset-manager-kprod.s3.amazonaws.com ✔ https://ecr.us-east-1.amazonaws.com ✔ https://494141260463.dkr.ecr.us-east-1.amazonaws.com ✔ https://prod-us-east-1-starport-layer-bucket.s3.amazonaws.com ✔ https://prod-us-east-1-starport-layer-bucket.s3.us-east-1.amazonaws.com ✔ https://runtime-fabric.s3.amazonaws.com ✔ configuration-resolver.prod.cloudhub.io:443 ✔ tcp://dias-ingestor-nginx.prod.cloudhub.io:5044 ✔ ``` With that, you can validate if your Runtime Fabric is fully connected to the control plane. If any of those are not responding, you will notice that your cluster will change into a **Degraded** state. Some of those endpoints can be obvious, for example, [https://anypoint.mulesoft.com](https://anypoint.mulesoft.com/) is the one serving most of the calls (API Manager and Exchange to give you an example). But some others like the amazonaws.com endpoints are used as the docker image repository. What we are trying to explain is that all of them have a specific purpose and will lead the Runtime Fabric to different behaviors. Another useful command is the one used to update your MuleSoft license.lic file. Remember that MuleSoft is a subscription-based platform and every year a new license.lic file is generated. That means that every year you need to update it in your Runtime Fabric. In order to do that, you can execute these two commands: 1. Transform the content of the license.lic file into base64 format with this command: ```bash base64 -w0 license.lic ``` 2. With the base64 string execute this: ```bash sudo ./rtfctl apply mule-license BASE64_ENCODED_LICENSE ``` 3. And if you want to check/validate the license: ```bash sudo ./rtfctl get mule-license ``` ## Deploy applications to the Runtime Fabric The deployment process is no different from what you have for CloudHub or even for a Standalone, from the Runtime Manager perspective. There are key differences to understand, not only for the deployment configuration but the dynamics that happen during and after the deployment. In this section, we will go into those differences. Let’s keep it simple, and let’s deploy an application using Runtime Manager from [https://anypoint.mulesoft.com](https://anypoint.mulesoft.com/). The name of the application is myRTFapp. And it contains a very simple structure: ![Anypoint Studio myrtfapp flow with Listener and Logger, Logger message set to payload](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-6.png) Our listener is configured to serve on port 8081 (HTTP). Since we are on Runtime Fabric, your listener must be listening on 8081 or 8082. The logger is going to be valuable for what we are going to explain in the next paragraphs. But first, let’s go through the deployment process. From the Runtime Manager perspective, the process itself is similar to CloudHub or Standalone Hybrid Model. You have to go into Runtime Manager and deploy the application, selecting your Runtime Fabric as your target: ![Runtime Manager Applications view highlighting the Deploy application button](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-7.png) And then: ![Deploy Application form selecting the ocigen2rtf Runtime Fabric as deployment target](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-8.png) Once you select an RTF-based deployment target, you will get the following different options that you can decide to configure/use while making the deployment: ![RTF deployment options: runtime version, replicas, cluster mode, rolling update and resource allocation](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-9.png) - You can decide the runtime version for your applications. That means that on top of Runtime Fabric you can have different applications using different MuleSoft Runtime versions. - Replicas: This is the desired number of replicas for your application. That means that your application is replicated and therefore can distribute its load along with those replicas. But also, that means that the Runtime Fabric will have to ensure to have those replicas up and running since that is the desired state for your application. That is one of the benefits of this deployment model. - You can run your application in cluster mode. Clustering is one of the options for MuleSoft Runtime, and that is possible to have in Runtime Fabric. If you check that box, your application will run in cluster mode. The checkbox will be enabled if you choose to have more than 1 replica for your application. That makes sense since in order to have a cluster you need at least two nodes. - Deployment Model: This is very useful when you are introducing a change to your application. The rollout will be incremental, updating replica by replica. This will translate into zero downtime for your application. - Reserved CPU, CPU Limit, and Memory. You can decide the amount of computing power your application needs. You can also configure specific properties for your application, just as you have it for CloudHub. And you can also configure the JVM properties for the Runtime. After you click on the **deployment** button several things happen behind the scenes: - The application/artifact is registered on Exchange. - A Docker-based image is uploaded to the Image Repository (AWS). - The Runtime Fabric creates the deployment using the image created in step [#2](https://www.prostdev.com/blog/hashtags/2). - The application gets deployed on top of the Runtime Fabric and a Pod with two containers is created: - One container holds the MuleSoft Runtime with your application. - The second container is a sidecar for the application, for monitoring purposes. The result at the Runtime Fabric level is the following: ![kubectl pod list showing myrtfapp pod in Init:0/1 then PodInitializing status](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-10.png) It starts initializing your pod, in this case: myrtpapp-79b9b4c488. Then it will transition into: ![kubectl pod list showing myrtfapp containers reaching 1/2 then 2/2 Running over 32 seconds](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-11.png) As you can see it started one of the containers (after 8 seconds) and then the second container (after 32 seconds). Those are the two containers that we explained in the previous paragraph. Once you see that both containers are READY and the STATUS is Running, your application is ready to be used. And from the Anypoint Web Console, you have: ![Anypoint console myrtfapp detail showing Running status, 1/1 replicas started on ocigen2rtf target](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-12.png) Up to this point, your deployment process is done. In the next section, we will learn how to get the logs from a particular application. ## Get the log files from my applications and the RTF platform The application is now running in your RTF. It is deployed in a pod and is serving requests. But before getting into the log files, there is something relevant to notice and is related to the RTF underlying infrastructure: Kubernetes. If you are familiar with Kubernetes, one of the main concepts and elements is the namespaces, which are a logical way to organize your applications. For our RTF-based deployment model, that is important, since a namespace will represent an Anypoint environment (for example Sandbox, Production). Therefore, when you deploy an application since you decide in which environment to deploy it, you need to know the ID for that environment in order to perform some actions at the RTF level. > [!NOTE] > the following commands are executed from the controller node. Let’s explain it with some commands: ```bash kubectl get namespaces ``` This will return the list of namespaces for your RTF: ![kubectl get namespaces output listing the Sandbox environment UUID plus rtf and kube namespaces](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-13.png) The first namespace that was returned is: 823d7ade-927a-4b2a-9f1e-2b56b06a6ce0. Which is our SANDBOX environment at Anypoint. How do I know that? Let’s get it from the Web Console. From the API Manager menu: ![API Manager Sandbox administration with the environment-info button highlighted](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-14.png) Click on the desired environment, and then click the button with the exclamation mark. It will return the environment information: ![Environment Information panel showing Sandbox name and its environment ID matching the namespace](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-15.png) Among other things, the environment ID, which matches with the namespace that we’ve explained. An RTF can be associated with one or more environments, it is up to you how you want to organize your applications and environments. That option is available from the RTF configuration page: ![RTF Associated Environments tab with Sandbox checked to associate it with the Runtime Fabric](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-16.png) In my case, Sandbox is the only environment associated with my RTF, and that is why we just see that namespace. With this explanation in mind, we will learn how to get the logs from our applications. 1. The first step is to get the namespace of the environment where you deployed your application. 2. With the namespace ID, we will execute: ```bash kubectl get pods -n ``` In our case: ```bash kubectl get pods -n 823d7ade-927a-4b2a-9f1e-2b56b06a6ce0 ``` This will return the list of created pods in that namespace. Since every pod is a MuleSoft Application, it will return the list of applications: ![kubectl get pods for the Sandbox namespace listing four running RTF application pods](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-17.png) In this case, we have 4 applications running on top of our RTF. You will notice that it is the very same list and status that you get from the web console: ![Anypoint console application list showing the same four apps Running on the ocigen2rtf server](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-18.png) 3. Now that you have the list of applications, you can get the log files from a specific application using the next command: ```bash kubectl logs deploy/ -n -c app ``` In our case: ```bash kubectl logs deploy/myrtfapp -n 823d7ade-927a-4b2a-9f1e-2b56b06a6ce0 -c app ``` Please notice that the app-name is the name that you decided to use when you deployed the application. Which happens to be the name of the POD, but without the random ID created by Kubernetes. 4. After executing the command, you will get the runtime log, which is the normal log file from your MuleSoft runtime: ![Terminal showing the Mule runtime application log retrieved via kubectl logs](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-19.png) 5. But if you want to have the log file open while you make a request to your application in order to see what it’s writing, you can use the following command (which is like a tail -f): ```bash kubectl logs deploy/ -n -c app --follow ``` In our case: ```bash kubectl logs deploy/myrtfapp -n 823d7ade-927a-4b2a-9f1e-2b56b06a6ce0 -c app --follow ``` With that command, the log file will be in your terminal. If you make a request you will see what the log reports. 6. For example, remember that we mentioned that our sample app had a logger processor that will print the whole payload? Let’s make a call to it and see the result in our log file. Let’s first get the IP of the Pod (in this case I will make a call directly to the pod, just for demo purposes, but in real life, you will have to do it using your ingress controller). To get the IP of the pod, execute this: ```bash kubectl get pods -n -o wide ``` In our case: ```bash kubectl get pods -n823d7ade-927a-4b2a-9f1e-2b56b06a6ce0 -o wide ``` And the result is: ![kubectl get pods -o wide output revealing each pod's IP address and node](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-20.png) The IP is the range of the CIDR block or the pods' network that we explained in our previous [article](https://www.prostdev.com/post/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1). Our application is being served at IP: 10.244.15.66 and port 8081. And the resource path: /myrtf Let's send a POST with some payload to it: ```bash curl -X POST http://10.244.15.66:8081/myrtf -d '{"message":"Hello RTF"}' ``` In our log file: ![Followed log showing the Logger printing the Hello RTF payload after the curl POST request](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-21.png) That is from the application perspective, but what about the logs of the agent that performs the deployments into RTF? Let's imagine that something went wrong during the deployment, and you want to learn what happened. You can use this command to get the logs of that agent: ```bash kubectl -n rtf logs -f deployment/agent ``` You will notice that when you execute that command, a lot of log entries will be retrieved. You can either save them to a file and then do some research on it, or consolidate all your RTF log files (both the platform and applications) into Splunk, Elasticsearch, or a Syslog platform. We will elaborate a little bit more in the next section. Other useful commands, but now from the rtfctl perspective, are the ones that are related to how to retrieve: - HEAP dumps from your application. - THREAD dumps from your application. Sometimes it is useful to get into that level, and you can achieve it using the following commands: ```bash sudo ./rtfctl heapdump sudo ./rtfctl threaddump ``` For example: ```bash sudo ./rtfctl threaddump myrtfapp ``` The output is: ![Terminal output of rtfctl threaddump showing JVM thread stack traces for myrtfapp](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-22.png) (It is much larger, but we just want you to see that it is a normal thread dump from your app) ```bash sudo ./rtfctl heapdump myrtfapp dump.dmp ``` ![Terminal showing rtfctl heapdump writing the myrtfapp heap to dump.dmp](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-23.png) At this point, you may be wondering if it would be easier to consolidate most of the log files of our platform into a single place. That is completely possible with RTF and is very straightforward to configure. Just go to your RTF at the Anypoint Platform Web Console: ![ocigen2rtf Runtime Fabric page with the Log Forwarding tab highlighted](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-24.png) In that tab you will see this: ![Log Forwarding service dropdown listing Azure, Elasticsearch, Graylog, Splunk, CloudWatch and Syslog](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-25.png) As you can see, there are several options for this. In our case we will use Syslog: ![Syslog external configuration form (host, port, udp mode, rfc5424) and which logs to forward](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-26.png) The green square is for the connectivity information you need to input and is related to your SYSLOG server. And also for the format of the syslog, in our case, we will use the format rfc5424. The orange square is for you to decide which logs you want to send to the syslog server. You will notice that there are several options. Decide on what fits better for you. In our example, we will send everything to the PaperTrail syslog UDP server. Once you’ve configured it you can send a text message: ![Syslog forwarding enabled, configuration applied, with a Send a test message link](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-27.png) Once you click on it at the top right corner you will get: ![Green toast notification reading Message sent, visit Syslog to verify the connection](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-28.png) And after that, you can see the events being sent. Most of these events are from the platform itself: ![Papertrail Events view streaming incoming RTF platform log entries](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-29.png) But we can filter it with the name of our application; myrtfapp: ![Papertrail search bar filtering all systems by the term myrtfapp](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-30.png) Let’s send a request to our app and we will see the log file in Papertrail: ```bash curl -X POST http://10.244.15.66:8081/myrtf -d '{ "array": [ 1, 2, 3 ], "boolean": true, "color": "gold", "null": null, "number": 123, "object": { "a": "b", "c": "d" }, "string": "Hello World" }' ``` And at PaperTrail: ![Papertrail showing myrtfapp Logger output with the JSON payload forwarded via Syslog](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-31.png) Now let’s see how to get inside a pod. ## How to get inside a pod Sometimes you will have to get within a pod in order to do a particular activity. For example: to test connectivity to an external resource. Or just because you are curious, and you want to see what is inside of it. In order to get into the pod, execute this command: ```bash kubectl exec --stdin --tty -n -- /bin/bash ``` In this case, we are using the entire pod name. For example: ```bash kubectl exec --stdin --tty myrtfapp-79b9b4c488-4dm7v -n 823d7ade-927a-4b2a-9f1e-2b56b06a6ce0 -- /bin/bash ``` Once you execute this, the command prompt will be returned and you will be inside the pod: ![kubectl exec opening a bash shell inside the myrtfapp pod at /opt/mule](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-32.png) As you can see in the image, I am inside the pod. If I execute: ```bash ps -fea ``` I will see the java process executing my MuleSoft Runtime: ![ps -fea run inside the pod showing the Java process running the Mule runtime](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-33.png) You can move to the apps folder and see your application: ![ls of /opt/mule/apps inside the pod listing the myrtfapp deployment and its anchor file](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-2-mgmt-operations-34.png) You cannot write or modify anything. You do not need it, that is the beauty of this. If you need to change something, for example, JVM properties. You will do it from the console, and a new POD will be created with such a configuration. ## Conclusion and next steps With this article, we finalized this small series of two articles about RTF on top of Oracle Cloud Infrastructure. You’ve learned some very basic but useful commands. You now have a glimpse of how the underlying RTF infrastructure works, and how it is related to Kubernetes. We will continue writing about RTF. Our next article will be targeted at the Operation Center web console, which is also very useful for management purposes. We will also explain the upgrade process of an RTF. Stay tuned! --- ## How to create a README file for your GitHub profile, part 1: Intro to Git, GitHub, and README files Source: https://prostdev.com/post/how-to-create-a-readme-file-for-your-github-profile-part-1-intro-to-git-github-and-readme-files | Published: Sep 14, 2021 | Category: Guides *GitHub repository with the final `README.md` file can be found at the end of the post.* Having a [GitHub](https://github.com/) account in today’s world is a **must-have** for any software developer. In this series of posts, I’ll show you how to create a striking `README.md` file to attach to your GitHub profile. If you want to see an example of the final result, take a look at [my GitHub profile](https://github.com/alexandramartinez)! *Note: my GitHub profile's README file may have changed over time. To see the exact same code demonstrated throughout this series, check out* [this repository](https://github.com/ProstDev/alexandramartinez)*.* ## Git vs. GitHub First of all, if you’re not familiar with [Git](https://en.wikipedia.org/wiki/Git), it’s a technology that is widely used by almost every software project nowadays. The main feature of Git is the ability to keep your code in “repositories” and improve the collaboration between several developers who may be working on the same code at the same time. Think of it as a [Google Doc](https://workspace.google.com/products/docs/) for programming. Here are some sites to learn about Git if you’ve never used it before: [Git Documentation](https://git-scm.com/doc), [Learn Git Branching](https://learngitbranching.js.org/). Now, a common misconception is thinking that **GitHub** is the same as **Git**. While GitHub is based on Git (for the majority), they’re two different things. GitHub is what you see (front-end): the user interface, the buttons, the menus, the images, etc. While Git is the technology used underneath (back-end): the ability to create and manage repositories, commit changes, share your code, collaborate with others, etc. Other sites/platforms are also based on Git, for example, [BitBucket](https://bitbucket.org/product) or [GitLab](https://about.gitlab.com/). So why GitHub? In my experience, almost all developers I’ve met keep their repositories on this platform. A lot of social networking sites have integrations to add your GitHub profile to your list of links or have the ability to embed your repositories in a post (for example, [Polywork](https://www.polywork.com/devalexmartinez/highlights/897f5203-0527-4334-9f17-ce1591129cc4)). And in my personal opinion, I think it’s easier to use. The UI/UX is beautiful, it has a ton of features that they keep improving regularly, and it’s 100% free! ## About README.md files Whenever you create a new repository, it is a best practice to create a `README.md` file at the root of your project. This file needs to explain the basics of your project: what it is, what it does, and how to install/use it. It’s basically a “welcome” page for new developers that want to use your code. Here’s a very nice example of a `README.md` file for the [Official Jenkins Docker image repository](https://github.com/jenkinsci/docker#readme). GitHub has the feature of attaching a README file to your profile to introduce yourself and/or give a summary of your GitHub page for new developers visiting your profile. You are free to personalize this file with whatever information you want to show to the public. Some people use images or GIFs, embed videos, add links, or give [CTAs](https://en.wikipedia.org/wiki/Call_to_action_(marketing)) here. Think of it as a tiny website where you get to show off your work, your hobbies and let people know who you are and what you do. ## Inspiration When I started creating my README, I was inspired by [this GitHub profile](https://github.com/victoria-lo). Then I noticed she has a repository with a compilation of some other [awesome GitHub profiles](https://github.com/victoria-lo/awesome-github-profiles), which inspired me even more! I didn’t know about a lot of the features that could be added to README files; I discovered them after I saw all these great profiles. ## Before reading the rest of the series If you’re new to Markdown, they may seem a bit complicated or overwhelming. But not to worry! In this series of posts, I’ll guide you through all the steps I did to get my README file up and running, even if you’re new to all this. If you’re new to Git, I do recommend you first get a sense of the basic terminology such as repository, blob, commit, local/remote, branch, etc. I found [this article](https://www.javatpoint.com/git-terminology) on Google. It may help you to have a rough idea of the theory. You don’t really need to know Git in order to follow the series. You can just follow the steps and still get the final outcome. However, it may seem a bit confusing if you don’t understand the concepts behind it. ## Recap - GitHub is based on Git, but they’re not the same thing. Git is the back-end part, and GitHub is the front-end part. - `README.md` files are used to explain the basics of a repository: what it does, how to install, and how to use it. - In GitHub, you can attach a README to your profile to welcome your readers. Don’t miss any articles! Remember to subscribe at the bottom of the page to receive email notifications as soon as new content is published. ✨ -Alex ## GitHub repository [ProstDev - alexandramartinez](https://github.com/ProstDev/alexandramartinez) --- ## Notification center for new versions of connectors/modules in Anypoint Studio 7.10.0 Source: https://prostdev.com/post/notification-center-for-new-versions-of-connectors-modules | Published: Aug 31, 2021 | Category: News Let’s do a quick and short blog post! This post is about the new Anypoint Studio 7.10.0 version which has been released this past July 29th, 2021. In this link you will find all the details of this new release: [https://docs.mulesoft.com/release-notes/studio/anypoint-studio-7.10-with-4.3-runtime-release-notes](https://docs.mulesoft.com/release-notes/studio/anypoint-studio-7.10-with-4.3-runtime-release-notes) ## New features summary Here is a summary of the new changes introduced in this new version, you will find the same summary within your Anypoint Studio installation at *Help > What’s new in Anypoint Studio?* **Introducing Metadata Assistant - New** MuleSoft is making it easier to work with metadata across flows and sub-flows. You can now extract and propagate metadata across sub-flows with a few clicks through a new wizard, where previously this was a laborious manual process. **Notification center for new versions of connectors - New** Get recommendations when new connector versions are available so you are on the latest innovation. **Refactor rename for flows - New** Now it will be easier to rename flow references. Changes on a flow name will be propagated automatically across flow references. **Themes - Improvement** Theme selection got smarter than ever. For new workspaces, it will automatically match your Operating System appearance preference selecting the Light or Dark theme accordingly. ## Notification Center for new versions of connectors Let’s talk about the new Notification Center for new versions of connectors which is a very cool feature. Have you ever been asked to update the dependencies of your MuleSoft project? Have you ever noticed that the modules or connectors of your projects are older versions? Have you had problems with not updating your code to the latest versions? The new notification center solves most of these problems. To use it you simply have to do the following. Let’s say you are developing a MuleSoft application where you need to use the database and file connectors. You will see in the `pom.xml` file those two dependencies. Similar to this: ![pom.xml dependencies showing the Database connector 1.10.0 and File connector 1.3.0](../../assets/blog/notification-center-for-new-versions-of-connectors-modules-1.png) As you can see, your project is using version 1.10.0 for the database connector and 1.3.0 for the file connector. Are those the latest version of the connectors? Could this cause an issue in your application? Do you want to verify and update them to the newest versions? Let’s check it out! Right-click on your project > Manage Dependencies > Manage Modules: ![Studio right-click project menu with Manage Dependencies then Manage Modules highlighted](../../assets/blog/notification-center-for-new-versions-of-connectors-modules-2.png) Automatically, Anypoint Studio will connect with Anypoint Exchange and will fetch all the alternative versions for your connectors: ![Modules properties listing connectors with blue update dots while fetching alternative versions](../../assets/blog/notification-center-for-new-versions-of-connectors-modules-3.png) The blue dots indicate that there are available new versions for those connectors. Then, you need to click on each connector, click on the Update Version button, and finally on Apply and Close button. ![Modules panel with "New version available" for Database and the Update version and Apply buttons](../../assets/blog/notification-center-for-new-versions-of-connectors-modules-4.png) You can confirm that the `pom.xml` file is automatically updated with the new dependencies versions. ![pom.xml reopened in Studio showing the connector dependencies updated to their new versions](../../assets/blog/notification-center-for-new-versions-of-connectors-modules-5.png) Finally, you can confirm that the versions of those dependencies are the latest for each connector at the Anypoint Connector Release Notes page: [https://docs.mulesoft.com/release-notes/connector/anypoint-connector-release-notes](https://docs.mulesoft.com/release-notes/connector/anypoint-connector-release-notes) ![Latest version of the Database Connector](../../assets/blog/notification-center-for-new-versions-of-connectors-modules-6.png) ![Latest version of the File Connector](../../assets/blog/notification-center-for-new-versions-of-connectors-modules-7.png) Easy, right? Enjoy and happy coding! --- ## Recommendation: Logitech MX Ergo Advanced (Plus) Wireless Trackball Mouse Source: https://prostdev.com/post/recommendation-logitech-mx-ergo-advanced-plus-wireless-trackball-mouse | Published: Aug 10, 2021 | Category: Opinion *Spanish version / versión en Español:* [Recomendación: Logitech MX Ergo Advanced (Plus) Wireless Trackball Mouse](https://www.alexandramartinez.world/post/recomendacion-logitech-mx-ergo-advanced-plus-wireless-trackball-mouse). I bought this mouse less than 24 hours ago, and I’m already completely in love with it. I’m honestly even a little mad at Logitech for not sending me an email with the marketing for this product as soon as it came out, *LOL*! ## My previous mouse Before starting with why I love my new purchase, let me give you a bit of background on my previous mouse. It was a [Logitech M570](https://amzn.to/3sTtftz) (also a wireless trackball mouse). I was pretty comfortable with it, but I always thought it needed more buttons. However, the newer versions still had pretty much the same number of buttons, so I never thought of upgrading it...until now. If you don’t know what a trackball mouse looks like, here it is: ![Logitech M570 Wireless Trackball Mouse.](../../assets/blog/recommendation-logitech-mx-ergo-advanced-plus-wireless-trackball-mouse-1.png) Why do I love these? - I don’t need a mouse pad. - I don’t need to move my hand; I just move my thumb. - Less desktop space is needed. - It can be used on almost any surface. I say it needs more buttons because you basically have (1) the regular left-click, (2) right-click, and (3) scroll wheel button; *plus,* you can find the two buttons between the trackball and the left-click button. ![Logitech M570 Wireless Trackball Mouse’s 5 programmable buttons.](../../assets/blog/recommendation-logitech-mx-ergo-advanced-plus-wireless-trackball-mouse-2.png) I wanted to program several buttons to do some of my more common tasks - *I’ll mention them later in the post.* Unfortunately, I was only able to use buttons 3, 4, and 5 for my custom functions. There are a lot of mice with more than 5 programmable buttons, but I wanted it to be specifically a trackball mouse. This brings me to my new toy! ## Logitech MX Ergo Advanced (Plus) Wireless Trackball Mouse ![Logitech MX Ergo Advanced Wireless Trackball Mouse’s 6 programmable buttons.](../../assets/blog/recommendation-logitech-mx-ergo-advanced-plus-wireless-trackball-mouse-3.png) [This mouse](https://amzn.to/3eVb5z6) has **9** buttons in total! The picture may be misleading because it’s only pointing to the *6 programmable* buttons, but it’s not counting the left-click, right-click, and the small button in the middle that says 1 and 2. These 3 buttons will always maintain their specific functions and cannot be changed. I’m sure you’re curious as to what I have programmed these 6 buttons to do. Here it goes: - Left-top button: Delete. - Left-bottom button: Enter/Return. - Scroll wheel left button: Move to the desktop to the left (for Mac computers). - Scroll wheel middle button: Screen capture. - Scroll wheel right button: Move to the desktop to the right (for Mac computers). - Button over the trackball: Maximize/minimize window. Before I got an additional keyboard, I didn’t have to use numbers 3 and 5 that often. I could just use my Macbook’s trackpad to swipe left/right and change my desktop. However, after acquiring the keyboard, this became a bit more work. I had to either reach out to the laptop’s trackpad with my left hand or use the keyboard shortcut with my right hand (which I had to switch between the keyboard and the mouse). Neither option was good enough for me. ![Reference picture of my current desktop setting.](../../assets/blog/recommendation-logitech-mx-ergo-advanced-plus-wireless-trackball-mouse-4.jpg) With my previous mouse, I had to start using the two additional buttons (4 and 5) to move to the left/right desktop. So, my previously programmed keystrokes (Delete and Enter/Return) were no longer in the picture. This, again, was not good enough for me. Now I get to have all my 4 favorite tasks PLUS 2 more buttons that make my life easier! ## Is that all? Of course not! This mouse is a MASTERPIECE! But the buttons were my biggest need. Also, the M570 requires one AA battery, but the MX Ergo can be recharged with a USB. By the way, remember the middle button with the 1 and 2 numbers underneath it that you could not program? ![Close-up of the mouse's 1 and 2 button for switching between two paired computers](../../assets/blog/recommendation-logitech-mx-ergo-advanced-plus-wireless-trackball-mouse-5.png) Well, that button is for alternating between 2 different computers with the same mouse! This is incredibly useful if you’re using several systems at the same time. It can be annoying to have to use different hardware for the same purpose. I tested this functionality, and it doesn’t even have a lag to it. Once you push the button, the other computer can be used immediately — no need to wait for even 1 second. It still has more great specs, but they weren’t *that* important for my day-to-day. You can read more about them [here](https://www.logitech.com/en-ca/products/mice/mx-ergo-wireless-trackball-mouse.html). Finally, I leave you this quick 30-second video for a preview! --- ## MuleSoft Runtime Fabric Deployed on Oracle Cloud Infrastructure (OCI) - Part 1 Source: https://prostdev.com/post/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1 | Published: Aug 3, 2021 | Category: Tutorials MuleSoft is so flexible and modern in such a way, that for deployment models where the customer needs to have control over the infrastructure, and CloudHub is not an option (maybe regulations concerns) it offers on premise deployments such as: - MuleSoft Standalone - MuleSoft Runtime Fabric - MuleSoft Private Cloud Edition The first two are a hybrid model, where the control plane remains in the cloud (MuleSoft) and the runtime plane is deployed on premise or in a computer on top of a cloud provider (Google, AWS, Oracle, Microsoft, Digital Ocean, etc). For Standalone, it’s the common MuleSoft runtime running directly at the OS level, which supports clustering, grouping of runtimes, etc. There is nothing extra on this type of deployment, it is basically to download the software, unzip it and run it. I am being practical right here, but in essence that is what you have to do in order to have it running on your own hardware. In this case the responsibility to maintain, patch, upgrade, operate the runtime is at the customer end; the control plane remains a MuleSoft responsibility, but ultimately where the workloads are running (runtime plane) is all at customer’s. Private Cloud Edition, is the option where you have both the Runtime Plane and the Control Plane at your end; you are in charge of maintaining everything. You install it, you patch it, you upgrade it, you monitor it, etc. But you have most of the capabilities at your disposal, similar to CloudHub; there are some differences, but in general it is very compatible with what you get in CloudHub. You can decide how much compute to assign to a specific MuleSoft application, or the amount of replicas, in a very similar CloudHub fashion. Then you have MuleSoft Runtime Fabric (RTF), which is the option we want to elaborate through this article. Runtime Fabric, as previously mentioned, is a hybrid deployment model. You still have the Control Plane in the cloud but the workloads (runtime plane) are running at the customer end. The main thing here with RTF is that the infrastructure that is behind the scenes running the fabric is a Kubernetes-based implementation (Kubernetes [https://kubernetes.io](https://kubernetes.io/)). But before thinking that this may be very complicated, continue reading this article where you will realize that some of the complexity that is coming to your mind right now, is going to get reduced. Let us start with the installation process, which is always time consuming because of the pre-requisites that we need to fulfill every time we install something on premise. With RTF is no exception, you need to have all the pre-requisites in place in order to start installing. But the good news is that it is very well documented. You will identify two ways of installing RTF: 1. **On top of an existent Kubernetes cluster that you already have**. For example, on top of AWS, GKE, AKE. This option is the best one if you want to leverage from what you already have in terms of Kubernetes. The installation documentation is here [https://docs.MuleSoft.com/runtime-fabric/latest/install-self-managed](https://docs.mulesoft.com/runtime-fabric/latest/install-self-managed). ![Runtime Fabric on Self-Managed Kubernetes.](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-1.png) 2. **Manual Installation**. This option is where you want to install RTF on top of your hardware, either if it is at your datacenter or on the compute at your cloud provider. This article is based on the manual installation and can be useful for you in case you do not have an existent Kubernetes cluster and want to test RTF. This manual installation is also useful to understand the dynamics behind RTF and is a very good option for customers who don't have a current Kubernetes cluster, but want to have a flexible/dynamic deployment model. The official documentation is this one: [https://docs.MuleSoft.com/runtime-fabric/1.9/install-manual](https://docs.mulesoft.com/runtime-fabric/1.9/install-manual). ## Infrastructure Summary I am going to use Oracle Cloud Infrastructure to explain the pre-requisites and part of the installation process. Why Oracle Cloud Infrastructure? I think it is a very good option for this demo and very simple to use. What I am using from Oracle Cloud Infrastructure (OCI) is the following: - Compute Instances - Storage - Load Balancers - Virtual Networks and Subnets - Security access rules (firewall) In this case I will use a three nodes RTF cluster, where: - One of the nodes is acting as the Controller - Two nodes are going to represent the Workers ## Compute & Storage ![OCI Compute Instances list: controlRTF, rtfworker2 and rtfworker1 all running VM.Standard.E2.2](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-2.png) Operative system can be any of the following: - Red Hat (RHEL) v7.4, v7.5, v7.6, v7.7, v7.8, v7.9, v8.0, v8.1, v8.2, v8.3 - CentOS v7.4, v7.5, v7.6, v7.7, v7.8, v7.9, v8.0, v8.1, v8.2, v8.3 - Ubuntu v18.04 (using Runtime Fabric Appliance version 1.1.1625094374-7058b20 or later) I am using CentOS. Within those instances I need to take care of this: - Disable swap memory. - Ensure that chrony (system clocks) status is in sync. I have provisioned storage to attach it to those compute instances: ![OCI Block Volumes list with worker, controller Docker and controller etcd volumes](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-3.png) The storage is distributed in the following way: ![Table mapping each compute node to its storage volume, purpose and size](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-4.png) The storage is just presented to the compute instance but is not formatted. The installation process will take it from there, format and attach it in the way RTF needs it. The storage for the controller, is for the following purpose: - A minimum of 60 GiB dedicated disk with at least 3000 provisioned IOPS to run the etcd ([https://etcd.io/](https://etcd.io/)) distributed database. This translates to a minimum of 12 Megabytes per second (MBps) disk read/write performance. - A minimum of 250 GiB dedicated disk with 1000 provisioned IOPS for Docker overlay and other internal services. This translates to a minimum of 4 MBps disk read/write performance. For the workers: - A minimum of 250 GiB dedicated disk with at least 1000 provisioned IOPS for Docker overlay and other internal services. This translates to a minimum of 4 MBps disk read/write performance. - Having 250 GiB ensures there is enough space to run applications, cache docker images, provide temporary storage for running applications, and provide log storage. In the local disk: - A minimum of 80 GiB dedicated disk for the operating system. - A minimum of 20 GiB for /tmp directory. - A minimum of 8 GiB for /opt/anypoint/runtimefabric directory. - A minimum of 1 GiB for /var/log/ directory. ## Networking Regarding networking, I have this VCN: ![OCI Virtual Cloud Networks list showing the rtfvcn network with CIDR block 11.0.0.0/16](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-5.png) I have a large CIDR Block 11.0.0.0/16 but you can use a smaller one. Just keep in mind the following: > The pod CIDR block must not overlap with IP addresses that pods or servers use to communicate. If services within the cluster or services that you installed on nodes need to communicate with an IP range that overlaps the pod or service CIDR block, a conflict can occur. If a CIDR block is in use, but pods and services do not use those IP addresses to communicate, there is no conflict. If you deploy more than one cluster, each cluster can reuse the same IP range, because those addresses exist within the cluster nodes, and cluster-to-cluster communications is relayed on the external interfaces. *Taken from the official documentation:* [https://docs.MuleSoft.com/runtime-fabric/1.9/install-prereqs](https://docs.MuleSoft.com/runtime-fabric/1.9/install-prereqs)*.* Also, within that VCN, I have a couple of subnets: ![OCI Subnets list with a private 11.0.1.0/24 subnet and a public 11.0.0.0/24 subnet](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-6.png) One is private and one public. For our scenario, in order to have access to the compute instances, all VMs are part of the Public subnet. Here is the summary: ![Table of the three nodes' primary VNIC details: private IPs, subnet and hostnames](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-7.png) My computer instances have the following characteristics: ![Compute instance specs: VM.Standard.E2.2, 2 GHz AMD EPYC 7551, 2 CPUs, 16 GB RAM](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-8.png) All compute instances are interconnected and need to be visible among each other though several ports and protocols. Those ports and protocols, in my case in Oracle Cloud Infrastructure, are reflected through a Security List. Which would be the equivalent to the configuration you have to put in place in your firewall. The summary is described in the following table: ![Security list table of required ingress rules: source, protocol, port range and description](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-9.png) As mentioned, that was all defined in my Default Security List: ![Default Security List for rtfvcn page showing 21 ingress rules](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-10.png) Another relevant thing is that both controllers and workers need to have access to the Control Plane, and therefore to the Internet to reach certain hostnames. Keep in mind that we are working with a hybrid model and the runtime plane needs access to the control plane for several purposes: - Monitoring - Access to the image repository - Access to Runtime Manager - Access to API Manager - Etc Within our infrastructure (OCI) we have an Internet Gateway that gives us access to the internet: ![rtfvcn details page with its Internet Gateway listed under the VCN](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-11.png) In my case with that simple Internet Gateway I get access to the internet from within my compute instances. In your case you may control this, and just enable access to the specific list of locations. That specific list is the following: ![Table of required outbound hostnames on port 443 for US and EU control planes](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-12.png) ![Continued hostnames table covering the deprecated port 5044 Lumberjack endpoints](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-13.png) ![Hostnames table for Anypoint asset, Kubernetes charts and Docker repositories on port 443](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-14.png) ![Hostnames table for Anypoint Exchange and Runtime Fabric Docker repository endpoints](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-15.png) ![Hostnames table for Runtime Fabric Docker image delivery and Configuration Resolver endpoints](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-16.png) Depending on if you are in the US or EU control plane, you will need to reach some of those locations. All this information is here: [https://docs.MuleSoft.com/runtime-fabric/1.9/install-prereqs](https://docs.mulesoft.com/runtime-fabric/1.9/install-prereqs) try to follow the official documentation to avoid any error. ## Installation Process Again, this article does not replace the official documentation. We are just trying to give you a glimpse of the whole process and share with you that the whole installation process is very accessible. And you can execute within hours, trust us. The first step is to declare in the Anypoint Platform web console that you want to create an RTF. That is a very simple step, just go here: ![Runtime Manager Runtime Fabrics page with the Create Runtime Fabric button highlighted](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-17.png) Select your environment, and then click on the Create Runtime Fabric button. A similar screen as the following will appear: ![Create Runtime Fabric dialog with install targets and the VMs or bare metal option highlighted](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-18.png) Give a name and select the VMs or bare metal radio button. Once you do that, this screen will appear: ![Runtime Fabric setup steps: install overview video, Download files button and activation data](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-19.png) Look at step 2 and step 3. Step 2 will download the required installation files, and Step 3 is your activation data for your installation. Please copy that and put it in a safe place. Once you download the file (a similar name to this rtf-install-scripts-20210709-b48dec6.zip), upload it to the three servers (controller, worker1, worker2). Now unzip it: ```bash mkdir -p ./rtf-install-scripts && unzip rtf-install-scripts.zip -d ./rtf-install-scripts ``` If this is for a PoC or demo purposes you can execute everything with root, but if this is for business usage then create a user and make it sudoer. Now that you’ve uploaded the installation scripts to every server, you are just a few steps away to install the Fabric. The installation is all based on scripts. The controller is the leader of all the work we are about to explain. I always like to understand first what I am going to execute, and then get into the box and do the work. Let’s do it. - We need to describe where we want to execute things and which role the different computes are going to take. And we will do it through setting a group of variables. Those variables will describe your intentions on deploying RTF. Some of those variables are optional. - Once those variables are set, one of the scripts that we have downloaded and then uploaded to our compute instances, is going to generate the config and the snippets to execute on every compute (controller, worker), depending on what you have defined on step 1. - The first snippet is executed on the controller. - Workers snippets can be executed in parallel, and will report themselves with the controller. Those are the high-level steps and a good way to understand what you are going to execute. Now let’s take a look to the variables we need to set: ![Table of RTF install variables with descriptions and the example values used here](../../assets/blog/mulesoft-runtime-fabric-deployed-on-oracle-cloud-infrastructure-oci-part-1-20.png) From all the information that is described in the previous table, up to this point you have all the information needed, but the `RTF_MULE_LICENSE`. The `RTF_MULE_LICENSE` is a file with extension `.lic` that you should have in case you have a valid and active subscription with MuleSoft. Once you have the file at your disposal, execute this: ```bash base64 -w0 license.lic ``` The output from that command is what you must set as the `RTF_MULE_LICENSE` variable value. The `RTF_ETCD_DEVICE` and `RTF_DOCKER_DEVICE` need to point to the volumes (storage) that we have provisioned in previous steps. ## Installation Execution Now it is time to execute the installation process. At this point we have some understanding of what we are about to execute and it is time to put all the pieces in place. 1. Go to the folder where you have unzipped the installer $HOME/rtf-install-scripts/ 2. Set the environment variables and generate the configuration ```bash RTF_CONTROLLER_IPS='11.0.0.5' \ RTF_WORKER_IPS='11.0.0.3 11.0.0.4' \ RTF_DOCKER_DEVICE='/dev/xvdc' \ RTF_ETCD_DEVICE='/dev/xvdb' \ RTF_ACTIVATION_DATA='f653432342-54433$4322' \ RTF_MULE_LICENSE='adfafasfasfasfasfasfdasdsff==' \ ./generate-configs.sh ``` 3. The execution of the previous step is going to generate and output like this: ```bash ========================================================= Runtime Fabric configuration generator ========================================================= Cluster topology: 1 controllers, 2 workers Instructions: 1. Create /opt/anypoint/runtimefabric directory and ensure it is writable from your ssh login 2. Copy each snippet below and execute on the appropriate machine 3. For each node, copy the init.sh script to the installation directory eg scp scripts/init.sh @node-ip:/opt/anypoint/runtimefabric 4. Execute sudo init.sh on each node. This should be done first on the leader (first controller) node, then concurrently on the other nodes 5. The nodes will join to the IP address given for the first controller and form your Runtime Fabric cluster. This process can take 10-25 minutes Note: You can monitor the progress of the installation on any of the nodes with tail -f /var/log/rtf-init.log 11.0.0.5: ========================================================= mkdir -p /opt/anypoint/runtimefabric && cat > /opt/anypoint/runtimefabric/env < /opt/anypoint/runtimefabric/env < /opt/anypoint/runtimefabric/env <>. Mule does not understand “\”. Make sure to replace “\” with “/”. So the final address you should be using for the “File Path” in the above configuration should be something like C:/Users/Documents/Test Files/<<File Name>>. Second Configuration You need to add the MIME Type. Since in this case we are dealing with an excel file, we will be keeping the MIME Type as application/xlsx. ![Read component MIME Type tab set to application/xlsx for the Excel file](../../assets/blog/dynamic-variable-in-mulesoft-6.png) **3.** Once this is done, we need to specify which sheet we are trying to read. A simple DataWeave script can help! ![Set Payload component with a DataWeave script reading payload."Sheet1"](../../assets/blog/dynamic-variable-in-mulesoft-7.png) By writing payload.“Sheet1”, we are instructing it to read the data which is there in the Sheet1. Now that you know how to read an excel file, we are good to go ahead. Let's explore now! ## Let's head back to our studio **1.** Once the file is read, we have logged the payload to see what is being fed to the mule flow (Refer to the prerequisites to learn about reading a file). ![Logged payload showing each record's Name, Dept, and Company as JSON objects](../../assets/blog/dynamic-variable-in-mulesoft-8.png) **2.** We can see that the payload is being read like above. So, to intercept the entire payload as individual chunks, we have used a for-each which divides the payload uniformly. **3.** On the onset of the flow, we have captured the queryString. A query string contains the total string which we have passed in the URL. ![GET request URL with the Company=ImaginaryA portion labeled as the query string](../../assets/blog/dynamic-variable-in-mulesoft-9.png) **4.** Once we reach inside the payload, we split the query string by the separator “=”. ![Set Variable named segBy with value vars.queryString splitBy "="](../../assets/blog/dynamic-variable-in-mulesoft-10.png) We can see **Company** as **segBy[0]** and **ImaginaryA** as **segBy[1]** (please check the above point for getting the references). **5.** Once that is done, we will write the DataWeave script to implement the dynamic variable concept. ![set-variable XML using a dynamicVar to build a variable name from segBy[0] at runtime](../../assets/blog/dynamic-variable-in-mulesoft-11.png) The name of the variable is kept in accordance with the feature, which we have mentioned in the query string, by which we want to segregate our records. In this case we want that feature to be Company. Here in the 3rd line, we have written dynamicVar = payload[vars.segBy[0] as String] as String - We have created a local variable named dynamicVar which will create a placeholder as per the number of companies in the payload. - This will be dynamic in nature, which means that the number of variables to be created will be decided during the runtime. The script that follows it simply checks if the company named variable is available or not. - If available, it pushes the records into that already available variable. - If not available, it creates the company named variable and then inserts the records. **6.** Once that is done, we will send an email to confirm whether our code has worked or not. In order to summon the variable, we will use the following script. ![Email Body config reading the dynamic variable via vars[vars.segBy[1]] as plain text](../../assets/blog/dynamic-variable-in-mulesoft-12.png) In our case since we have 4 companies in our payload, 4 variables will be created. Now, if you remember the query string, we want the records belonging to the company **ImaginaryA.** The command **segBy[1]** will return the value **ImaginaryA** (please check step 4). Hence, by the command we are trying to access the variable created for **ImaginaryA**. **7.** Once the process completes, as per the POC we should receive a mail which should look something like this: ![Received email listing only the ImaginaryA employee records in CSV format](../../assets/blog/dynamic-variable-in-mulesoft-13.png) So, finally we have a solution which can address our issue. ## GitHub repository [ProstDev GitHub - Dynamic Variable Mule](https://github.com/ProstDev/dynamic-variable-mule) --- ## My top 5 DataWeave tips to make your life easier Source: https://prostdev.com/post/my-top-5-dataweave-tips-to-make-your-life-easier | Published: Jun 1, 2021 | Category: Guides It took me some time to find and fully understand how to use these “hacks” *(they’re not really hacks, but they’ve made my life so much easier, so I’ll call them hacks)*. Trust me; they’re truly worth it, especially if you’re just starting to learn DataWeave. - Using the DataWeave Playground - Using the “dw” output - Using the “log” function - Using the “do” statement - Assigning types to variables and functions ## 1. Using the DataWeave Playground This is my absolute favorite tool, and I really hope more functionality gets added into it (*cough, cough, a DataWeave Notebook, please?).* I use this tool ALL the time. This is basically a tool that lets you experiment with DataWeave without doing it from Anypoint Studio. You can find the docker image to install in your local machine in this post (no previous Docker experience needed) - [How to run locally the DataWeave Playground Docker Image](https://www.prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image)*.* But wait! Before you do that, the awesome DataWeave team already created an online site that you can access without having to run Docker. Check it out here: [https://developer.mulesoft.com/learn/dataweave](https://developer.mulesoft.com/learn/dataweave) ![DataWeave Playground web interface with input explorer, script editor, and JSON output panels](../../assets/blog/my-top-5-dataweave-tips-to-make-your-life-easier-1.png) ## 2. Using the “dw” output When you’re working with certain data types, like JSON, you don’t have the same visibility as some of the values’ types. For example, JSON converts the Dates into Strings. You can’t be absolutely sure about the exact type of output unless you follow the code. Here’s an example: ```dataweave %dw 2.0 output application/json --- "2000-01-01" as Date // "2000-01-01" ``` (I’m adding in the comments the output that I’m getting) Simply by looking at “2000-01-01”, you’re not sure if it’s a String or a Date (without looking at the code, of course). If I remove the “as Date” coercion, the output remains the same. ```dataweave %dw 2.0 output application/json --- "2000-01-01" // "2000-01-01" ``` Now try changing the “json” output for a “dw” output and see the magic. Without coercion: ```dataweave %dw 2.0 output application/dw --- "2000-01-01" // "2000-01-01" ``` With coercion: ```dataweave %dw 2.0 output application/dw --- "2000-01-01" as Date // |2000-01-01| ``` Ah! You can see the difference now! And that’s not all. By using this output, you can also see a function’s definition. Here’s an example of the “[plusplus](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-plusplus)” function: ```dataweave %dw 2.0 output application/dw --- ++ // (arg0:Array | String | Object | Date | LocalTime | Date | Time | Date | TimeZone | LocalDateTime | TimeZone | LocalTime | TimeZone, arg1:Array | String | Object | LocalTime | Date | Time | Date | TimeZone | Date | TimeZone | LocalDateTime | TimeZone | LocalTime) -> ??? ``` Pretty cool, right? ## 3. Using the “log” function Isn’t it frustrating when a small part of your script is not working, and you’re not sure why? You can’t actually debug it like you’d debug a Mule Flow. But guess what? You *can* output expressions into the console. Let’s say that I have the following script: ```dataweave %dw 2.0 output application/json --- (random() * 100) > 74 ``` I’m either getting true or false (Boolean, not String), depending on whether the randomly generated number is more than 74 or not, but I don’t know what the actual random number is. Well, I can use the “[log](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-log)” function to see which number it is. And the best part? Adding this into my script doesn’t affect anything I do. I don’t have to add a new variable only for the log. I can just surround the value that I want to log into the console in parentheses. ```dataweave %dw 2.0 output application/json --- log(random() * 100) > 74 ``` This is what it looks like from the DataWeave Playground: ![DataWeave Playground log viewer showing the random number 61.91 logged below the script](../../assets/blog/my-top-5-dataweave-tips-to-make-your-life-easier-2.png) You can experiment a bit with it to see what you can and can’t do with this function. For example, you can’t do this: ```dataweave %dw 2.0 output application/json --- log(if(true)) "ok" else "not ok" // Invalid input '"', expected Function Call ``` ## 4. Using the “do” statement You can create a local context by using the “[do](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-flow-control#control_flow_do)” statement. This includes variables, functions, and the script. This way, you can have variables inside your functions that are not accessible outside of the function. I use this a lot to not end up with a lot of global variables that can’t be re-used for other purposes. Consider this script: ```dataweave %dw 2.0 output application/json var people = [ { name: "Alex", age: 90 }, { name: "Elsa", age: 24 } ] --- (people map do { var newName = if ($.name == "Elsa") "Anna" else $.name --- { name: newName, age: $.age } }) ``` ![DataWeave Playground running the given script and showing the transformed output.](../../assets/blog/my-top-5-dataweave-tips-to-make-your-life-easier-3.png) If we wanted to re-use the "newName" variable from outside of this context (set by "do"), we would receive an error because the variable only exists inside the context where it was created. ```dataweave (people map do { var newName = if ($.name == "Elsa") "Anna" else $.name --- { name: newName, age: $.age } }) + newName // Unable to resolve reference of: `newName`. ``` In this example, I created a context inside the “map” function, but notice how I need to re-declare the object for the map. This is because there’s a new script section inside the “map” (right under the three dashes (---). A regular map would look like this: ```dataweave people map { name: $.name, age: $.age } ``` I don’t need to add additional curly brackets ({ }) in the regular map. But because I opened a new context, I need to treat the new space just like a regular script. You *can* continue creating more contexts inside contexts, but please **don’t** do this. :D I’m not sure if this will create performance issues later, but it doesn’t look good, and you may end up with “[Spaghetti code](https://en.wikipedia.org/wiki/Spaghetti_code).” It took me some time and practice to get used to this statement. The best way to master it is to use it frequently over several days (not only a lot of times in one day but for many days). ## 5. Assigning types to variables and functions When I first started developing in DataWeave, I didn’t assign any [type](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-type-system) to the functions or variables I was creating because it’s not really needed. Here’s an example: ```dataweave %dw 2.0 output application/json var str: String = "Hello" fun concat(str1: String, str2: String): String = str1 ++ str2 --- concat(str, " World!") // "Hello World!" ``` As you can see, my variable “str”, the two parameters of the “concat” function, and even the return type of the function itself, are all of type “String.” This has helped me in different ways: **1.** I don’t get confused when I have a big script and forget which kind of data a variable was. I can easily look at the data type I assigned to it. **2.** My mistakes and potential bugs are reduced because I know exactly what kind of data I *could* get back. For example, when you use a [selector](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-selectors) like the Index selector, it *is* possible that the data you get is a Null, instead of the data you were expecting. ```dataweave %dw 2.0 output application/json fun getValue(array: Array, index: Number): Number | Null = array[index] // Return type can be a Number (because it's an array of numbers), but it can also be a Null if the index wasn't found. Hence the “Number | Null” union. --- getValue([1, 2, 3], 5) ``` **3.** I get early errors in my script before deploying and running my Mule App, saving me time. **4.** It’s easier for the next developer to understand what the code is doing by looking at the data types. **5.** Probably the best thing of all is that I learned how to use [Function Overloading](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-functions#overloading-functions) when I’m expecting different functionality depending on the data type. ```dataweave %dw 2.0 output application/json fun concat(value1: String, value2: String): String = value1 ++ value2 fun concat(value1: Number, value2: Number): String = (value1 as String) ++ (value2 as String) fun concat(value1: Any, value2: Any): String = "Other types" --- { strings: concat("Hello", " World!"), // Hello World!" numbers: concat(20, 20), // "2020" other: concat({},[]) // "Other types" } ``` ## Recap - You can use the DataWeave Playground Docker Image locally by following [this post](https://www.prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image), or directly using [the public site](https://developer.mulesoft.com/learn/dataweave) (if available). - Use the output “application/dw” to see the internal data structures or read the definition of a function. - You can “debug” your script by using the “[log](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-log)” function and output data into the console. - Create local variables and functions using the “[do](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-flow-control#control_flow_do)” statement. - Assign [types](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-type-system) to variables and functions to avoid confusion later, reduce mistakes, catch errors early, help the next developer understand your code, and use [Function Overloading](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-functions#overloading-functions) when needed. I hope these tips help you in your DataWeave journey. *Prost!* -Alex --- ## Easiest way to integrate automatic code review of MuleSoft apps using SonarQube and Docker Container Source: https://prostdev.com/post/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container | Published: May 25, 2021 | Category: Tutorials ## Using SonarQube and Docker The quickest way to have an installation of SonarQube up and running is using a Docker container. SonarQube is a platform for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, [code smells](https://en.wikipedia.org/wiki/Code_smell) and security vulnerabilities. This definition is not by me, it is from Wikipedia ([https://en.wikipedia.org/wiki/SonarQube](https://en.wikipedia.org/wiki/SonarQube)). If you want to know more about SonarQube and all its capabilities, you can visit their website: [https://www.sonarqube.org/](https://www.sonarqube.org/developer-edition/?gads_campaign=South-America-SonarQube&gads_ad_group=SonarQube&gads_keyword=sonarqube&gclid=Cj0KCQjwvYSEBhDjARIsAJMn0lga8FBAQxpXuiDOFvD4zK28tuA1_8TlLkBTOtDcFaWIahnsU2DtyKUaAgbGEALw_wcB) On the other hand, Docker delivers software in packages called containers (they are ready to use). It is very cool, isn’t it? If you want to learn more about Docker, please visit their website: [https://www.docker.com/](https://www.docker.com/) ## Docker Hub Let’s use Docker Hub (the world’s largest library and community for container images) to find a SonarQube Docker container to perform our MuleSoft applications code reviews. Step by step: - Go to Docker Hub: [https://hub.docker.com/](https://hub.docker.com/) - Sign in (or sign up if you do not have an account already) - Download the following image: mulesonarqube - Run the Docker image. That is all! Now you have a SonarQube instance up and running, and it is ready to perform MuleSoft code analysis. ## SonarQube Docker Container To download the Docker image (it will take a while): ```bash docker pull fperezpa/mulesonarqube:7.7.3 ``` ![Terminal showing docker pull of the mulesonarqube:7.7.3 image with all layers Pull complete](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-1.png) To run the Docker image: ```bash docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 fperezpa/mulesonarqube:7.7.3 ``` You should see something like this in your command line: ![Terminal with docker run then docker ps showing the sonarqube container Up on ports 9000 and 9092](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-2.png) Or in your Docker Desktop: ![Docker Desktop showing the running sonarqube container's startup logs](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-3.png) If you want to know more about Docker Desktop (available for Windows, Mac and Linux), visit this website: [https://www.docker.com/products/docker-desktop](https://www.docker.com/products/docker-desktop) ## Go to your SonarQube instance That is it! Your SonarQube instance is ready to use. Go to: [http://localhost:9000/projects](http://localhost:9000/projects) (use admin/admin as default username and password to Log In, you can change it later). You should see something like this: ![Empty SonarQube Projects dashboard with a Create new project button](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-4.png) There is a set of Rules already configured in your SonarQube instance to analyze MuleSoft code. You can find those clicking on Rules (top menu) and then on Mule (left menu). If you are feeling lucky you can modify those rules as needed. ![SonarQube Rules page filtered by the Mule language, listing 54 Mule rules](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-5.png) There are two Quality Profiles: one for Mule 4 applications and another one for Mule 3 applications. The default Quality Profile is Mule 4. Explore it by clicking on Quality Profiles (top menu) and then filtering profiles by Mule (picking list). ![SonarQube Quality Profiles showing MuleSoft Rules for Mule 3.x and Mule 4.x, with 4.x default](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-6.png) Continue exploring your SonarQube, enjoy the trip! ## Important! XML configuration SonarQube already comes with an XML plugin. But…Your MuleSoft code is also XML. The plugin we are going to use inspects XML files, so you need to remove the XML files from the default XML plugin in SonarQube. This will avoid the XML plugin checks for your MuleSoft application and lets the Mule plugin into action. To do that, click on Administration (top menu), then Configuration and General Settings (top left menu), click on XML (left menu) and remove the .xml extension from the XML plugin configuration. You should see something like this (click on the red cross to remove .xml line): ![SonarQube XML plugin File suffixes setting listing .xml, .xsd and .xsl with remove crosses](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-7.png) Then click on the Save button and you are done! Your SonarQube is ready to check your MuleSoft code. ![XML File suffixes after removing .xml, now only .xsd and .xsl, with the Save button](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-8.png) ## Use the Maven Plugin We are about to use a Maven plugin to verify the quality of our MuleSoft code using SonarQube. This plugin contains a set of rules and metrics that are going to be used and calculated every time a project is being inspected. This is a plugin developed by MuleSoft and it is unlicensed. If you want to know more about this plugin, please refer to the official repository: [https://github.com/mulesoft-catalyst/mule-sonarqube-plugin](https://github.com/mulesoft-catalyst/mule-sonarqube-plugin) To use this plugin, open a command line and navigate to your MuleSoft project’s home directory and run the following command: ```bash mvn sonar:sonar -Dsonar.host.url=http://localhost:9000 -Dsonar.sources=.\src ``` - -Dsonar.host.url is the URL where your SonarQube instance is running. - -Dsonar.sources is the path where the code to analyze is (typically src folder under your MuleSoft project home). I have quickly created a Kafka producer-consumer MuleSoft app and I will analyze this code. You should see something like the following: ![Terminal output of mvn sonar:sonar starting analysis of the kafka-producer project](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-9.png) Finally, the plugin completes the code analysis, and the report will be available at your SonarQube instance: ![Maven output ending in ANALYSIS SUCCESSFUL and BUILD SUCCESS with the report URL](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-10.png) ## SonarQube Report You will find the report in your SonarQube instance. Just navigate to Projects (top menu), you should see your project there. ![SonarQube Projects list with the kafka-producer project Passed, showing 2 bugs and 5 code smells](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-11.png) Click on your project and explore the report details. ![kafka-producer project overview with Bugs, Vulnerabilities, Code Smells and Duplications metrics](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-12.png) Let’s take a quick look at the Bugs section. ![SonarQube Issues view listing two bugs including credentials should be managed with properties](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-13.png) ## Conclusion The report is highlighting a bug that says that credentials and resources should be managed with application properties. This was intentional. The application has an HTTP Listener with hardcoded host and port values. This is not a good practice to develop MuleSoft applications, those should be placed in a properties file. ![Bug detail showing the offending HTTP Listener config XML with hardcoded host and port 8081](../../assets/blog/easiest-way-to-integrate-automatic-code-review-of-mulesoft-apps-using-sonarqube-and-docker-container-14.png) You can also explore the Mule default Rules definition in your SonarQube instance and modify/add rules as needed. If you have different MuleSoft coding standards you can enrich that set of rules. This is the first step to semi-automate code reviews for MuleSoft projects. After this you can integrate this Maven plugin with a CI/CD pipeline. This is very cool, right? --- ## Trigger Events in Salesforce; Receive Them in MuleSoft Source: https://prostdev.com/post/trigger-events-in-salesforce-receive-them-in-mulesoft | Published: May 18, 2021 | Category: Tutorials We all know that MuleSoft is now part of Salesforce. It's been more than two years since Salesforce acquired it. But even before the acquisition, those two have had a strong integration. In my opinion, as a person who has been working in the integration space for more than 20 years, one of the most attractive things within their integration through the MuleSoft Salesforce connector, is the ability to subscribe to events that occur within Salesforce. One of the challenges that has always been present when you are integrating an application is the ability to generate events that 3rd parties can subscribe to. This is a good element for an Event-Driven Architecture (EDA), and really a differentiator if you are building a connectivity/integration initiative. Well, Salesforce offers a lot of alternatives to generate events. Not only within Salesforce boundaries, but also across them. And one key player here for that regard, is MuleSoft via its Salesforce Connector. Salesforce offers: - Push Topic Event - Generic Event - CDC - Platform Event - etc. Which are different alternatives to generate events within Salesforce, after a change has occurred in one of its objects or when you programmed that the event should occur (APEX, Workflow). Some of them are more flexible than others in terms of the customization and data that the event can contain. But something that they have in common, is that they can be used by the MuleSoft Salesforce Connector, and with that, trigger a MuleSoft flow. And now within a MuleSoft flow, you can pretty much do whatever you want with the data and send it to other systems, or publish the information through another event mechanism for 3rd parties. Let's imagine you want to inform another system every time a create, update or delete for a Lead happens. But you do not want to be polling Salesforce every now and then to look up for new Leads. What you want is to be notified that the event happened and then manipulate the information and send it to others, or process them, or whatever you want to do with it. In the last paragraph I said "you do not want to be polling," I mentioned it because that is a common practice, but a very bad one. Rather than doing that, you can subscribe for events and process them only when they happen and you are ready to receive them. That is a cleaner and more effective approach, and it can help you to understand the value of an Event-Driven Architecture. MuleSoft is a very versatile platform. It offers an API platform, but also an API implementation and integration platform. Not only that, it can be used to expose asynchronous APIs, to serve as part of an EDA architecture. In this post I am going to show you how we can create a MuleSoft Flow that can react to three different events generated by Salesforce: 1. On Create Event 2. On Modify Event 3. On Delete Event We will use Lead as the entity in Salesforce. We are interested to know if a Lead was created, updated, or deleted. But again, we want to react to the event. We are not going to be asking Salesforce if a Lead was created, updated or deleted. No! We are going to subscribe and react. ## Creating the Mule Project In order to do that, let's create a MuleSoft project that will look like this: ![Mule project with three flows, each a Salesforce trigger (New, Modified, Deleted Object) into a Logger.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-1.jpg) We have three different flows: one per event. After we receive the event, we simply send it to the logger. In order to achieve this, we need two things: - Create the MuleSoft project and use the Salesforce connector. In order to configure the Salesforce connector, you need a developer account ([https://developer.salesforce.com/](https://developer.salesforce.com/)). - Create a Lead in Salesforce and play with it: update it and delete it. The steps are: 1) Create a MuleSoft Project: ![New Mule Project dialog with Mule Server 4.3.0 EE runtime selected and an empty project name.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-2.jpg) 2) Name it: “triggers01” or any name you would like to use. 3) Once the project is created, let's create a properties file that will include the Salesforce connectivity information. ![salesforce.yaml under src/main/resources holding sfdc username, password and token properties.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-3.png) The token is generated via the developer.salesforce.com portal, as well as the user and password. 4) The properties (yaml) file needs to be located in the folder that is indicated in the previous image (`src/main/resources`). 5) Now let's create two entries in our Global Elements tab: ![Global Configuration Elements tab with Configuration properties and Salesforce_Config entries.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-4.jpg) 6) For the Salesforce Configuration, you should input this: ![Salesforce Config properties with Basic Authentication username, password and security token from the YAML.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-5.jpg) As you can see we have configured username, password and token. We’re getting the values from the properties file. It is a good practice to use this type of configuration. Once you typed it, click on Test Connection: ![Salesforce Config showing a "Test connection successful" confirmation dialog.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-6.jpg) 7) Now it’s time to create the flow. As we've previously mentioned, we will react to events. Therefore, we are not going to drag and drop HTTP listeners. We are going to drag and drop a Salesforce Connector capability. 8) Let's start with the On Create Event. But before that, make sure you have added the Salesforce Connector into your Mule Palette. Like this: ![Mule Palette with the Salesforce connector module added alongside Core and HTTP.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-7.jpg) 9) Now, let's add the On New Object. Please locate it, and drag and drop it in the Canvas: ![Salesforce palette operations list with the On New Object trigger highlighted.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-8.jpg) 10) Now you should have this: ![A single flow with the On New Object Salesforce trigger dropped onto the canvas.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-9.jpg) 11) Repeat the same formula for these two events: ![Salesforce palette with the On Modified Object and On New Object triggers highlighted.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-10.jpg) 12) Drag and drop them into the canvas. And now you should have this: ![Three flows each starting with a Salesforce trigger: On New, On Modified, and On Deleted Object.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-11.png) 13) Now let's add a logger activity into the On New Object flow, and configure it like this: ![Logger added after the On New Object trigger, configured with the payload as its message.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-12.jpg) 14) Repeat the same formula for the other two flows. Now you will have this: ![All three flows now complete: each Salesforce trigger connected to its own Logger.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-13.jpg) 15) Now let's configure the three events. Let's start with the On New Object event. Double click the activity and configure it like this: ![On New Object config: Lead object type, Fixed Frequency scheduling, every 5 seconds.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-14.jpg) As you can see we've configured different elements: - Display Name: It can be whatever you want. - Connector Configuration: It should be already selected, since you created it in previous steps. - Object Type: This is the entity to which you want to be informed if a new one is created, modified or deleted. In this case we are going to use Lead. - Scheduling Strategy: It is the frequency to which we are going to get the event messages. - Since: This is a key element because events in Salesforce are independent of the Connector. So, if you are already publishing events when you start the MuleSoft app, it will retrieve the messages that were created before. But if you choose a date, then only the events created after that date will be retrieved. So be careful with this field. - Frequency: It is related to the Scheduling Strategy. In our case it will be every five seconds. - Time Unit: In this case, seconds. 16) Now let's do the same thing for the On Modify and On Delete, like this: ![On Modified Object config with object type Prospecto (Lead) and Fixed Frequency scheduling.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-15.jpg) and ![On Deleted Object config with object type Prospecto (Lead) and Fixed Frequency scheduling.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-16.jpg) 17) Now we are ready to test this. Right click your project and Run the project. The project should be in the DEPLOYED status. Like this: ![Mule console showing the triggers01 application in DEPLOYED status on Mule Server 4.3.0 EE.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-17.jpg) Now we are ready to create some Leads in Salesforce, and see how our MuleSoft application reacts to them. ## Triggering the Mule Flows Go to https://salesforce.com and log in with your credentials. Look for the Leads tab: ![Salesforce Lightning Sales home in Spanish with the Prospectos (Leads) tab in the navigation.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-18.jpg) (Here you see it in Spanish, because my defaults are in Spanish. But in English it is “Lead”). Let's create a new Lead: ![New Lead form in Salesforce filled in for Jose Perez at company SPS, with email perez@me.com.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-19.jpg) Just fill the necessary elements, and click create the Lead. After that, get back to Anypoint Studio, and see what was written in the log: ![Two Mule log lines from triggers01Flow and triggers01Flow1 logged after creating the Lead.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-20.jpg) You have two new lines, and if you scroll to the right, you will see the details of the Lead you've created: ![Logged Lead details scrolled right, showing FirstName=Jose and Title=CTO for the new record.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-21.jpg) Why two? Every time you create an object, it generates two events: one for creation, and one for modification. Now let's test the modification event. Let's edit the Lead changing the contact email: ![Lead detail page for Mr. Jose Perez with the email field perez@me.com highlighted before editing.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-22.jpg) Let's edit the email address: ![Editing the Lead's email field to jose_perez@me.com in the Salesforce inline editor.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-23.jpg) After you've edited, you will see a 3rd line in your log, where you can see the updated email address: ![Mule log of the modify event showing the updated Email=jose_perez@me.com value.](../../assets/blog/trigger-events-in-salesforce-receive-them-in-mulesoft-24.jpg) Pretty simple, right? With this, you can change the perspective on how you can integrate Salesforce with other applications. The connector is very powerful and the events layer in Salesforce as well. And they are integrated, so you can do the math and conclude that this is a great ability within MuleSoft. --- ## Mule 4 Unsplash Connector Source: https://prostdev.com/post/mule-4-unsplash-connector | Published: May 11, 2021 | Category: Tutorials *GitHub repositories with the projects can be found at the end of the post.* ## What is Unsplash? Unsplash is a website dedicated to sharing stock photography under the Unsplash license. Since 2021, it has been owned by Getty Images. The website claims over 207,000 contributing photographers and generates more than 17 billion photo impressions per month on their growing library of over 2 million photos. Basically Unsplash allows you to download images for free and use them under Unsplash license on your web pages and projects. Personally I’m a big fan of this community. I’ve been uploading photographs for the last couple years and it’s really amazing how many of the images uploaded have been used in a number of sites like BuzzFeed, Adobe Spark and Trello. Also many people share on Twitter and email how they wanna use my images. Unsplash has a [REST API](https://unsplash.com/documentation) that provides a bunch of nice features like image search, photo actions, collections, topics and users. Basically you can use all these actions to create any kind of automated application. This time I have created a new custom connector for Mule 4 to use a couple operations (Get Random Photo and Search Photos). ## Creating the custom connector Creating a custom module (connector) is easier than you think, so let me list the steps I followed. 1. Executing the maven archetype command generate in your terminal. ```bash mvn org.mule.extensions:mule-extensions-archetype-maven-plugin:generate ``` 2. Then a few questions need to be answered related to the project. ``` Enter the name of the extension: Enter the extension's groupId: Enter the extension's artifactId: Enter the extension's version: 1.0.0 Enter the extension's main package: ``` 3. Run the mvn eclipse:eclipse command in the terminal, so this way when we can import our projects and let it be recognized by the IDE (in case you are using Eclipse IDE). I personally like IntelliJ IDEA. 4. Import the generated project into the IDE. In my case, since I’m using the IntelliJ IDEA I select the maven project (*it’s easier if the IDE is in charge of pulling the maven dependencies into the project*). ![IntelliJ IDEA Import Project dialog with the Eclipse external-model option selected](../../assets/blog/mule-4-unsplash-connector-1.png) Finally, the connector is imported and we are able to start working on it. 5. Now for this specific connector we are going to use only two of the auto-generated java classes: **UnsplashConfiguration** and **UnsplashOperations.** - **UnsplashConfiguration.** Allows to set up any general parameter we might use. In this case we needed to set the Unsplash Access Key required for API calls. - **UnsplashOperations**. Here we will specify our first operations in the connector. We should be able to include more operations later. For now we will only have **GetRandomPhoto** and **SearchPhoto.** - **GetRandomPhoto.** Will retrieve a random picture from the API. - **SearchPhoto.** Will let you search based on a keyword and retrieve some picture information. ## Get Random Photo This operation will allow you to get a random set of photos from Unsplash and be able to specify some additional attributes like count (number of images you want to retrieve), keyword (specify the keyword for images you want to get from Unsplash) and orientation (it can be portrait, landscape or squarish). The connector will show the operations this way: ![Get random photo operation config with Count, Keyword, and Orientation portrait/landscape/squarish](../../assets/blog/mule-4-unsplash-connector-2.png) ## Search Photo The search operation allows you to search photos on the Unsplash API specifying a few optional attributes like **search terms** (this will tell the search the photos you are looking for), **page** (specify the page you are in), and **per page (**tells how many records per page you want). ![Search photo operation config with Search term, Page, and Per page fields in Anypoint Studio](../../assets/blog/mule-4-unsplash-connector-3.png) ## Installing the Mule module The installation is really easy. Basically we just need to run the command: ```bash mvn clean install ``` After running the command from terminal or IntelliJ IDEA, the new module will appear on the add modules palette. ![Mule Palette Add Modules view with the custom Unsplash module listed under Workspace](../../assets/blog/mule-4-unsplash-connector-4.png) Then we just need to drag and drop the module for our project. In this specific case I have created a simple application that uses both operations. Search operation in this case contains a foreach component that allows you to get the photo from Unsplash API and place the actual image in your local directory. Here is a sample of images downloaded from Search: ![Two Mule flows using the Unsplash operations and a folder of photos downloaded from Search](../../assets/blog/mule-4-unsplash-connector-5.png) ## GitHub repository You can get both projects from the repositories: Connector: [https://github.com/emoran/mule4-unsplash-connector](https://github.com/emoran/mule4-unsplash-connector) Connector implemented code: [https://github.com/emoran/unsplash-connector-implementation](https://github.com/emoran/unsplash-connector-implementation) Hope you enjoy this project and hopefully it will help with any type of project / integration you’re working on. --- ## ProstDev's New Blog Post Template Source: https://prostdev.com/post/prostdev-s-new-blog-post-template | Published: May 4, 2021 | Category: Guides Hi there! You’re probably here because you want to create a blog post with us. Awesome! Let’s get started. If you’re a new writer, please take a moment to read through this post to understand the different guidelines and standards when publishing in ProstDev. Also, feel free to look at the other content creation articles to read about best practices and tips for writing: [Content Creation articles](http://www.prostdev.com/blog/categories/content-creation). - Google Doc Template - Activating Word Count - Blog Post Title (under 100 characters) - Excerpt (max. 140 characters) - SEO Description (max. 500 characters) - Article Formatting - Start the Review Process ## Prerequisites If you’re not familiar with any of the following tools/terms, please take a moment to get acquainted with them before starting this guide. - [Google Drive / Google Docs](https://support.google.com/drive/answer/2424384?co=GENIE.Platform%3DDesktop&hl=en) - [SEO (Search Engine Optimization)](https://en.wikipedia.org/wiki/Search_engine_optimization) - [Gist files](https://docs.github.com/en/github/writing-on-github/creating-gists) ## 1. Google Doc Template First of all, open the following Google Doc template to have a starting point: [https://docs.google.com/document/d/1oCEdZXUnRY86MOogWeTR6Jm5IbGyvVhqoyjD9oXY-eA/edit?usp=sharing](https://docs.google.com/document/d/1oCEdZXUnRY86MOogWeTR6Jm5IbGyvVhqoyjD9oXY-eA/edit?usp=sharing) Make sure to be logged in to Google Drive, or click on the “Sign In” button in the top right corner. ![Top right corner when you’re not signed in.](../../assets/blog/prostdev-s-new-blog-post-template-1.png) ![Top right corner after you’re signed in.](../../assets/blog/prostdev-s-new-blog-post-template-2.png) After this, you can select File > Make a copy. ![Google Docs File menu open with the "Make a copy" option highlighted](../../assets/blog/prostdev-s-new-blog-post-template-3.png) Change the name and select in which folder you want to save it. Click “OK.” ![Google Docs "Copy document" dialog with a name field and a folder set to My Drive](../../assets/blog/prostdev-s-new-blog-post-template-4.png) Now you can start editing the template with your own content! > [!NOTE] > Please continue using Google Docs to create your post because the review process will happen on this platform. You can also create your post in other platforms or tools, but you’ll then have to move the content into a Google Doc to continue with the process. ## 2. Activating Word Count This is a neat trick that I *always* use when I’m reviewing or writing a post. In your Google Doc, select Tools > Word count. ![Google Docs Tools menu open with the "Word count" option highlighted](../../assets/blog/prostdev-s-new-blog-post-template-5.png) After this, activate the “Display word count while typing” option and click “OK.” ![Word count dialog with the "Display word count while typing" checkbox enabled](../../assets/blog/prostdev-s-new-blog-post-template-6.png) This will add a floating menu/button in the bottom left corner of the document to see the different counts. ![Floating word-count button in the document showing page, word, and character totals](../../assets/blog/prostdev-s-new-blog-post-template-7.png) If you haven’t selected any text, the count will be from the entire document. If you highlight some text, then this count will be only from the selected area. This will help you to keep the desired word/character count in the next steps. > [!NOTE] > Try to keep your article less than 2,000 words if possible. The average is 1,500. If you’re writing a very long post, it’s best to break it down into several articles and create a series instead. ## 3. Blog Post Title (under 100 characters) You can find some ideas to choose your title in this post - [7 tips to start writing your technical blog post](https://www.prostdev.com/post/7-tips-to-start-writing-your-technical-blog-post)*.* You will want to include some keywords so people can find your content more easily when they search on Google or other search engines (but keep the title under 100 characters!). For example, if your content is about programming in Java, make sure to include the keyword “Java” in your title. Or if the article is also focusing on showing how the “toString” method works, then have that keyword too. A good title could be something like “How to use the toString method in Java.” Identify between 2-5 keywords that *must* be included in the title first, and then see how you can structure it to be under 100 characters. If your title is too long, some keywords will have to be cut out, but don’t worry! You’ll learn how to still include them in other parts of the post for a better [SEO](https://en.wikipedia.org/wiki/Search_engine_optimization). ## 4. Excerpt (max. 140 characters) Add a very short description (less than 140 characters) of your content so the audience can get a better idea of what they’re about to read. This can also be a “hook” to keep readers interested in your content. > [!NOTE] > It’s not necessary to include any keywords here. This is just for the actual reader and not for the search engines. For example, “In this post, I explain how to use the ‘toString’ method and some of the most performant ways to use it.” ## 5. SEO Description (max. 500 characters) *Now* we’re back to the SEO keywords that you couldn’t include in your title from step 3. Add a paragraph of 500 characters or less with the keywords that didn’t fit in the title because of the character restriction. This text does not have to appear in the actual article. It can be added to the SEO Description without the need to add it to the post. This description will appear in the search results (for example, from Google), so try to think like your target audience. Ask yourself, “What would a person be *googling* to find my article?” For example, “How to use the ‘toString’ method in Java. Understanding ‘toString’. Transforming an Integer / Boolean into a String.” Try searching for some stuff yourself to see what the description includes in the results to give you a better idea of what you could put in this paragraph. Just make sure that what you add in the SEO Description is something you actually included in your content. Otherwise, your audience will feel “fooled” and won’t come back to see more *(also, it’s not cool to add keywords just to get clicks. You’re creating quality content!)*. ![Example of a Google search with the keywords "dataweave concatenate strings"](../../assets/blog/prostdev-s-new-blog-post-template-8.png) ## 6. Article Formatting Add the first subtitle after the introduction (steps 3-5) to separate the sections. After this, you can start developing your article. Some formatting to keep in mind: **1. Screenshots/pictures** If you’re adding a screenshot or a picture, make sure you explain what is demonstrated before you show the image. You can also include additional information about the image under the picture. For example: ![Screenshot taken from the "Anypoint Platform Chrome Extension" blog post by Edgar (Yucel) Moran.](../../assets/blog/prostdev-s-new-blog-post-template-9.png) **2. Code** Same as an image, explain what the code is doing before you show it. There are 3 different types of code that can be added in the article: in-line code, small scripts (approx. 1-10 lines), or large scripts (more than 10-15 lines). Unfortunately, there’s no formatting for in-line code, so please add your variable name, property name, or any other code in between quotes. For example: *Use the “secondVar” variable to...* Short pieces of code can be directly included in the post. For example: ![Screenshot taken from the "How to integrate Solace PubSub+ Cloud with MuleSoft" blog post by Pravallika Nagaraja.](../../assets/blog/prostdev-s-new-blog-post-template-10.png) Large pieces of code should be added in a [gist file](https://docs.github.com/en/github/writing-on-github/creating-gists), and then provide the gist file link in your Google Doc. For example: [https://gist.github.com/alexandramartinez/798a12afe843c7218ae2564b9a0bc9d7](https://gist.github.com/alexandramartinez/798a12afe843c7218ae2564b9a0bc9d7) The final output will look something like this: ![Screenshot taken from the "How to integrate Solace PubSub+ Cloud with MuleSoft" blog post by Pravallika Nagaraja.](../../assets/blog/prostdev-s-new-blog-post-template-11.png) ## 7. Start the Review Process Once your article is all done, click that “Share” button in the top right corner of the screen. ![Top-right corner of Google Docs with the blue "Share" button highlighted](../../assets/blog/prostdev-s-new-blog-post-template-12.png) Select the “Anyone with the link” option, and you can either select “Commenter” or “Editor” as the role. The difference is that a “Commenter” can *suggest* changes, but you have to review and accept them, whereas an “Editor” can modify the document just as you can. If you’re not familiar with the review process, you can select “Editor.” ![Google Docs share dialog set to "Anyone with the link" with the role dropdown showing Editor](../../assets/blog/prostdev-s-new-blog-post-template-13.png) After that, click the “Copy link” button. Now go to [ProstDev.com/contact](https://www.prostdev.com/contact) and go to the “Contribute” section. Paste the copied link into the “Link to your article” text box. Fill out the rest of the fields, and click “Submit.” ![Contribute form found in ProstDev.com/contact to submit your content.](../../assets/blog/prostdev-s-new-blog-post-template-14.png) That’s it! After that, we’ll review your article and provide feedback if anything needs to be changed. Once the review process is complete, we’ll schedule the content and create the social media. ## Recap - **Make a copy** of the Google Doc Template into your own Google Drive. ([https://docs.google.com/document/d/1oCEdZXUnRY86MOogWeTR6Jm5IbGyvVhqoyjD9oXY-eA/edit?usp=sharing](https://docs.google.com/document/d/1oCEdZXUnRY86MOogWeTR6Jm5IbGyvVhqoyjD9oXY-eA/edit?usp=sharing)). - **Activate the Word Count** option and try to keep your document less than 1,500-2,000 words. - **Create your title** using keywords for a better SEO (max. 100 characters). - **Create the post’s excerpt** for the reader (max. 140 characters). - **Create the post’s SEO Description** (not added to the blog post) with additional keywords for the search engines (max. 500 characters). - **Make sure to add explanations** before adding media and follow the code formatting for in-line, short, or large scripts. - **Share your document** at [www.prostdev.com/contact](https://www.prostdev.com/contact) to start the review process. If you have additional questions, take a look at the other blog posts in the [Content Creation category](https://www.prostdev.com/blog/categories/content-creation) for best practices and advice. You can also contact us or comment here if you still have questions. Hope to see your articles soon :D *Prost!* -Alex --- ## The Power of cURL - Part II Source: https://prostdev.com/post/the-power-of-curl-part-ii | Published: Apr 27, 2021 | Category: Tutorials Welcome to part II of the The Power of cURL. If you are new to cURL, I recommend checking out my previous post [here](https://www.prostdev.com/post/the-power-of-curl). There I provide an overview of cURL and how to invoke HTTP methods from the command line. I go into detail on how to invoke APIs using HTTP GET and POST requests, as well as how to pass query parameters, URI parameters and headers. This post will explain how to invoke APIs using HTTPS via **cURL**. ## Review of cURL [cURL](https://curl.se/) is a tool used to transfer files. The cURL command can be used inside scripts or from the command line. cURL provides support for common protocols like HTTP, HTTPS, FTP and much more. This article will focus on using the cURL command to invoke integrations that use HTTPS. ## cURL Requests cURL commands begin with the keyword “curl” followed by an option. REQUEST OPTION: To specify the request method in an HTTP/HTTPS request. Use the --request or -X options. - SYNTAX: ```bash curl --request '' ``` - OR: ```bash curl -X '' ``` > [!NOTE] > This article does not use ' ' around the url but I recommend using quotes when passing query parameters. If possible, try to make using single-quotes around the url a habit. ## Demo Time The cURL commands provided in this article will be demonstrated by first invoking the JSON Placeholder API. It is a fake API used for building prototypes. Learn more about it [here](https://jsonplaceholder.typicode.com/). The latter portion of the article will be demonstrated by invoking a sample Mule API configured for HTTPS. Let’s get started! Let's make a request to the infamous JSON Placeholder API. EXAMPLE REQUEST ```bash curl --request GET https://jsonplaceholder.typicode.com/todos ``` ![Terminal showing a curl GET to the JSON Placeholder API returning a list of to-do objects](../../assets/blog/the-power-of-curl-part-ii-1.png) Notice we get a JSON list of to-do objects. **Simple, right?** This works because the server hosting [https://jsonplaceholder.typicode.com](https://jsonplaceholder.typicode.com/) has a digital certificate that is issued by a trusted Certificate Authority (CA). When an HTTPS request is invoked, the server will present its certificate. If the certificate is in the API client’s trust store, the API client recognizes the server as trusted and will accept the API’s response. **What happens if the server hosting the API is not trusted? Or not in the API client’s trust store?** This time I have configured HTTPS via TLS on a simple Mule API on my local machine. When invoking the API through curl, I get the following error. For this example, I have used a self-signed certificate. EXAMPLE REQUEST ```bash curl --request GET https://localhost:8081/curl/helloget ``` ![Terminal showing the curl error "SSL certificate problem: self signed certificate"](../../assets/blog/the-power-of-curl-part-ii-2.png) Notice the error I get back when invoking the API. “curl: (60) SSL certificate problem: self signed certificate.” This error occurred because the certificate was not from a trusted source. This certificate is not present in machines. But why is cURL screaming? The self-signed certificate is not trusted, that means it is not in my computer’s trust store. cURL offers 4 options in which one can resolve this type of error. I will go over 2 options but feel free to check out all four options [here](https://curl.se/docs/sslcerts.html). **Option 1:** Disable SSL certificate validation* A work-around to this is to add the --insecure or the -k option. These two options disable peer SSL certificate validation. - SYNTAX: ```bash curl --request '' --insecure ``` - OR: ```bash curl --request '' -k ``` > [!NOTE] > This option is insecure and disables SSL certificate validation. I recommend only using this option when invoking and/or testing internal APIs and you are certain the host is from a trusted source. EXAMPLE REQUEST ```bash curl --request GET https://localhost:8081/curl/helloget --insecure ``` ![Terminal showing a successful Mule API response after adding the curl --insecure option](../../assets/blog/the-power-of-curl-part-ii-3.png) **Option 2:** Provide the certificate during the invocation This option requires you to retrieve and save the certificate (or public key) of the API host. Then save this certificate in a location on your machine and reference it during the API invocation. Unlike option 1, SSL certificate validation is still in place and is therefore a safer option. - SYNTAX ```bash curl --request --cacert '' ``` EXAMPLE REQUEST ```bash curl --request --cacert C:Desktop/demo.cer https://localhost:8081/curl/helloget ``` ![Terminal showing a successful Mule API response using the curl --cacert certificate option](../../assets/blog/the-power-of-curl-part-ii-4.png) Notice in both option 1 and option 2, we were able to get a response from the Mule API. ## Conclusion cURL is a powerful tool. The 2 articles in this series only scratch the surface of cURL. If you would like to learn more, review the [cURL man page](https://curl.se/docs/manpage.html) and the [cURL ebook](https://everything.curl.dev/). Lastly, I am Whitney Akinola. I am a fellow MuleSoft Developer and content creator. Feel free to check out my technical content[here](https://www.whitneyakinola.io/blog). I am also looking for integration topics to write about. If you have a topic you would like me to explore and write about, please feel free to request. You can [contact me](https://linktr.ee/whitneyakinola) personally via LinkedIn or Twitter. --- ## 3 ways to import a function/module in DataWeave 2.0 Source: https://prostdev.com/post/3-ways-to-import-a-function-module-in-dataweave-2-0 | Published: Apr 20, 2021 | Category: Guides There are different ways to import a module in DataWeave. You need to do this for the modules that are not a part of the [core functions](https://docs.mulesoft.com/mule-runtime/4.3/dw-core). Here is a list of all the modules in DataWeave: [Mule Docs - DataWeave Reference](https://docs.mulesoft.com/mule-runtime/4.3/dw-functions). ## 1. Import module directly You can simply import the module without specifying the functions’ names. If you do this, you will need to reference the module every time you want to use that module’s function. Like this: ```dataweave %dw 2.0 output application/json import dw::core::Strings --- Strings::capitalize("alex martinez") ``` To reference the function, you need to use the name of the module, then add two colons (::), and finally, the name of the function. After doing this, you can continue using the function as usual. ## 2. Import all functions from a module You can import all of the functions from the module by using the asterisk (*) and using the keyword “from.” This will let you use the function directly without referencing the module’s name every time. Like this: ```dataweave %dw 2.0 output application/json import * from dw::core::Arrays --- { field1: [true, false] every ($), field2: [true, false] some ($) } ``` This approach, however, will import all of the functions that are found inside the module. Normally, this doesn’t affect performance. But I personally want to see which functions exactly I am importing to my code. For example, in the previous script, I don’t know if “every” or “some” are functions that come from the Arrays module or if they’re core functions. For this reason, my favorite approach is the 3rd option described below. ## 3. Import one or more functions from a module To explicitly import each function separately, you have to list them instead of just writing the asterisk (*). Like this: ```dataweave %dw 2.0 output application/json import every, some from dw::core::Arrays --- { field1: [true, false] every ($), field2: [true, false] some ($) } ``` This will still let you call the function directly. The difference is that you’ll have to list all of the functions you’re using. I prefer this approach because sometimes you may not know which function comes from which module (especially for big scripts). With this, you can just search for the function and see which module is importing it. Of course, it can become annoying if you’re changing functions often, and you remove and re-add them several times. You will also get an error if you forget to list one of the functions that you’re using. For example, the following script shows an error because the “some” function is not listed in the import statement: ```dataweave %dw 2.0 output application/json import every from dw::core::Arrays --- { field1: [true, false] every ($), field2: [true, false] some ($) // Unable to resolve reference of: `some`. } ``` The option you choose really depends on your personal preference or the standards of your project. By the way, do you know how to create your own modules and reference them for your DataWeave scripts? If not, check this out! - [Custom Modules in Mule 4 - DataWeave 2.0](https://www.prostdev.com/post/custom-modules-in-mule-4-dataweave-2-0). -Alex --- ## How to integrate Solace PubSub+ Cloud with MuleSoft - Negative Scenario (Error Handling) Source: https://prostdev.com/post/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling | Published: Apr 13, 2021 | Category: Tutorials *GitHub repository with the Mule project can be found at the end of the post.* In my previous post ([How to integrate Solace PubSub+ Cloud with MuleSoft](https://www.prostdev.com/post/how-to-integrate-solace-pubsub-cloud-with-mulesoft)), I explained the happy path of the Solace PubSub+ and MuleSoft Integration. In this post, let us see how we can handle an error efficiently. Even though the error handling logic varies depending on use cases, business requirements, organizational processes, etc. I believe the core approach to error handling for message-based integrations would more or less remain the same for different applications. The most common approach consists of the following steps: - In case of an error, the error is captured in the error handler logic. - Enrich the message with additional information that could be later used for reprocessing. - Keep track of the “deliveryCount” and other useful information in the “User Properties.” - Publish the message to an Error Topic for retrying. - Process the message from the error queue and depending on the “redelivery_count” attribute, either route it back to the source queue or the dead letter queue (or Dead Message Queue). Further reprocessing can happen by the operations team. Let us see how I have implemented the above approach in the following use case. Before that, please take a look at my [previous blog post](https://www.prostdev.com/post/how-to-integrate-solace-pubsub-cloud-with-mulesoft) on how to publish the message to the Solace Topic using the JMS “Publish” connector. **1)** The message is received by the “On New Message” JMS Connector. The incoming message is transformed in the “Request Transformation” component, While attempting to create the Salesforce Account using the “Create” connector, there is an error (in case the account already exists). The choice router validates if a successful response is received from SFDC. In case of failure, an error is thrown. (Note that you can also add an “Until Successful” block around the “Create” SFDC connector to retry for a specified amount of time before throwing an error.) ![Subscribe flow receiving a JMS message, transforming it, and creating a Salesforce Account with a Choice router](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-1.png) The error is caught in the “On Error Propagate” error handler and invokes the “processSFDCError” flow. ![Choice router branches plus an On Error Propagate handler referencing the processSFDCError flow](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-2.png) ![processSFDCError flow: Transform Message, Logger, and a sendNegativeAck Flow Reference](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-3.png) In case of Salesforce Connectivity error, then the error is thrown in the “Create” SFDC connector and caught in the “SFDC:ERROR” (On Error Propagate) as the error mapping is done in the connector level as shown in the following screenshot. ![Salesforce Create connector Error Mapping mapping ANY error type to SFDC:ERROR](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-4.png) **2)** The message is published to the Error Topic as shown in the following screenshot: ![sendNegativeAck flow setting error variables then publishing the message to the error queue](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-5.png) Note that I am capturing the “DELIVERYCOUNT”, “EXCEPTIONDETAILS” and “TIMESTAMP” in the “User Properties” field. ![JMS Publish config to the error topic with DELIVERYCOUNT, EXCEPTIONDETAILS, and TIMESTAMP user properties](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-6.png) **3)** The message from the Error Queue is received in the retry-flow. In this flow, if the “redelivery_count” (`attributes.properties.jmsxProperties.jmsxDeliveryCount`) value is less than or equal to the “maxRedeliveryCount” variable, which will be fetched from the properties file, then the message will be published to the source Topic. ![retry-flow listening on the error destination and setting flow, destinationTopic, and deadLetterTopic variables](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-7.png) ![Solace sample-test-error-queue showing one queued message after the first failure](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-8.png) Since the redelivery count is 1, the message is published back to the source Topic. ![Mule Debugger on the retry-processing-flow Choice that checks redelivery_count against maxRedeliveryCount](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-9.png) ![Solace sample-test-queue showing the message republished to the source queue](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-10.png) **4)** The message is received by the subscribing flow and tries to process it. In case of error during the second time, the message is again published to the Error Topic with redelivery_count=2. ![Mule Debugger variables showing redelivery_count = 2 after the second failure](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-11.png) **5)** In case of failure during the third retry, the message is again published to the Error Topic with redelivery_count=3. ![Mule Debugger variables showing redelivery_count = 3 after the third failure](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-12.png) **6)** Since the “redelivery_count” now equals the “maxRedeliveryCount” variable, the message is published to the Dead Letter Topic or Dead Message Topic. A notification email can be triggered to the operations team for further processing of this message. ![Debugger showing redelivery_count equals maxRedeliveryCount 3, so the message is published to the DLQ](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-13.png) ![Solace sample-test-dead-letter-queue showing the message after the max retries were exceeded](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-14.png) ![Solace Queues list with the dead-letter, error, and source test queues and their message counts](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-15.png) The message in the Error Topic is Acknowledged with the “retryAckId” variable as shown in the following screenshot. ![retry-flow acknowledging the error-topic message with the retryAckId variable](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-16.png) **7)** In case of a Transformation error, the message can be sent directly to the Dead Letter Queue as retrying does not make any sense. ![process-transformation-error flow publishing the message straight to the DLQ instead of retrying](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-negative-scenario-error-handling-17.png) Thank you for reading my blog post :) Happy Learning, Pravallika Nagaraja ## GitHub repository [ProstDev GitHub - Solace MuleSoft Integration](https://github.com/ProstDev/solace-mulesoft-integration) --- ## How to integrate Solace PubSub+ Cloud with MuleSoft Source: https://prostdev.com/post/how-to-integrate-solace-pubsub-cloud-with-mulesoft | Published: Apr 6, 2021 | Category: Tutorials *GitHub repository with the Mule project can be found at the end of the post.* In this article, I will demonstrate how to integrate Solace PubSub+ with MuleSoft to publish and consume the messages. ## What is PubSub+? PubSub+ is a platform that enables the design, discovery, streaming and full lifecycle management of events across distributed enterprises. It provides all of the tools necessary to support a modern event-driven architecture. The main components of the platform are PubSub+ Event Brokers, PubSub+ Event Portal and PubSub+ Cloud Console. *More information in the* [PubSub+ for Developers](https://www.solace.dev/) *website.* ## Advantages and Disadvantages of Event-Driven Approach Advantages - Flexibility - Scalability - Agility - Responsiveness - Real-time Interactions - Processing on-demand Disadvantages - More careful design required - More thought goes in design and tracking Now let us see the steps involved in Integrating Solace and MuleSoft. ## 1. Creating a Solace Cloud Trial Account You can create a Free Solace trial account using the following link: [https://console.solace.cloud/login](https://console.solace.cloud/login) ![Solace PubSub+ Cloud login page with email and password fields](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-1.png) Click on “Sign Up” and fill in all the details. You will receive a welcome email in your inbox. Now you will be able to sign in with your credentials in the login page. ![Solace 60-day free trial sign-up form with name and password fields](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-2.png) ## 2. Creating the Messaging Service Once you login with your credentials, you will see a welcome screen like the following one. ![PubSub+ Cloud welcome console showing Event Management and Event Streaming tiles](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-3.png) Now click on “Cluster Manager,” located at the left side of the screen to create a Service. According to the [official Solace documentation for Managing Services](https://docs.solace.com/Configuring-and-Managing/Managing-Services.htm#mc-main-content) - Solace PubSub+ event brokers support the following Services: - Solace Message Format (SMF)—This Service allows clients to communicate with an event broker on the Message Backbone interface. - Solace Element Management Protocol (SEMP)—This Service allows management applications to communicate with an event broker on the Management interface. - Web transport—This Service allows Web clients to communicate with an event broker. - Solace Representational State Transfer (REST)—This Service allows REST clients to communicate with an event broker using standard HTTP requests. - Message Queuing Telemetry Transport (MQTT)—This Service allows clients to communicate with an event broker using the MQTT messaging protocol. - Advanced Message Queuing Protocol (AMQP)—This Service allows AMQP clients to communicate with an event broker using the AMQP open standard application layer protocol. Let us spin up a Service. First select the Service Type you want to use. There are two types of Services: - Enterprise: A highly available Service on dedicated infrastructure with customizable connections - Developer: A Service on dedicated infrastructure with 100 connections. In this case, I have selected “Developer.” ![Create Service screen with Enterprise and Developer service type options](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-4.png) Then, select the Cloud. In my case, I chose “Amazon Web Services,” or “AWS” for short. As you can see from the following screenshot, there are different clouds that you can choose from. Select any cloud depending on your need. ![Cloud picker dropdown listing AWS, Google Cloud, Microsoft Azure and Private Cloud](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-5.png) ![Create Service form with Amazon Web Services chosen and a region map](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-6.png) Select the Region. In my case, I chose “US East (N. Virginia)” because it’s closer to me. ![AWS region selector with US East N. Virginia highlighted on the world map](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-7.png) Give the Service a name. A good naming convention is to create it using the Cloud and the Environment name, like “sol-aws-dev.” You can check with your team if there are any standards to name the Messaging Services. ![Service Details step with broker version and service name sol-aws-dev entered](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-8.png) After that, select the Message VPN name and start the Service by clicking “Start Service.” Your Service should now be up and running. ![Cluster Manager Services list showing the sol-aws-dev developer service Running](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-9.png) ## 3. Configuring the Solace KeyStore in MuleSoft Once we have a Messaging Service, we’ll need to configure the KeyStore in MuleSoft in order to connect to the Service. Click on the Messaging Service to open the Services’ settings. Once inside, click on the “Connect” tab, located at the top of the screen, and select the protocol you want to use to connect the Service. In our POC, we use the “Solace Messaging” option. ![Connect tab listing client libraries: Solace Messaging, AMQP, MQTT and REST](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-10.png) After you select the protocol, you will see all the Connection Details, which will be used in MuleSoft (JMS Config) in order to connect to Solace. ![Solace Messaging connection details: username, password, VPN, host and Download PEM](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-11.png) As you can see from the screenshot above, you can download the TrustStore certificate by clicking on the “Download PEM” link. This PEM file must be converted to a JKS format. In order to convert this file, we’ll use the KeyStore Explorer tool *(you can download it here -* [https://keystore-explorer.org/downloads.html](https://keystore-explorer.org/downloads.html)*).* Open KeyStore Explorer and create a new JKS KeyStore Type as shown in the following screenshot. ![KeyStore Explorer New KeyStore Type dialog with JKS selected](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-12.png) To import the PEM certificate, select the “Import Trusted Certificate” option (command+T or Ctrl+T). ![KeyStore Explorer context menu with Import Trusted Certificate highlighted](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-13.png) Select the downloaded PEM certificate from Solace and give it a name to import it. ![KeyStore Explorer showing the imported digicert global root ca RSA certificate](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-14.png) Click on “File” and “Save As.” Enter the password of your choice and confirm. Please keep a note of the password you entered as you have to use this in the MuleSoft Configuration (JMS Config). You will see a screen similar to the next one to save the converted JKS file to your system. ![Save KeyStore As dialog saving DigicertCA.jks into a keystore folder](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-15.png) Now all you have to do is import this JKS file into your Mule Project. I put it under `src/main/resources/keystores/` but you can use it as long as it’s under `src/main/resources`. ![Studio Package Explorer with DigicertCA.jks under src/main/resources/keystores](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-16.png) ## 4. Configuring the Message Service In order to establish a connectivity between the clients and the event broker, we need to configure the Messaging Service as per our needs. Click on the “Manage” tab at the top of the screen, or next to the “Connect” tab that we just used. ![Manage tab showing Event Broker settings and PubSub+ Broker Manager quick settings](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-17.png) Locate and click the “Message VPN” button in the middle, under the PubSub+ Broker Manager Quick Settings label. A new window will open. ![PubSub+ Broker Manager Message VPN summary dashboard with usage gauges](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-18.png) If you click on the “Services” tab, you will see all the Services that are enabled. ![Message VPN Services tab listing REST settings, ports and client connection limits](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-19.png) Here are the settings that I created for the Proof of Concept. You can configure your own settings or based on mine if you’re experimenting. You can also modify the message VPN settings by clicking on the “Show Advanced Settings” as shown in the following screenshot. ![Message VPN Settings tab with the Show Advanced Settings link circled](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-20.png) In the “Stats” you will be able to see the number of messages published in your message VPN, the number of messages received, etc. ![Message VPN Stats tab showing message counts and data rate statistics](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-21.png) ### Client Connections There is no need to explicitly set the client connections since all these settings are pre-filled. ![Client Connections list showing connected Solace clients and their addresses](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-22.png) ### Access Control - Client Authentication This tab contains details about what authentication method you wish to use. By default “Basic Authentication” is enabled. If you want to use the “Client Certificate Authentication,” you could enable that. In my case I am using “Basic Authentication.” You could also see the settings of “Client Profiles,” “ACL Profiles” and “Client Usernames” as shown in the following screenshots. ![Access Control Client Authentication tab with Basic Authentication enabled](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-23.png) ### Access Control - Client Usernames ![Client Usernames tab listing default and solace-cloud-client usernames](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-24.png) ## 5. Configuring Queues Guaranteed messages in Solace PubSub+ are stored in event broker endpoints. There are two types of endpoints: queues and topic endpoints. Topic endpoints are used by JMS (that is, durable topic subscriptions). *To understand the basics of Messaging, exchange Patterns, Queues and Topics, please refer to this site:* [https://docs.solace.com/PubSub-Basics/Message-What-Is.htm](https://docs.solace.com/PubSub-Basics/Message-What-Is.htm)*.* I have created the following 3 Queues by clicking on the “+ Queue” button as shown in the screenshot. ![Queues list with sample-test-queue, error-queue and dead-letter-queue, all durable](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-25.png) ### Differences between Queues and Topics Queues: - Allow one-to-one communication (Point-to-Point). - Typically used when you want to send a single event to exactly one consumer. Topics: - Allow one-to-many communication (Publish-Subscribe). - Typically used when you want to send a single event to multiple consumers. I have created a Topic Subscription for each Queue as shown in the following screenshots. *To understand why we need to map a Topic to a Queue please refer to* [https://docs.solace.com/PubSub-Basics/Core-Concepts-Endpoints-Queues.htm](https://docs.solace.com/PubSub-Basics/Core-Concepts-Endpoints-Queues.htm)*.* ![Subscriptions tab for sample-test-queue mapped to sample-test-topic](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-26.png) ![Subscriptions tab for sample-test-error-queue mapped to sample-test-error-topic](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-27.png) ![Subscriptions tab for the dead-letter-queue mapped to sample-test-dead-letter-topic](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-28.png) ## 6. Configuring the MuleSoft Project In the MuleSoft Project, the following dependencies must be added in the `pom.xml` file. There are three ways to do this. You can add the modules directly into the pom, you can add them through Exchange from the Mule Palette, or you can right click on the project > Manage Dependencies > Manage (or Add) Modules. We need to add the Solace and JMS modules to develop our Mule Application. After adding these modules, the JMS connectors will be available in the Mule Palette: ![Studio Mule Palette searching JMS, showing Ack, Consume, Publish operations](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-29.png) Once you have these, create a Global Element for the JMS Configuration with your desired settings and the connection information from Solace (as mentioned in the “Configuring the Solace KeyStore in MuleSoft” section). For more information on the JNDI Connection properties refer to [https://docs.solace.com/Solace-JMS-API/JNDI-Connection-Properti.htm#mc-main-content](https://docs.solace.com/Solace-JMS-API/JNDI-Connection-Properti.htm#mc-main-content) Here’s an example: > [!NOTE] > The GitHub repository with all this code can be found at the end of the post. You can double-check the settings and properties there if you have some doubts. ![JMS Config global element with connection factory and JNDI name resolver settings](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-30.png) ![JMS Config provider properties mapping Solace JNDI and SSL truststore values](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-31.png) ## 7. Publishing Message to Solace Topic In my case, I added an HTTP Listener in my Mule flow to be able to listen to a request with the content that I want to publish into the Solace Topic. Here you can see how I have configured the JMS Publish connector: ![Mule publish flow with Listener, JMS Publish to Solace Topic and a Logger](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-32.png) ## 8. Testing the “Happy Path” Start the Mule App in your local to make sure that everything was correctly configured. Once the app is up and running, send a POST request from [Postman](https://www.postman.com/downloads/) (you can also use the [Advanced REST Client](https://install.advancedrestclient.com/install), [cURL](https://curl.se/), or any other tool to send a request to your local app). The Mule app will read the payload you send here and will publish it into the Solace Topic. ![Postman POST to localhost:8081/publish with a JSON account body](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-33.png) For this demo, I sent a JSON Body like this one: ```json [ { "name":"Elon Musk", "billingAddress":"Royalwest Drive", "billingCity":"Toronto", "billingCountry":"USA", "billingPostalCode":"mp3lp7", "billingState":"NY" } ] ``` As you can see in the following screenshot, the message published to the Topic can be seen in the Solace Queue mapped to it. ![Messages Queued tab for sample-test-queue showing one spooled 353-byte message](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-34.png) ## 9. Subscribing a Message from the Solace Queue The queued messages must now be processed. In my case, the message that is queued contains information from a Salesforce Account. I have to subscribe to this queue in a flow, transform it, and create an Account object in SFDC. To achieve this, I have created a new flow called “solace-jms-subscribe.” ![Mule subscribe flow consuming the queue, transforming and creating an SFDC Account](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-35.png) In the “On New Message” component settings, please note that the “Acknowledgement Mode” is set to “Manual” since we want to delete the message in the queue only after further processing it and sending the positive Acknowledgement. ![On New Message JMS settings with Queue consumer and MANUAL acknowledge mode](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-36.png) There are different Acknowledgement modes as shown. *More information on all the settings can be found in* [https://docs.mulesoft.com/jms-connector/1.7/jms-ack](https://docs.mulesoft.com/jms-connector/1.7/jms-ack)*.* ![Acknowledge Mode dropdown listing AUTO, DUPS_OK, IMMEDIATE and MANUAL options](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-37.png) When the project is deployed in Anypoint Studio, this message is automatically consumed by the “On New Message” JMS Subscriber as shown in the following screenshot: ![Mule Debugger showing the consumed message payload and the deployed app console](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-38.png) The message is then transformed to the required format. ![Transform Message DataWeave mapping queue fields to Salesforce account fields](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-39.png) And the account is created in Salesforce successfully using the “Create” Connector. After this, a positive Acknowledgement is sent back to Solace using the “Positive Ack” JMS connector. ![Salesforce All Accounts list including the newly created Elon Musk account](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-40.png) The message in the Queue is now deleted: ![Messages Queued tab for sample-test-queue now showing zero messages queued](../../assets/blog/how-to-integrate-solace-pubsub-cloud-with-mulesoft-41.png) Thank you so much for reading my post and I hope it will be of some use. In the next post we'll dive into the negative scenario and error handling. Happy Learning :) Pravallika Nagaraja ## GitHub repository [ProstDev GitHub - Solace MuleSoft Integration](https://github.com/ProstDev/solace-mulesoft-integration) --- ## The Power of cURL Source: https://prostdev.com/post/the-power-of-curl | Published: Mar 30, 2021 | Category: Tutorials [Advanced Rest Client](https://install.advancedrestclient.com/install), [Postman](https://www.postman.com/downloads/) and [SoapUI](https://www.soapui.org/downloads/soapui/) are top API clients you may have used while building or testing API integrations. These applications are useful. However there are cases where invoking the API from the command line is advantageous and is made possible through **cURL.** ## What is cURL? [cURL](https://curl.se/) is a tool used to transfer files. The cURL command can be used inside scripts or from the command line. cURL provides support for common protocols like HTTP, HTTPS, FTP and much more. This article will focus on using the cURL command to invoke integrations that use HTTP. The cURL commands provided in this article will be demonstrated by invoking a sample API built in Mule 4, running on my local machine. Let’s get started! ## How do I start? Open your command prompt (Windows) or terminal (Mac/Linux). Run ```bash curl ``` ![Git Bash terminal where running curl prints "try 'curl --help' for more information"](../../assets/blog/the-power-of-curl-1.png) Look for the following output: ```bash curl: try 'curl --help' for more information ``` This is a good indicator that cURL is installed on your local machine. If not, you will need to do a fresh install of cURL. Download cURL [here](https://curl.se/download.html). cURL commands typically use the following syntax: ```bash curl [options] [url] ``` ## Need help? Use the --help option to provide a list of options. ```bash curl --help ``` ![curl --help output listing common options like --data, --header, --output, and --verbose](../../assets/blog/the-power-of-curl-2.png) Use the --manual option to reference the “man” page (or manual). ```bash curl --manual ``` ![curl --manual page showing the cURL name, synopsis, and description of supported protocols](../../assets/blog/the-power-of-curl-3.png) > [!NOTE] > Since I am using a Windows machine, I often use the command line from a Git Bash terminal. ## Demo Time The remainder of this article will focus on HTTP. All demonstrated cURL command examples will be HTTP requests (unless otherwise specified) and invoked against a simple Mule API running on my local machine. The API takes in the HTTP request and provides a breakdown of the request in the response. REQUEST OPTION: To create an HTTP/HTTPS request. Use the --request option. SYNTAX: ```bash curl --request '' ``` > [!NOTE] > URL will include the protocol (http/https). ### **GET** GET requests can be written in two formats: 1) Short-hand SYNTAX: ```bash curl '' ``` EX: ```bash curl 'http://localhost:8081/curl/helloget' ``` 2) Long-hand SYNTAX: ```bash curl --request GET '' ``` EX: ```bash curl --request GET 'http://localhost:8081/curl/helloget' ``` ![curl GET request to /curl/helloget and the JSON response echoing the request method and path](../../assets/blog/the-power-of-curl-4.png) ### **QUERY PARAMS** SYNTAX: ```bash curl --request '?=...&=' ``` EX: ```bash curl --request GET 'http://localhost:8081/curl/helloget?firstname=Daffy&lastname=Duck' ``` ![curl GET with firstname and lastname query params echoed back in the response queryParams](../../assets/blog/the-power-of-curl-5.png) ### **URI PARAMS** SYNTAX: ```bash curl --request '/' ``` EX: ```bash curl --request GET 'http://localhost:8081/curl/helloget/1' ``` ![curl GET to /curl/helloget/1 with the uriParams id of 1 echoed in the response](../../assets/blog/the-power-of-curl-6.png) ### **HEADERS** SYNTAX: ```bash curl --request '' \ --header ':' ``` *Single Header* EX: ```bash curl --request GET 'http://localhost:8081/curl/helloget' \ --header 'Authorization: Bearer xybaneoovganfZbe' ``` ![curl GET with a single Authorization Bearer header echoed in the response headers](../../assets/blog/the-power-of-curl-7.png) *Multiple Headers* EX: ```bash curl --request GET 'http://localhost:8081/curl/helloget' \ --header 'client_id: bond' \ --header 'client_secret: 0007' ``` ![curl GET with client_id and client_secret headers echoed in the response headers](../../assets/blog/the-power-of-curl-8.png) ### **POST** SYNTAX: ```bash curl --request '' \ --header 'Content-Type: ' \ --data-raw '' ``` EX: ```bash curl --request POST 'http://localhost:8081/curl/hellopost' \ --header 'Content-Type: application/json' \ --data-raw '{ "RQ" : {"message" : "hello prostdev readers"}}' ``` ![curl POST to /curl/hellopost with a JSON body echoed back in the response requestBody](../../assets/blog/the-power-of-curl-9.png) ### **Print Output** Interested in saving the output of your API invocation? Use the --output option. SYNTAX: ```bash curl --request --output output.txt ``` EX: ```bash curl --request GET 'http://localhost:8081/curl/helloget'\ --output output.txt ``` ![curl GET request using --output output.txt to save the response to a file](../../assets/blog/the-power-of-curl-10.png) ![The saved output.txt opened in Notepad showing the JSON response from the curl request](../../assets/blog/the-power-of-curl-11.png) ## Lessons Learned - Although quotes are not required around the URL, when sending requests with query parameters make sure you wrap requests around quotes. Without the quotes, I noticed that cURL dropped the query parameters. - Wrap your request URL in “double-quotes” if using command prompt and ‘single-quotes’ if using command line terminal. I found issues with ‘’ and “” between command prompt and command line. - Multi-line in command prompt is different from multi-line on command line. - Add “\” at the end of the line to enter a new line on a multi-line request from the command line. - Add “^” at the end of the line to enter a new line on a multi-line request from the command prompt. ## Conclusion This post invoked an API through HTTP. However, most of the time we invoke APIs over HTTPS. If you are interested in an HTTPS version of this post, please let us know through a comment. If you would like to learn more about cURL, check out the cURL documentation [here](https://curl.se/docs/httpscripting.html). Lastly, I am Whitney Akinola. I’m a fellow MuleSoft Developer and content creator. Feel free to check out my technical content [here](https://www.whitneyakinola.io/blog). --- ## How to check for empty values in an array in DataWeave | Part 4: Arrays Module Source: https://prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module | Published: Feb 23, 2021 | Category: Tutorials I had this use-case where I had to make sure all the values inside an [array](https://en.wikipedia.org/wiki/Array_data_structure) were not empty. By “empty,” I’m referring to an empty string (“”), a null value (*null*), or even an empty array ([]). There were some cases when I would receive a null value instead of an array. The array could only contain strings or nulls, but not objects. However, I’ll be adding tests to check for empty objects as well. > [!NOTE] > The term “null” refers to a null value, whereas “Null” refers to the data type. The same rule applies to string/String, array/Array, and object/Object. I can have an empty array ([]), which is of the type Array, or a string “abc,” which is of the type String. In this series of posts, I explain 6 different approaches to achieve (almost) the same output using different DataWeave functions/operators. As we advance through the posts, the functions will become easier to read, and the logic will have fewer flaws. - [Part 1: Using sizeOf, groupBy, isEmpty, and default](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default) - This solution had some readability issues, the code was showing a warning, and it didn’t fully work for all of our use cases. - [Part 2: Using sizeOf, filter, isEmpty, and default](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default) - This solution still had some readability issues, and it still didn’t fully work for all of our use cases. - [Part 3: Using isEmpty and filter](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter) - This solution worked as expected, but we can do better. ;) - **Part 4: Using the Arrays Module, Pattern Matching, and Function Overloading.** I use the DataWeave Playground throughout these articles. You can follow [this post](https://www.prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image) to set up a local Docker image (no previous Docker experience is needed), or you can also open a new Mule project and use the Transform Message component. > [!NOTE] > You can also use the online DataWeave Playground tool from [this link](http://dwlang.fun/), if available. ## Using the Arrays Module I’ll skip the step-by-step process that we’ve been doing for the past 3 posts to show you 3 different approaches this time. Let’s take a look at the function we created in the last post. ```dataweave %dw 2.0 output application/json fun containsEmptyValues(arr) = if (isEmpty(arr)) true else not isEmpty(arr filter isEmpty($)) --- { nullValue: containsEmptyValues(null), emptyArray: containsEmptyValues([]), arrayWithEmptyString: containsEmptyValues([""]), arrayWithNull: containsEmptyValues([null]), arrayWithEmptyString2: containsEmptyValues(["1", ""]), arrayWithNull2: containsEmptyValues(["1", null]), arrayWithValues: containsEmptyValues(["1", "2"]), arrayWithEmptyObject: containsEmptyValues([{}]), arrayWithEmptyObject2: containsEmptyValues(["1",{}]), arrayWithNonEmptyObject: containsEmptyValues([{a:"b"}]) } ``` ![Part 3 isEmpty/filter function tested against ten cases, all returning the expected results](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module-1.png) We will be using the “[some](https://docs.mulesoft.com/mule-runtime/4.3/dw-arrays-functions-some)” function from the [Arrays module](https://docs.mulesoft.com/mule-runtime/4.3/dw-arrays). To do this, we first need to import the function from the array. ```dataweave %dw 2.0 output application/json import some from dw::core::Arrays fun containsEmptyValues(arr) = if (isEmpty(arr)) true else not isEmpty(arr filter isEmpty($)) --- ``` > [!NOTE] > You can also import all the functions from a module using “import from…” After that, we’ll replace the “else” expression with “arr some isEmpty($)”. This will check if at least one of the items inside the “arr” array is an empty value. If this is the case, it will return a true value. ```dataweave %dw 2.0 output application/json import some from dw::core::Arrays fun containsEmptyValues(arr) = if (isEmpty(arr)) true else arr some isEmpty($) --- ``` ![Function using the Arrays module some function, producing the same correct ten-case results](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module-2.png) As you can see, we receive the same expected output. We didn’t remove the first condition from the code because when we try to do “arr some isEmpty($)” when arr is null or when arr is an empty array ([]), the output will be false instead of the true value that we’re expecting. ![some isEmpty on null and on an empty array both returning false instead of true](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module-3.png) ## Using Pattern Matching (match/case) This alternative uses [pattern matching](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-pattern-matching) with the match/case statements (sort of like the switch/case Java statements). This is what our code would look like: ```dataweave %dw 2.0 output application/json import some from dw::core::Arrays fun containsEmptyValues(arr) = arr match { case [] -> true case a is Array -> a some isEmpty($) else -> isEmpty(arr) } --- ``` ![match/case version of the function with correct results across the ten test cases](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module-4.png) In this case, we’re immediately returning true when we receive an empty array ([]). If our data is not an empty array, then we go to the second “case,” which checks if the parameter is of type Array and runs the logic that includes the “some” function. Finally, if none of those two conditions match (when the value is null), we go to the “else” part and run the “isEmpty” function. ## Using Function Overloading [Function overloading](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-functions#overloading-functions) executes pattern matching logic underneath. If performance is an issue, use this solution instead of the previous one. This is what our code would look like for our two types (Null and Array): ```dataweave %dw 2.0 output application/json import some from dw::core::Arrays fun containsEmptyValues(value: Null) = true // first fun fun containsEmptyValues(arr: Array) = ( // second fun if (arr == []) true // if (isEmpty(arr)) true else arr some isEmpty($) ) --- ``` ![Function-overloading version with separate Null and Array functions, correct ten-case results](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module-5.png) We’re defining the same function twice, both with just one parameter. The difference is that the first one expects a Null data type, and the second one expects Array. By making this distinction, we can immediately return a true value if the parameter is a null value (the first function) and the rest of the logic is handled by the second function. Note that you can define the condition either as “if (arr == []) true” or as “if (isEmpty(arr)) true”, and both will work as expected. You can also use some pattern matching under the second function instead of using an if/else statement. This code would look something like this: ```dataweave fun containsEmptyValues(arr: Array) = arr match { //second fun case [] -> true else -> arr some isEmpty($) } ``` ## Recap Let’s finalize this series with a summary of all the approaches we have learned throughout these posts. **Solution 1: sizeOf, groupBy, isEmpty, default** ```dataweave fun containsEmptyValues(arr) = sizeOf((arr groupBy isEmpty($))."true" default []) > 0 ``` - Not very easy to read (too many parentheses). - DataWeave shows a warning. - Null and empty array return false instead of true. **Solution 2: sizeOf, filter, isEmpty, default** ```dataweave fun containsEmptyValues(arr) = sizeOf(arr default [] filter isEmpty($)) > 0 ``` - Slightly easier to read, but still confusing. - Null and empty array return false instead of true. **Solution 3: isEmpty, filter** ```dataweave fun containsEmptyValues(arr) = if (isEmpty(arr)) true else not isEmpty(arr filter isEmpty($)) ``` - Works as expected, but may not be the best solution for larger payloads. **Solution 4: Arrays Module** ```dataweave import some from dw::core::Arrays fun containsEmptyValues(arr) = if (isEmpty(arr)) true else arr some isEmpty($) ``` - Works as expected, and the performance might be better for larger payloads. **Solution 5: Pattern Matching** ```dataweave import some from dw::core::Arrays fun containsEmptyValues(arr) = arr match { case [] -> true case a is Array -> a some isEmpty($) else -> isEmpty(arr) } ``` - Works as expected and leaves room to implement more “case” statements to handle additional data types or conditions. **Solution 6: Function Overloading** ```dataweave import some from dw::core::Arrays fun containsEmptyValues(value: Null) = true // 1) using if/else fun containsEmptyValues(arr: Array) = ( if (arr == []) true // if (isEmpty(arr)) true else arr some isEmpty($) ) // 2) using match/case // fun containsEmptyValues(arr: Array) = arr match { // case [] -> true // else -> arr some isEmpty($) // } ``` - Works as expected and leaves room to implement more functions to handle additional data types. - Function overloading executes pattern matching logic underneath. If performance is an issue, use this solution instead of solution 5. Oh! And before I forget, there was a **quiz** in the previous post. The question was, “Do the following scripts generate the same output?” ```dataweave 1) not true or true 2) ! true or true ``` The answer is: NO! They do *not* return the same value. The first one returns false, while the second one returns true. Why? This is how these scripts are really run: ```dataweave 1) not (true or true) = not (true) = false 2) (! true) or true = (false) or true = true ``` Mind-blowing, right?! You can read more about this in the [Mule Docs - Logical Operators](https://docs.mulesoft.com/mule-runtime/4.3/dw-operators#logical_operators). And now we’re really done with this series! Which solution would you choose? Don’t miss any DataWeave content! Subscribe to our newsletter or follow us on social media to become a DataWeave champion. :D Keep on practicing! -Alex --- ## How to check for empty values in an array in DataWeave | Part 3: isEmpty, filter Source: https://prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter | Published: Feb 16, 2021 | Category: Tutorials I had this use-case where I had to make sure all the values inside an [array](https://en.wikipedia.org/wiki/Array_data_structure) were not empty. By “empty,” I’m referring to an empty string (“”), a null value (*null*), or even an empty array ([]). There were some cases when I would receive a null value instead of an array. The array could only contain strings or nulls, but not objects. However, I’ll be adding tests to check for empty objects as well. > [!NOTE] > The term “null” refers to a null value, whereas “Null” refers to the data type. The same rule applies to string/String, array/Array, and object/Object. I can have an empty array ([]), which is of the type Array, or a string “abc,” which is of the type String. In this series of posts, I explain 6 different approaches to achieve (almost) the same output using different DataWeave functions/operators. As we advance through the posts, the functions will become easier to read, and the logic will have fewer flaws. - [Part 1: Using sizeOf, groupBy, isEmpty, and default.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default) - [Part 2: Using sizeOf, filter, isEmpty, and default.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default) - **Part 3: Using isEmpty and filter.** - [Part 4: Using the Arrays Module, Pattern Matching, and Function Overloading.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module) I use the DataWeave Playground throughout these articles. You can follow [this post](https://www.prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image) to set up a local Docker image (no previous Docker experience is needed), or you can also open a new Mule project and use the Transform Message component. > [!NOTE] > You can also use the online DataWeave Playground tool from [this link](http://dwlang.fun/), if available. ## Building the solution I’ll create an array to start building and testing this solution: ```dataweave %dw 2.0 output application/json --- ["notEmpty", "", null] ``` **Step 1**: Just as we did in the previous solution, we’ll use the “filter” function, followed by “isEmpty” to retrieve the empty items from the array. ```dataweave ["notEmpty", "", null] filter isEmpty($) ``` ![DataWeave Playground filtering ["notEmpty","",null] by isEmpty, outputting just the "" and null entries.](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter-1.png) **Step 2**: This time, we will not be counting the values with “sizeOf.” Instead, let’s use another “isEmpty” function for this filtered array ([“”, null]). This function can let us know if the array contains any value or an empty array ([]). ```dataweave isEmpty(["notEmpty", "", null] filter isEmpty($)) ``` ![DataWeave Playground wrapping the filtered array in isEmpty, which outputs false.](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter-2.png) With this, we know that the original array ([“notEmpty”, “”, null]) indeed contains empty values (because it’s not empty after filtering it). **Step 3**: Although our goal was to receive a true value if there were empty values, not a false, right? Not to worry! We can simply negate the whole thing with the “[not](https://docs.mulesoft.com/mule-runtime/4.3/dw-operators#logical_operators)” operator. ```dataweave not isEmpty(["notEmpty", "", null] filter isEmpty($)) ``` ![DataWeave Playground negating the result with not isEmpty, which outputs true.](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter-3.png) **Quiz**: Do the following scripts generate the same output? ```dataweave 1) not true or true 2) ! true or true ``` *The answer will be revealed in the next post...Surprise! - There will be more! :)* And now…*drumroll*...we get to the test that broke our two previous solutions…What if we send a null value instead of an array? ```dataweave not isEmpty(null filter isEmpty($)) ``` ![DataWeave Playground running not isEmpty(null filter isEmpty($)) without error, outputting false.](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter-4.png) It didn’t break this time! Woohoo! Since we’re no longer using the “[sizeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sizeof)” function, which only accepts Array, Object, Binary, and String data types (not Null), we don’t have that issue anymore! “[isEmpty](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-isempty)” is cool with Array, String, Null, and Object. Look at that! We’re done! We’re already returning a Boolean, so there’s no need to create a condition. Now we just have to make this a function and call it with our test data. ## Setting up test values To create the function, we can simply define it over the 3 dashes (---) using the “fun” keyword. We can copy and paste our previous code for this new function and replace the [hardcoded](https://en.wikipedia.org/wiki/Hard_coding) array (or null value) with the function’s argument. Like this: ```dataweave %dw 2.0 output application/json fun containsEmptyValues(arr) = not isEmpty(arr filter isEmpty($)) --- ``` To test our use-cases, we’ll be using the same payload we created in our previous solutions: ```dataweave %dw 2.0 output application/json fun containsEmptyValues(arr) = not isEmpty(arr filter isEmpty($)) --- { nullValue: containsEmptyValues(null), emptyArray: containsEmptyValues([]), arrayWithEmptyString: containsEmptyValues([""]), arrayWithNull: containsEmptyValues([null]), arrayWithEmptyString2: containsEmptyValues(["1", ""]), arrayWithNull2: containsEmptyValues(["1", null]), arrayWithValues: containsEmptyValues(["1", "2"]), arrayWithEmptyObject: containsEmptyValues([{}]), arrayWithEmptyObject2: containsEmptyValues(["1",{}]), arrayWithNonEmptyObject: containsEmptyValues([{a:"b"}]) } ``` ![containsEmptyValues function tested against ten cases; nullValue and emptyArray wrongly return false.](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter-5.png) But the “nullValue” and “emptyArray” fields are returning false instead of the true value that we need. **Step 4**: Let’s add a condition to check if the argument (arr) is empty and to immediately return true if that’s the case. Don’t forget to add the “else” with the logic we already had. ```dataweave fun containsEmptyValues(arr) = if (isEmpty(arr)) true else not isEmpty(arr filter isEmpty($)) ``` ![Fixed containsEmptyValues using an if (isEmpty(arr)) guard, now returning true for nullValue and emptyArray.](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter-6.png) Oh, look at that! We did it!! ## Does it work as expected? It works for ALL of our use-cases! But there are more ways to achieve the same output. Have you heard of function overloading or pattern matching using match/case? We can also achieve this solution with these two additional approaches. Plus, we will review how to use the Arrays module! Remember that the answer to the **quiz** will be revealed in the next post. Here is the question: Do the following scripts generate the same output? - not true or true - ! true or true Subscribe now to receive a notification as soon as the next content is published! *Prost!* -Alex --- ## How to check for empty values in an array in DataWeave | Part 2: sizeOf, filter, isEmpty, default Source: https://prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default | Published: Feb 9, 2021 | Category: Tutorials I had this use-case where I had to make sure all the values inside an [array](https://en.wikipedia.org/wiki/Array_data_structure) were not empty. By “empty,” I’m referring to an empty string (“”), a null value (*null*), or even an empty array ([]). There were some cases when I would receive a null value instead of an array. The array could only contain strings or nulls, but not objects. However, I’ll be adding tests to check for empty objects as well. > [!NOTE] > The term “null” refers to a null value, whereas “Null” refers to the data type. The same rule applies to string/String, array/Array, and object/Object. I can have an empty array ([]), which is of the type Array, or a string “abc,” which is of the type String. In this series of posts, I explain 6 different approaches to achieve (almost) the same output using different DataWeave functions/operators. As we advance through the posts, the functions will become easier to read, and the logic will have fewer flaws. - [Part 1: Using sizeOf, groupBy, isEmpty, and default.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default) - **Part 2: Using sizeOf, filter, isEmpty, and default.** - [Part 3: Using isEmpty and filter.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter) - [Part 4: Using the Arrays Module, Pattern Matching, and Function Overloading.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module) I use the DataWeave Playground throughout these articles. You can follow [this post](https://www.prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image) to set up a local Docker image (no previous Docker experience is needed), or you can also open a new Mule project and use the Transform Message component. > [!NOTE] > You can also use the online DataWeave Playground tool from [this link](http://dwlang.fun/), if available. ## Building the solution I’ll create an array to start building and testing this solution: ```dataweave %dw 2.0 output application/json --- ["notEmpty", "", null] ``` **Step 1**: We’ll use the “filter” function, followed by “isEmpty” to retrieve the empty items from the array. ```dataweave ["notEmpty", "", null] filter isEmpty($) ``` ![DataWeave filter isEmpty script returning an array of the empty string and null](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default-1.png) Compared to the solution in the previous post, which was using “groupBy” to retrieve all the empty values, we can get the same result in one step instead of two. **Step 2**: As we did in the previous solution, we need to count how many values are in this array ([“”, null]). We’ll be using the “sizeOf” function as well. ```dataweave sizeOf(["notEmpty", "", null] filter isEmpty($)) ``` ![sizeOf wrapping the filter isEmpty script, outputting 2](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default-2.png) Will you look at that? There’s no green line! In the solution prior, we were getting a warning from DataWeave saying that it was auto-coercing the data type. Well, not this time, DataWeave! Although…We do need to test what would happen if we sent Null instead of Array. Let’s see. ```dataweave sizeOf(null filter isEmpty($)) ``` ![DataWeave error: unable to call sizeOf with a Null argument from filtering null](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default-3.png) And…That’s an error. :( When you try to filter a null value in this case, the result is also null. So when we try to call the “[sizeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sizeof)” function with Null, it fails because it only accepts Array, Object, Binary, and String data types as arguments. How can we fix this? We use the “default” operator. Pretty much like in the previous solution, but this time in a different place. ```dataweave sizeOf(null default [] filter isEmpty($)) ``` This operator will check for any null value and default the value to an empty array ([]). Now the “filter” will receive the empty array and will also output an empty array. The “sizeOf” function does work on empty arrays, and we end up with a zero (which is what we expect). ![sizeOf with null default [] filter isEmpty now returning 0 without error](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default-4.png) Now let’s put back the array we had before, instead of the null value, and continue to the next step. ```dataweave sizeOf(["notEmpty", "", null] default [] filter isEmpty($)) ``` **Step 3**: We want to return a Boolean depending on whether there are empty values in the array or not. Let’s add a condition to return true when there *are* empty values or false if there aren’t any. ```dataweave if ( sizeOf(["notEmpty", "", null] default [] filter isEmpty($)) > 0 ) true else false ``` ![if-condition script checking sizeOf greater than 0, returning true](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default-5.png) We got what we wanted! Now we just have to make this a function and call it with different values to make sure that it works for all our cases. ## Setting up test values To create the function, we can simply define it over the 3 dashes (---) using the “fun” keyword. We can copy and paste our previous code for this new function, and replace the [hardcoded](https://en.wikipedia.org/wiki/Hard_coding) array with the function’s argument. Like this: ```dataweave %dw 2.0 output application/json fun containsEmptyValues(arr) = if ( sizeOf(arr default [] filter isEmpty($)) > 0 ) true else false --- ``` To test our use-cases, we’ll be using the same payload we created in our previous solution: ```dataweave %dw 2.0 output application/json fun containsEmptyValues(arr) = if ( sizeOf(arr default [] filter isEmpty($)) > 0 ) true else false --- { nullValue: containsEmptyValues(null), emptyArray: containsEmptyValues([]), arrayWithEmptyString: containsEmptyValues([""]), arrayWithNull: containsEmptyValues([null]), arrayWithEmptyString2: containsEmptyValues(["1", ""]), arrayWithNull2: containsEmptyValues(["1", null]), arrayWithValues: containsEmptyValues(["1", "2"]), arrayWithEmptyObject: containsEmptyValues([{}]), arrayWithEmptyObject2: containsEmptyValues(["1",{}]), arrayWithNonEmptyObject: containsEmptyValues([{a:"b"}]) } ``` ![containsEmptyValues function tested against ten cases, with true/false results per case](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default-6.png) Note that there are 3 fields at the end to test the behavior of the function with Object. Even though this was not our initial use-case, our “containsEmptyValues” function works with the Object data type as well. Why? Because we’re using the “[isEmpty](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-isempty)” function, which accepts the Array, String, Null, and Object data types. There was a **quiz** in the previous post. The question was, “*What happens when we change the “isEmpty” function to “*[isBlank](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-isblank)*” instead?*” and that same question can also be asked for this solution. The **answer** is that there is an error whenever you try to pass a value that’s not String or Null. In other words, the last 3 fields (arrayWithEmptyObject, arrayWithEmptyObject2, and arrayWithNonEmptyObject) would fail because we’d be sending an object ({}) instead of a string or a null value inside the array. ## Does it work as expected? It works for *most* of our use-cases. However, it’s not the best solution. Here are some of the cons of this function: - It’s slightly easier to read than the previous solution. However, there are still some parentheses that may cause confusion. - We still have the same problem as last time. The null value (null) and the empty array ([]) are returning a false value, but they should be returning a true. And that’s it for the second function! Remember that the following post will explain a more elegant solution, so subscribe now to receive an email as soon as the next part is published! Next time there will be dramatic changes, you’ll see! *Prost!* -Alex --- ## How to check for empty values in an array in DataWeave | Part 1: sizeOf, groupBy, isEmpty, default Source: https://prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default | Published: Feb 2, 2021 | Category: Tutorials I had this use-case where I had to make sure all the values inside an [array](https://en.wikipedia.org/wiki/Array_data_structure) were not empty. By “empty,” I’m referring to an empty string (“”), a null value (*null*), or even an empty array ([]). There were some cases when I would receive a null value instead of an array. The array could only contain strings or nulls, but not objects. However, I’ll be adding tests to check for empty objects as well. > [!NOTE] > The term “null” refers to a null value, whereas “Null” refers to the data type. The same rule applies to string/String, array/Array, and object/Object. I can have an empty array ([]), which is of the type Array, or a string “abc,” which is of the type String. In this series of posts, I explain 6 different approaches to achieve (almost) the same output using different DataWeave functions/operators. As we advance through the posts, the functions will become easier to read, and the logic will have fewer flaws. - **Part 1: Using sizeOf, groupBy, isEmpty, and default.** - [Part 2: Using sizeOf, filter, isEmpty, and default.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-2-sizeof-filter-isempty-default) - [Part 3: Using isEmpty and filter.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-3-isempty-filter) - [Part 4: Using the Arrays Module, Pattern Matching, and Function Overloading.](https://www.prostdev.com/post/how-to-check-for-empty-values-in-an-array-in-dataweave-part-4-arrays-module) I use the DataWeave Playground throughout these articles. You can follow [this post](https://www.prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image) to set up a local Docker image (no previous Docker experience is needed), or you can also open a new Mule project and use the Transform Message component. > [!NOTE] > You can also use the online DataWeave Playground tool from [this link](http://dwlang.fun/), if available. ## Building the solution I’ll create an array to start building and testing this first solution. Something like this: ```dataweave %dw 2.0 output application/json --- ["notEmpty", "", null] ``` **Step 1**: We’ll use the “groupBy” function to separate the non-empty and empty items from the array (using the “isEmpty” function). ```dataweave ["notEmpty", "", null] groupBy isEmpty($) ``` This will give us an object with two fields: one “false” and one “true.” The “false” field contains an array with the values that are not empty ([“notEmpty”]), and the “true” field contains an array with the values that are empty ([“”, null]). ![DataWeave Playground: groupBy isEmpty output grouping the array into false and true keys](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default-1.png) **Step 2**: Since we want to count the number of values that *are* empty, let’s go ahead and extract the “true” field. ```dataweave (["notEmpty", "", null] groupBy isEmpty($))."true" ``` ![DataWeave Playground: selecting the "true" key returns an array of the empty and null values](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default-2.png) **Step 3**: To count how many values are in this array ([“”, null]), let’s use the “sizeOf” function. ```dataweave sizeOf((["notEmpty", "", null] groupBy isEmpty($))."true") ``` ![DataWeave Playground: sizeOf returns 2 with a green type warning underline on the script](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default-3.png) Oh, what’s that green line? Ah…It turns out that DataWeave is assuming that any data type can be returned here. It’s not an error, but it’s a warning. You can get rid of it by explicitly coercing the data inside the “sizeOf” function into Array. ```dataweave sizeOf((["notEmpty", "", null] groupBy isEmpty($))."true" as Array) ``` > [!NOTE] > The “[sizeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sizeof)” function only accepts the Array, Object, Binary, and String data types as arguments. However, if you send a null value instead of an array, this will immediately fail. You can’t coerce Null to Array. ```dataweave sizeOf((null groupBy isEmpty($))."true" as Array) ``` What can we do now? Oh! What about the “default” operator? ```dataweave sizeOf((null groupBy isEmpty($))."true" default []) ``` ![DataWeave Playground: a null input with default [] returns 0 from sizeOf](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default-4.png) That works! Now let’s put back the array we had before, instead of the null value, and continue to the next step. ```dataweave sizeOf((["notEmpty", "", null] groupBy isEmpty($))."true" default []) ``` **Step 4**: We want to return a Boolean depending on whether there are empty values in the array or not. Let’s add a condition to return true when there *are* empty values or false if there aren’t any. ```dataweave if ( sizeOf((["notEmpty", "", null] groupBy isEmpty($))."true" default []) > 0 ) true else false ``` ![DataWeave Playground: an if/else on sizeOf greater than 0 returns the Boolean true](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default-5.png) We got what we wanted! Now we just have to make this a function and call it with different values to make sure that it works for all our cases. ## Setting up test values To create the function, we can simply define it over the 3 dashes (---) using the “fun” keyword. We can copy and paste our previous code for this new function, and replace the [hardcoded](https://en.wikipedia.org/wiki/Hard_coding) array with the function’s argument. Like this: ```dataweave %dw 2.0 output application/json fun containsEmptyValues(arr) = if ( sizeOf((arr groupBy isEmpty($))."true" default []) > 0 ) true else false --- ``` Now we can create the rest of the payload that we’ll be using to test. Let’s create an object that will contain some different examples of the data that the function will receive. Something like this: ```dataweave %dw 2.0 output application/json fun containsEmptyValues(arr) = if ( sizeOf((arr groupBy isEmpty($))."true" default []) > 0 ) true else false --- { nullValue: containsEmptyValues(null), emptyArray: containsEmptyValues([]), arrayWithEmptyString: containsEmptyValues([""]), arrayWithNull: containsEmptyValues([null]), arrayWithEmptyString2: containsEmptyValues(["1", ""]), arrayWithNull2: containsEmptyValues(["1", null]), arrayWithValues: containsEmptyValues(["1", "2"]), arrayWithEmptyObject: containsEmptyValues([{}]), arrayWithEmptyObject2: containsEmptyValues(["1",{}]), arrayWithNonEmptyObject: containsEmptyValues([{a:"b"}]) } ``` ![DataWeave Playground: the containsEmptyValues function tested against many cases returning true or false](../../assets/blog/how-to-check-for-empty-values-in-an-array-in-dataweave-part-1-sizeof-groupby-isempty-default-6.png) Note that there are 3 fields at the end to test the behavior of the function with Object. Even though this was not our initial use-case, our “containsEmptyValues” function works with the Object data type as well. Why? Because we’re using the “[isEmpty](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-isempty)” function, which accepts the Array, String, Null, and Object data types. **Quiz**: What happens when we change the “isEmpty” function to “[isBlank](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-isblank)” instead? *The answer will be revealed in the next post.* ## Does it work as expected? It works for *most* of our use-cases. However, it’s not the best solution. Here are some of the cons of this function: - It’s not very easy to read. There are a lot of parentheses that may confuse the developers that end up maintaining your code. - DataWeave is still showing a warning. Yes, the code runs just fine, and there are no errors, but warnings are there for a reason! - The null value (null) and the empty array ([]) are returning a false value, but they should be returning a true. This is a bug that needs to be fixed. Aaaand…We’re done with the first function! Remember that the following posts will explain more elegant solutions, so subscribe now to receive an email as soon as the next parts are published! Remember that the answer to the quiz, “What happens when we change the isEmpty function to isBlank instead?” will be revealed in the next post. Keep learning! -Alex --- ## Using Salesforce Search in Mule 4 Source: https://prostdev.com/post/using-salesforce-search-in-mule-4 | Published: Jan 26, 2021 | Category: Tutorials *GitHub repository with the Mule project can be found at the end of the post.* One of the most used actions when we work with Salesforce integrations is the use of query. It allows us to pull information from any table, do some subqueries, and pull relationship data. But there’s one action we, as developers, don’t use very often (sometimes we don’t even know this operation is available for us) and it is the Salesforce Object Search Language (**SOSL**). ## SOSL and SOQL Based on the [Force.com Developer documentation](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_sosl_intro.htm): A SOQL query is the equivalent of a SELECT SQL statement and searches the org database. SOSL is a programmatic way of performing a text-based search against the search index. Whether you use SOQL or SOSL depends on whether you know which objects or fields you want to search, plus other considerations. Use SOQL when you know which objects the data resides in, and you want to: - Retrieve data from a single object or from multiple objects that are related to one another. - Count the number of records that meet specified criteria. - Sort results as part of the query. - Retrieve data from number, date, or checkbox fields. Use SOSL when you don’t know which object or field the data resides in, and you want to: - Retrieve data for a specific term that you know exists within a field. Because SOSL can tokenize multiple terms within a field and build a search index from this, SOSL searches are faster and can return more relevant results. - Retrieve multiple objects and fields efficiently where the objects might or might not be related to one another. - Retrieve data for a particular division in an organization using the divisions feature. - Retrieve data that’s in Chinese, Japanese, Korean, or Thai. Morphological tokenization for CJKT terms helps ensure accurate results. Here are some differences between SOSL and SOQL. ![Image taken from Force.com Developer documentation](../../assets/blog/using-salesforce-search-in-mule-4-1.png) You might be asking yourself how this is helpful and how you can use it. Well, let’s think of a scenario. ## Scenario The general idea is to be able to process **User** information coming from any source and use the information to be able to validate if a **Contact** or **Lead** already exists in the platform using a specific external Id field. Based on the result we should be able to update a Contact / Lead or create a brand new Lead record. ![Flowchart deciding whether a user exists as a Contact or Lead, then updating or creating a Lead](../../assets/blog/using-salesforce-search-in-mule-4-2.png) ## Implementation I will create a pretty simple application to demonstrate how we can accomplish this. The Mule application would be created in Mule 4 and I will set a few records in a DataWeave component to simulate the input payload. ## input-data-flow ![input-data-flow: Scheduler, Logger, two Transform Message steps, and a Flow Reference](../../assets/blog/using-salesforce-search-in-mule-4-3.png) This flow contains a scheduler to manually trigger the integration for demonstration purposes. Then we set the incoming payload in a DataWeave component just like this: There’s a variable called “**originalPayload**”, which will be used to filter the information out once we get Salesforce information. In the next DW component (Preparing Search Request) we just convert all the external Id values from the original response to a plain string value concatenated by **OR**, making this an understandable payload value for the Salesforce search. The code looks like this: ```dataweave %dw 2.0 output application/java --- (payload map { ids: "\"" ++ ($.id) ++ "\"" }.ids) joinBy " OR " ``` ## salesforce-search-flow ![salesforce-search-flow: Salesforce Search then Transform Message steps mapping, grouping, and collecting results](../../assets/blog/using-salesforce-search-in-mule-4-4.png) This flow will be in charge of making the search call into Salesforce, grouping the response and creating the variables we need to filter the originalPayload with the existing records. In the Salesforce search, I will pass the next expression: ``` FIND { :ids } IN ALL FIELDS RETURNING Contact(Id,external_id__c,Email), Lead(Id,external_id__c,email) ``` Where the :ids parameter is the previous string we created separating the Ids by OR. In this search we are asking Salesforce to retrieve the records from Contacts and Leads searching in all the fields. After the information is returned, we can tell which fields we need from each object. ![Salesforce Search config with a SOSL FIND string returning Contact and Lead fields, ids set to payload](../../assets/blog/using-salesforce-search-in-mule-4-5.png) "Mapping Search Response" component just creates a map of the Salesforce results (payload.searchRecords). After this, we will group the information by type. We will use this script: ```dataweave %dw 2.0 output application/java --- (payload groupBy ((value, index) -> value."type")) ``` In the same component, I’m creating a variable called **salesforceResponseMap** which contains a key-value map we can access using a value to get the full record. ```dataweave %dw 2.0 output application/java --- { (payload map { (($.external_id__c):$) if $.Id != null }) } ``` “Collect by type” is a different variable that allows us to separate the records from the Contacts and Leads we found and set the Id as the main key in order to be able to filter the data in the next components. At this point we already know which Contacts and Leads have been found. ```dataweave %dw 2.0 output application/java --- { fromContacts: payload.Contact map (salesforceContact, IndexOfContact)->{ (id: salesforceContact.external_id__c) if (salesforceContact.external_id__c != null), }, fromLeads: payload.Lead map (salesforceLead, indexOfLeads)->{ (id: salesforceLead.external_id__c) if (salesforceLead.external_id__c != null), } } ``` ## filter-and-collecting-records ![filter-and-collecting-records flow with Transform Message steps to filter contacts and leads, then a Flow Reference](../../assets/blog/using-salesforce-search-in-mule-4-6.png) This flow will filter the data from the original payload by removing existing contacts from Salesforce and leaving the records that need to be created as Leads. “Filter Contacts / Update Contact” will take any existing records from the groupedObjects.fromContacts variable based on the Id using this script: ```dataweave %dw 2.0 output application/java --- vars.originalPayload filter (not (vars.groupedObjects.fromContacts.id contains ($.id))) ``` We are basically removing the records to an array from another one. In the same component we are doing basically the same but without the not operator so it means we are collecting the information that needs to be updated as Contact and we are able to map the fields we need to update. ```dataweave %dw 2.0 output application/java --- (vars.originalPayload filter ((vars.groupedObjects.fromContacts.id contains ($.id))) map (contact, indexOfContact) -> { Id: vars.salesforceResponseMap[contact.id].Id, FirstName: contact."First Name" }) ``` “Filter Leads / Update Leads” is basically the same but using the Leads group. Finally the remaining component collects the remaining information of records that need to be created as Leads in Salesforce and we can map the information. ## enqueue-batch-jobs The meaning of this job is just to set the payloads for update and create records, the only additional thing on this component is that we are specifying the **sObject** and externald **variables**, so, instead of adding a batch component for each type, dynamically we are dynamically passing the sObject for updates and sObject and externalId for upsert calls. This means we can reuse our batch processes. ![enqueue-batch-jobs flow alternating Transform Message and Flow Reference calls to update and upsert batches](../../assets/blog/using-salesforce-search-in-mule-4-7.png) Finally we can see the **batch processing flow**. One batch will focus in updating the objects and just control the response from Salesforce with a DW component like this: ```dataweave %dw 2.0 output application/json --- payload.items map { id: $.id, success: $.successful, (field: $.errors[0].fields[0]) if $.successful == false, (message: $.errors[0].message) if $.successful == false, (statusCode: $.errors[0].statusCode) if $.successful == false } ``` Basically, we can collect the responses and use them. ![update-salesforce-records-batch job: a Batch Step updating records, mapping the response, and logging](../../assets/blog/using-salesforce-search-in-mule-4-8.png) ![upsert-salesforce-records-batch job: a Batch Step upserting records, a Transform Message, and a Logger](../../assets/blog/using-salesforce-search-in-mule-4-9.png) There are some things to consider when using SOSL over SOQL: One of the advantages of this is that we are able to retrieve multiple objects in a call and we are saving a couple of API calls - This can be used on processes that need just a few records. Massive amounts of data might include some complexity on how we create the SOSL expression, but in the end we can just adjust the limits ([https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_sosl_limits.htm](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_sosl_limits.htm)) in case we need to. Let me know if you think this is helpful and I will be happy to enhance this process as well. ## GitHub repository You can pull the code from this [repository](https://github.com/emoran/yucelmoran-salesforce-search) if you want to see the whole process working. --- ## DataWeave 2.0 core functions cheatsheet Source: https://prostdev.com/post/dataweave-2-0-core-functions-cheatsheet | Published: Jan 19, 2021 | Category: Guides This is a compilation of all the core functions that can be used in DataWeave 2.0 according to [MuleSoft's official documentation](https://docs.mulesoft.com/dataweave/latest/dw-core), separated by input and output. The link to each function's official documentation page is provided in the list. There you can find more details about the functions and examples of how to use them. You can use ctrl+F or cmd+F to search for specific keywords and get to the function you're looking for. > [!IMPORTANT] > This cheatsheet has been last updated for **DataWeave version 2.12** (Mule 4.12). ## Array to Array **Input:** Array | **Output:** Array - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. - [Minus minus (--)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-minusminus) - Removes specified values from an input value. - [distinctBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-distinctby) - Iterates over the input and returns the unique elements in it. - [filter](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-filter) - Iterates over an array and applies an expression that returns matching values. - [find](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-find) - Returns indices of an input that match a specified value. - [flatMap](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-flatmap) - Iterates over each item in an array and flattens the results. - [flatten](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-flatten) - Turns a set of subarrays into a single, flattened array. - [map](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-map) - Iterates over items in an array and outputs the result into a new array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest array according to the provided expression. - [orderBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-orderby) - Reorders the array's elements based on the given criteria. - [reduce](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-reduce) - Applies a reduction expression to the elements in the array. - [unzip](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-unzip) - Groups the values of the given sub-arrays by matching indices (or indexes) and returns new sub-arrays with the matching indices. *Note: performs the opposite of zip.* - [zip](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-zip) - Merges elements from two arrays into an array of arrays. *Note: performs the opposite of unzip.* ## Array to Boolean **Input:** Array | **Output:** Boolean - [contains](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-contains) - Returns true if an input contains a given value, false if not. - [isEmpty](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isempty) - Returns true if the given input value is empty, false if not. - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest boolean according to the provided expression. - [reduce](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-reduce) - Applies a reduction expression to the elements in the array. ## Array to Date **Input:** Array | **Output:** Date - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest date according to the provided expression. ## Array to DateTime **Input:** Array | **Output:** DateTime - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest datetime according to the provided expression. ## Array to LocalDateTime **Input:** Array | **Output:** LocalDateTime - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest localdatetime according to the provided expression. ## Array to LocalTime **Input:** Array | **Output:** LocalTime - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest localtime according to the provided expression. ## Array to Number **Input:** Array | **Output:** Number - [avg](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-avg) - Returns the average of numbers listed in the array. - [indexOf](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-indexof) - Returns the index of the first occurrence of the specified element in this array, or -1 if this list does not contain the element. - [lastIndexOf](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-lastindexof) - Returns the index of the last occurrence of the specified element in a given array, or -1 if the array does not contain the element. - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest number according to the provided expression. - [reduce](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-reduce) - Applies a reduction expression to the elements in the array. - [sizeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sizeof) - Returns the number of elements in an array. - [sum](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sum) - Returns the sum of numeric values in the given array. ## Array to Object **Input:** Array | **Output:** Object - [groupBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-groupby) - Returns an object that groups items from an array based on specified criteria, such as an expression or matching selector. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest object according to the provided expression. - [reduce](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-reduce) - Applies a reduction expression to the elements in the array. ## Array to String **Input:** Array | **Output:** String - [joinBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-joinby) - Merges an array into a string and uses the provided string as a separator between each item in the list. *Note: this is the opposite operation of splitBy.* - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest string according to the provided expression. - [reduce](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-reduce) - Applies a reduction expression to the elements in the array. ## Array to Time **Input:** Array | **Output:** Time - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest time according to the provided expression. ## Array to TimeZone **Input:** Array | **Output:** TimeZone - [max](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-max) - Returns the highest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [maxBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-maxby) - Iterates over the array and returns the highest value of [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) elements from it. - [min](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-min) - Returns the lowest [Comparable](https://docs.mulesoft.com/munit/latest/comparable-matchers-reference) value in the given array. - [minBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-minby) - Iterates over the array and returns the lowest timezone according to the provided expression. ## Array and String to String **Input:** Array, String | **Output:** String - [joinBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-joinby) - Merges an array into a string and uses the provided string as a separator between each item in the list. *Note: this is the opposite operation of splitBy.* ## Date to Boolean **Input:** Date | **Output:** Boolean - [isLeapYear](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isleapyear) - Returns true if it receives a date for a leap year, false. if not. ## Date to Number **Input:** Date | **Output:** Number - [daysBetween](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-daysbetween) - Returns the number of days between two dates. ## Date and LocalTime to LocalDateTime **Input:** Date, LocalTime | **Output:** LocalDateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## Date and Time to DateTime **Input:** Date, Time | **Output:** DateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## Date and TimeZone to DateTime **Input:** Date, TimeZone | **Output:** DateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## DateTime to Boolean **Input:** DateTime | **Output:** Boolean - [isLeapYear](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isleapyear) - Returns true if it receives a date for a leap year, false. if not. ## LocalTime and Date to LocalDateTime **Input:** LocalTime, Date | **Output:** LocalDateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## LocalTime and TimeZone to Time **Input:** LocalTime, TimeZone | **Output:** Time - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## LocalDateTime to Boolean **Input:** LocalDateTime | **Output:** Boolean - [isLeapYear](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isleapyear) - Returns true if it receives a date for a leap year, false. if not. ## LocalDateTime and TimeZone to DateTime **Input:** LocalDateTime, TimeZone | **Output:** DateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## Number to Array **Input:** Number | **Output:** Array - [to](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-to) - Returns a range with the given numbers. ## Number to Boolean **Input:** Number | **Output:** Boolean - [isDecimal](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isdecimal) - Returns true if the given number contains a decimal, otherwise false. - [isEven](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-iseven) - Returns true if the number or numeric result of a mathematical operation is even, false if not. - [isInteger](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isinteger) - Returns true if the given number is an integer, otherwise false. - [isOdd](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isodd) - Returns true if the given number is odd, otherwise false. ## Number to Number **Input:** Number | **Output:** Number - [abs](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-abs) - Returns the absolute value of a number. - [ceil](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-ceil) - Rounds the number up to the nearest whole number. - [floor](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-floor) - Rounds a number down to the nearest whole number. - [mod](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-mod) - Returns the modulo (the remainder after dividing the dividend by the divisor). - [pow](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-pow) - Raises the value of the given base number to the given power. - [randomInt](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-randomint) - Returns a pseudo-random whole number from 0 to the given number. - [round](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-round) - Rounds the number up or down to the nearest number. - [sqrt](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sqrt) - Returns the square root of the given number. ## Number to Range **Input:** Number | **Output:** Range - [to](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-to) - Returns a range with the given numbers. ## Object to Array **Input:** Object | **Output:** Array - [entriesOf](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-entriesof) - Returns an array of key-value pairs that describe the key, value, and any attributes in the input object. - [keysOf](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-keysof) - Returns an array of keys from key-value pairs within the input object. - [namesOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-namesof) - Returns an array of strings with the names of the keys from the given object. - [pluck](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-pluck) - Iterates over an object and returns an array of keys, values, or indices (or indexes) from the object. Useful for mapping an object into an array. *Note: similar to mapObject - mapObject returns an object and pluck returns an array.* - [valuesOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-valuesof) - Returns an array of the values from the given object's key/value pairs. ## Object to Boolean **Input:** Object | **Output:** Boolean - [isEmpty](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isempty) - Returns true if the given input value is empty, false if not. ## Object to Number **Input:** Object | **Output:** Number - [sizeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sizeof) - Returns the number of key/value pairs in an object. ## Object to Object **Input:** Object | **Output:** Object - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. - [Minus minus (--)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-minusminus) - Removes specified values from an input value. - [distinctBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-distinctby) - Iterates over the input and returns the unique elements in it. - [filterObject](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-filterobject) - Iterates a list of key-value pairs in an object and applies an expression that returns only matching objects, filtering out the rest from the output. - [groupBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-groupby) - Groups elements of an object based on criteria that the groupBy uses to iterate over elements in the input. - [mapObject](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-mapobject) - Iterates over the object using a mapper that acts on keys, values, or indices (or indexes) of that object. *Note: similar to pluck - mapObject returns an object and pluck returns an array.* - [orderBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-orderby) - Reorders the object's elements based on the given criteria. ## Object and Array to Object **Input:** Object, Array | **Output:** Object - [Minus minus (--)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-minusminus) - Removes specified values from an input value. ## String to Array **Input:** String | **Output:** Array - [find](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-find) - Returns indices of an input that match a specified value. - [splitBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-splitby) - Splits the given string into an array based on the given separating string. *Note: this is the opposite operation of joinBy.* ## String to Boolean **Input:** String | **Output:** Boolean - [contains](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-contains) - Returns true if an input contains a given value, false if not. - [endsWith](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-endswith) - Returns true if a string ends with a provided substring, false if not. - [evaluateCompatibilityFlag](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-evaluatecompatibilityflag) - Returns the value of the compatibility flag with the specified name. - [isBlank](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isblank) - Returns true if the given string is empty, composed of whitespaces, or null; otherwise false. - [isDecimal](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isdecimal) - Returns true if the given number contains a decimal, otherwise false. *Note: this function is designed for Number instead of String. You will receive a warning from DataWeave, or an error if the provided String contains something that can't be coerced into Number.* - [isEmpty](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-isempty) - Returns true if the given input value is empty, false if not. - [startsWith](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-startswith) - Returns true if the string starts with the given substring, otherwise false. ## String to Number **Input:** String | **Output:** Number - [indexOf](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-indexof) - Returns the index of the first occurrence of the specified string in this string, or -1 if this list does not contain the element. - [lastIndexOf](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-lastindexof) - Returns the index of the last occurrence of the specified element in a given array, or -1 if the array does not contain the element. - [sizeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-sizeof) - Returns the number of characters in a string. ## String to Object **Input:** String | **Output:** Object - [groupBy](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-groupby) - Returns an object that groups characters from a string based on specified criteria, such as an expression or matching selector. ## String to String **Input:** String | **Output:** String - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. - [filter](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-filter) - Iterates over a string and applies an expression that returns matching values. - [lower](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-lower) - Returns the given string in lowercase characters. - [replace (with)](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-replace) - Performs a string replacement from the given substring with the provided string. - [trim](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-trim) - Removes any blank spaces from the beginning and end of the given string. - [upper](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-upper) - Returns the given string in uppercase characters. ## String and Regex to Array **Input:** String, Regex | **Output:** Array - [find](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-find) - Returns indices of an input that match a specified value. - [match](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-match) - Matches the given regular expression from the given string and separates the results into capture groups inside an array. *Note: match will return a full match from the string. For several matches, use scan instead.* - [scan](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-scan) - Returns an array with all of the matches found in the given string, using the given regular expression. *Note: scan will return all of the matches from the string. For a full match, use match instead.* - [splitBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-splitby) - Splits the given string into an array based on the regular expression. *Note: this is the opposite operation of joinBy.* ## String and Regex to Boolean **Input:** String, Regex | **Output:** Boolean - [contains](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-contains) - Returns true if an input contains a given value, false if not. - [matches](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-matches) - Returns true if an expression matches the entire given string, otherwise false. ## String and Regex to String **Input:** String, Regex | **Output:** String - [replace (with)](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-replace) - Performs a string replacement from the substring found with the given regular expression. ## Time and Date to DateTime **Input:** Time, Date | **Output:** DateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## TimeZone and Date to DateTime **Input:** TimeZone, Date | **Output:** DateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## TimeZone and LocalDateTime to DateTime **Input:** TimeZone, LocalDateTime | **Output:** DateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## TimeZone and LocalTime to DateTime **Input:** TimeZone, LocalTime | **Output:** DateTime - [Plus plus (++)](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-plusplus) - Concatenates two values. ## Others Some of these next functions don't have any input parameters or don't return any output. Some accept more than two arguments. - [log](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-log) - Without changing the value of the input, log returns the input as a system log. So this makes it very simple to debug your code, because any expression or subexpression can be wrapped with log and the result will be printed out without modifying the result of the expression. The output is going to be printed in application/dw format. - [logDebug](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-logdebug) - Helper function that logs messages at Debug level. - [logError](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-logerror) - Helper function that logs messages at Error level. - [logInfo](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-loginfo) - Helper function that logs messages at Info level. - [logWarn](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-logwarn) - Helper function that logs messages at Warn level. - [logWith](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-logwith) - Without changing the value of the input, logWith returns the input as a system log at the specified level. - [match / case](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-pattern-matching) - Pattern matching can be achieved by using a combination of match and case. Can be applied to literal values, expressions, data types, or regular expressions. - [now](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-now) - Returns current date and time in a DateTime. - [onNull](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-onnull) - Executes a callback function if the preceding expression returns a null value and then replaces the null value with the result of the callback. - [random](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-random) - Returns a pseudo-random number between 0.0 and 1.0. - [read](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-read) - Reads a string or binary and returns parsed content (for example: application/json, application/xml, application/csv, etc.). - [readUrl](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-readurl) - Reads a URL, including a classpath-based URL, and returns parsed content. *Note: similar to the read function.* - [then](https://docs.mulesoft.com/dataweave/latest/dw-core-functions-then) - Works as a pipe that passes the value returned from the preceding expression to the next (a callback) only if the value returned by the preceding expression is not null. - [typeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-typeof) - Returns the type of the given value. - [uuid](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-uuid) - Returns a v4 UUID using random numbers as the source. - [with](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-with) - Helper function used with [replace](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-replace), [update](https://docs.mulesoft.com/mule-runtime/4.3/dw-values-functions-update), or [mask](https://docs.mulesoft.com/mule-runtime/4.2/dw-values-functions-mask). - [write](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-write) - Writes a value as a string or binary in a supported format. - [xsiType](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-xsitype) - Creates a xsi:type type attribute. *Note: useful for XML data types.* - Additional to the plus plus (++) function, you can also concatenate objects by using the Object Destructor {( )}. Learn more about this approach [here](https://www.prostdev.com/post/combining-objects-concatenation-in-dw-2-0) or learn how to use $( ) to concatenate strings [here](https://blogs.mulesoft.com/dev-guides/how-to-tutorials/review-concatenation-functions-dataweave/). ## Null-safe functions Here is a list of the functions that don't need an additional null-checker implemented. In other words, they accept null values and won't throw any error. - [distinctBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-distinctby) - [filter](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-filter) - [filterObject](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-filterobject) - [flatMap](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-flatmap) - [flatten](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-flatten) - [groupBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-groupby) - [isBlank](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-isblank) - [isEmpty](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-isempty) - [log](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-log) - [lower](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-lower) - [map](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-map) - [mapObject](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-mapobject) - [orderBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-orderby) - [pluck](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-pluck) - [trim](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-trim) - [typeOf](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-typeof) - [upper](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-upper) If you have any suggestions, comments, or noticed that something's wrong with the provided information; please reach out with a comment on this post or contact us [here](https://www.prostdev.com/contact). I hope this was useful for you! -Alex --- ## Understanding APIs (Part 6): What are HTTP Status Codes? Source: https://prostdev.com/post/understanding-apis-part-6-what-are-http-status-codes | Published: Jan 12, 2021 | Category: Guides We’re almost done learning the basic concepts we need to understand APIs better. Let’s recap what we have learned so far: - [Understanding APIs (Part 1): What is an API?](https://www.prostdev.com/post/understanding-apis-part-1-what-is-an-api) - We defined the initial diagram. - [Understanding APIs (Part 2): API Analogies and Examples](https://www.prostdev.com/post/understanding-apis-part-2-api-analogies-and-examples) - We talked about the Restaurant and Calculator analogies and defined the Human Resources API example. - [Understanding APIs (Part 3): What are HTTP Methods?](https://www.prostdev.com/post/understanding-apis-part-3-what-are-http-methods) - We learned the 5 most popular HTTP methods: GET, POST, PUT, PATCH, and DELETE. - [Understanding APIs (Part 4): What is a URI?](https://www.prostdev.com/post/understanding-apis-part-4-what-is-a-uri) - We learned that a URL is just a form of URI. We broke down a URI into protocol, host, and path. - [Understanding APIs (Part 5): Intro to Postman and Query Parameters](https://www.prostdev.com/post/understanding-apis-part-5-intro-to-postman-and-query-parameters) - We learned how to use Postman and what Query Parameters are. ![Diagram defined in Part 1, which contains the 4 aspects of the API + the Implementation.](../../assets/blog/understanding-apis-part-6-what-are-http-status-codes-1.png) The last 3 posts mentioned components sent in the Request that tell the API what data we want to receive. This time, we’ll be talking about a part of the Response: the HTTP Status Codes. ## What are HTTP Status Codes? HTTP Status Codes are numbers returned by the API which are included in the Response. They indicate if the operation you wanted to perform was a success or not, and they describe the kind of success/failure that happened. It’s basically a conversation with the server where it tells you, “here is the data you requested,” “your Request is not correct; please take a look,” “I’m having internet issues; please try again later.” There are a lot of Status Codes reserved for certain API Responses, but we will be looking at the most popular ones: 200, 201, 202, 204, 400, 401, 403, 404, and 500. You can find a complete list [here](https://www.restapitutorial.com/httpstatuscodes.html) with their definitions. > [!NOTE] > Any developer, API designer, or architect can choose the status codes they want the API to return. Some APIs only return 200, while others only return 200, 400, and 500. However, I will follow the best practices shown by [REST API Tutorial](https://www.restapitutorial.com/httpstatuscodes.html). **200 OK** This code is returned when the Request is a success, and there are no errors whatsoever. It’s mainly returned when using a GET or a POST. The Response will contain the data requested by the GET or the data created/updated by the POST. **201 Created** This code means that the Request is a success and that a new resource was created. For example, a new employee was added to the company list successfully. The Response usually contains the data that was just created. It’s mainly returned when using a POST or a PUT. **202 Accepted** This code is returned when some processing is still pending, but it hasn’t been completed yet. This code is especially useful when the back-end processing of the information will take too long to complete, so the user receives a confirmation of the Request along with a pointer to monitor the processing status. Note that this doesn’t mean that the Request was a success. It just means that the server received it, but it can still fail its processing. **204 No Content** This code means that the Request is a success, but there is no data to be returned. For example, when deleting a resource, nothing is returned because it no longer exists. It’s mainly returned when using DELETE. **400 Bad Request** The Request contains a syntax error that could not be understood by the server. For example, if your Request needs to have a field called “FullName” and you don’t send it to the API, it will return this error. **401 Unauthorized** This code is returned when the Request must contain some authorization information to use the API, like a username and a password. However, this data was not included or was wrong. **403 Forbidden** This code is returned when the API refuses to return some information. The Request and the authorization are ok, but for some other reason the data can’t be returned. For example, when you have a session opened for too long, it expires, and when you click a button or request additional info, you can’t anymore. **404 Not Found** This code is one of the most commonly seen from a web browser. It means the server didn’t find what you were looking for, mostly because it doesn’t exist. For example, if you try to access [http://google.com/test](http://google.com/test), you will get this error because the “test” path doesn’t exist. **500 Internal Server Error** This code is returned when there is an error inside the API. The Request was perfectly fine, and there were no errors when calling the server, but then something failed inside the Implementation. This is the most generic error that can be used when the server fails. ## HTTP Status Codes in Postman Go to Postman and send a new GET Request to “google.com”. You will receive an HTML response, and you will be able to see the HTTP Status Code right over the Response. > [!NOTE] > If you’re new to Postman, you should check out the previous post of this series: "[Understanding APIs (Part 5): Intro to Postman and Query Parameters](https://www.prostdev.com/post/understanding-apis-part-5-intro-to-postman-and-query-parameters)." ![Postman GET request to google.com showing the 200 OK status code highlighted above the response.](../../assets/blog/understanding-apis-part-6-what-are-http-status-codes-2.png) **Quiz**: What HTTP Status Code do you receive when sending a GET Request to “https://api.twitter.com/2/tweets/1234” from Postman? *The answer will be revealed in the next post.* ## Recap - **HTTP Status Codes** are numbers returned in the API’s Response to indicate if the Request was successfully processed or if there was an error. - **200 OK** means the Request is a success and returns the requested information. - **201 Created** means there is a new resource added, and the Response contains this new information. - **202 Accepted** means the Request is accepted but is still processing. - **204 No Content** means the Request is successful, but no data can be returned. - **400 Bad Request** means there is something wrong with your Request. - **401 Unauthorized** means you didn’t send the proper credentials to access the URI or perform the Operation. - **403 Forbidden** means the information can’t be returned for some reason, like a time limit. - **404 Not Found** means the resource/URI you are trying to access does not exist. - **500 Internal Server Error** means something went wrong inside the Implementation or within the server. I need to tell you this joke because it’s too good to miss. A parent tells the teenager to clean their room. The teenager responds, “202,” and the parent goes away. Some hours later, the parent comes back to see that the room is still dirty and asks why it isn’t clean yet. The teenager answers, “I said 202, not 200.” Sorry. [#GeekyHumor](https://www.prostdev.com/blog/hashtags/GeekyHumor) *Prost!* -Alex --- ## Why should an editor review your blog posts? Source: https://prostdev.com/post/why-should-an-editor-review-your-blog-posts | Published: Jan 5, 2021 | Category: Opinion Having a suitable format for your article (using bold, headers, subtitles, bullet points) is necessary so your readers don’t get lost in-between paragraphs. We already discussed this point in “Tip [#4](https://www.prostdev.com/blog/hashtags/4) - Develop the article” from a previous post - “[7 tips to start writing your technical blog post](https://www.prostdev.com/post/7-tips-to-start-writing-your-technical-blog-post).” But what about grammar? What about consistency? Sometimes we think we’re clear when we explain something because it makes sense in our heads. What if someone else reads the same text but understands it completely differently? This post will give you my perspective on why having editors review your content is the best way to bullet-proof your articles from possible misunderstandings. ## Create a review process for yourself If you want to create quality content for your readers, you should have a review process in place. For those of you who are software developers, you (should) probably have a Code Review process for when you create new code. You may create unit tests for your code or maybe some functional testing. Then your code is ready to be uploaded to a Pull Request, where a technical leader (or anyone else) will take a look at your code to make sure that you’re not making mistakes. Why shouldn’t you be the one approving your own Pull Request? Because you may miss things that other people might notice. Well, the same thing happens when you create content. It doesn’t have to be a long, bureaucratic process. You can at least use an external tool to correct the basic grammar, or you can share your document with another person before publishing to check that what you wrote makes sense to someone else. The best quality, though, comes from a professional editor. ## Using an external tool to review your grammar is not 100% accurate Using software to check your grammar doesn’t necessarily mean that the grammar will be perfect. Software makes mistakes! [Grammarly](https://www.grammarly.com/), [Google Docs](https://docs.google.com/), or [Microsoft Word](https://www.microsoft.com/en-us/microsoft-365/word) are great tools for editing, but they were programmed by humans (who also make mistakes). I normally create all my posts in Google Docs and double-check that my writing’s basic syntax is correct. If I get a warning or an error there, it’s *almost* always giving me the right suggestion. But remember, although these tools are great places to start editing, they aren’t foolproof. ## So, why should you consider consulting with a professional editor? External software may show you the misspelled words or the grammatically correct way to write a sentence. But it won’t tell you when you’re using a word too often or when a sentence “sounds weird” in a casual setting. Here are some of the advantages that I’ve experienced after working with editors for my content: - **Spelling.** Yes, the external software may get almost all of the misspelled words, but trust me, not all of them. - **Over-using commas.** It turns out that I use a lot of unnecessary commas when writing! None of those external tools have ever caught this. - **Misused punctuation.** I consider myself to have an excellent grasp on how to use almost all punctuation marks. Well, I definitely don’t know how to use *all* of them, and I get confused sometimes (*do you know the difference between an em dash and a hyphen without googling? ‘Cause I certainly don’t*). - **Keeping consistency.** Just the other day, I was writing a post and used “Google” every time I mentioned it…except once. Only one single time, I wrote “google,” and I didn’t notice (of course, my editor did!). Staying consistent throughout your article is important, so you don’t confuse your readers. - **Using synonyms.** Sometimes I think too fast, and I end up writing the same word 10 different times in the same paragraph. When you read that, it sounds way too repetitive! Well, your editor may know some synonyms to refer to the same thing without the paragraph sounding awkward. - **Re-writing sentences.** As a native Spanish speaker, my train of thought is normally in Spanish. There are some times when I *think* in English because I’m writing in English. But other times, I think first in Spanish, and then the translation is not as good as I had hoped for (like using Google Translate, LOL). Having an English-speaker editor helps a lot because the same sentence that makes sense to me (because it’s translated from Spanish) may sound weird to them, so they reformulate it. My documents sometimes end up with more than 50 suggestions/comments from my editors! And there I was, thinking that what I wrote was perfect and understandable to everyone. I hope this post was helpful to you. Of course, this is just my point of view on creating content, but I strongly recommend having someone else review your articles for higher quality posts. :) *Prost!* -Alex --- ## Implementing Pagination with MuleSoft Source: https://prostdev.com/post/implementing-pagination-with-mulesoft | Published: Dec 15, 2020 | Category: Tutorials ## Summary There could be a situation, in which we are consuming an API and the result is returned in multiple pages. Your business case needs the Mule Application to have a capability to process all results from all pages and return the response as per specification. Therefore, we are presented with a business problem to resolve – how to process n number of pages and return the combined response back to the calling client. As per design your backend API only returns x number of records per page and depending on the total number of records it could be n number of pages. So, as a consumer of that backend system and gateway to the client, you are responsible to maintain pagination. ## Solution Approach Let’s talk about how we resolve this problem effectively using MuleSoft. One simple solution to this problem is implementing a recursive flow that calls the backend system API n number of times until all records are fetched and no new pages exist. Pros: Main advantage of this approach is the simplicity and also that it’s easier to implement. Cons: - Synchronous approach. Hence, the next page cannot be fetched until the processing of prior pages are completed. - Response would be slow. - Main disadvantage is, if we have many records and if you need to iterate too many times, Mule Runtime might crash with “Too many child contexts” error. An alternative solution to this problem is to implement this in a more complex way and asynchronous processing could be a saviour here. To demonstrate this approach, we will be calling the GitHub API to return all the repositories under a specific user/organization. There are a few steps of authentication before we can call the GitHub API to get all the results. ## Authentication for GitHub API There are two ways to authenticate through the GitHub REST API. 1) Basic authentication ```bash $ curl -u "username" https://api.github.com ``` 2) OAuth2 token (sent in a header) ```bash $ curl -H "Authorization: token OAUTH-TOKEN" https://api.github.com ``` The recommended way of authentication is OAuth tokens using Authorization Header. As a part of this implementation, we will be doing authorization as OAuth using a Personal Access Token. Personal Access Tokens can be generated for each user and can be sent as Authorization header. If you want to know more details about how to create a Personal Access Token for your GitHub account, you can follow [this link](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token). ## Implementation Main implementation involves two steps: Step 1: Initial step is to get the results for the first page. This is handled by the GetFirstPageResponse sub-flow which essentially calls the GitHub API with page number as 1. Step 2: Once we have page 1 result, we will call ProcessPageResults sub-flow to fetch subsequent pages and appending the results of all pages. ## Step 1: Getting First Page from GitHub API Getting the first page is straight forward. We set up essential variables like pageNumber which will facilitate calling the GitHub API. Then, we invoke the GitHub API to get the results of the first page. For more details about how to call the GitHub API, you can follow the section “Invoking GitHub API.” Next step is storing this result in an Object Store. We store the results of first page in Object Store to smoothly append the results of all pages at the end of the process which we will be seeing in a moment. ![GetFirstPageResponse sub-flow setting pageToBeVisited, calling InvokeGithubAPI and storing the result](../../assets/blog/implementing-pagination-with-mulesoft-1.png) ## Step 2: Processing Subsequent Pages From the previous section, we have fetched the first page of GitHub API, now it’s our turn to process the results from other pages. In this section, we will be discussing about processing of subsequent pages asynchronously. ![ProcessPageResults sub-flow with a For Each that stores status, sets VM payload and publishes](../../assets/blog/implementing-pagination-with-mulesoft-2.png) Once we have the first page, we will examine the response headers. Usually, in the response header, GitHub indicates the information about pagination. GitHub indicates the following information in the header called **link**. - **Prev**: contains the link to the previous page. - **Next**: contains the link to the next page. - **First**: contains the link to the first page. - **Last**: contains the link to the last page. Our concern is all about the last page link information. We will be extracting page numbers from this link type. That would indicate how many more pages we still need to process. For example: If the page number of the last link says 35, then we have 35 - 1 = 34 pages to be processed (since the first page has already been processed). After getting the information about how many more pages we still have to process, we can make up an array of page numbers which are still required to process. ```dataweave %dw 2.0 output application/json var linkHeader = attributes.headers.link var linkHeaderArr = (linkHeader splitBy ",") fun createLinkHeaderDetails() = { linkHeaderDetails: linkHeaderArr map ( createLinkHeaderObject($) ) } fun createLinkHeaderObject(linkHeaderObj) = { "pageNumber": (((((((linkHeaderObj splitBy ";") map (trim($)))[0]) splitBy "&")[1]) splitBy "=")[1])[0], "linkType": readString(((((linkHeaderObj splitBy ";") map (trim($)))[1]) splitBy "=")[1]) } fun readString(readString) = read(readString,"application/json") fun getLastPageNumber() = createLinkHeaderDetails().linkHeaderDetails filter ($.linkType == "last") --- (1 to (getLastPageNumber().pageNumber[0] - 1)) map { pageNumber: ($ + 1) } ``` Next step is: for each page number in a page number array, we are setting up the following: - We will store a flag in Object Store for each specific page number that indicates the processing status. Initially it will be false. - We will be setting up a payload that contains the page number we want to fetch. This is done using a simple DataWeave script to set up the payload for the VM endpoint flow. ```dataweave %dw 2.0 output application/java --- { pageNumber: payload.pageNumber } ``` Then, we will be publishing this payload to a VM endpoint which is exposed by another flow that essentially fetches the results from the GitHub API. This will be discussed in the section “Retrieve Details from GitHub API.” This approach makes this process asynchronous since it won’t wait for the VM endpoint to come back with a response. The VM Publish endpoint works like Fire and Forget, so, the Mule thread won’t wait for this process to be completed. That’s why we call another sub flow named WaitUntilAllPagesAreProcessed outside of the For Each scope which is responsible for checking whether all pages are completely processed or not. This flow is not going to end until all pages are processed. ![WaitUntilAllPagesAreProcessed sub-flow with a Choice branching on whether all pages are processed](../../assets/blog/implementing-pagination-with-mulesoft-3.png) The process to know if all pages have been processed is determined by checking the entry in Object Store for each page number that we set before publishing to the VM endpoint. If the page results are processed, this entry is supposed to be updated to true in Object Store which is part of discussion in the “Retrieve Details from GitHub API” section. We can check for the Object Store entries for all the pages with a simple DataWeave script. ```dataweave %dw 2.0 output application/json fun checkForCompletion(payload) = not sizeOf ((payload pluck $) default [] filter ($ == false)) >0 --- checkForCompletion(payload) ``` If the processing flag of any page is still false, we will impose a small delay with a DataWeave script and then call WaitUntilAllPagesAreProcessed recursively until all pages processing is completed. This delay is imposed to avoid having Mule to crash because of too many recursive calls to the flow. This delay can be configured from configuration parameters and we will keep it to a minimum number (i.e., 1000 ms) which will allow the processing of the pages to be completed and avoid too many calls to the same flow. If the processing of all pages is complete, we fetch the final response from Object Store entry and transform it as per our requirements to form a final response to client. ## Retrieve Details from GitHub API In this section we will be talking more about the flow which is responsible for fetching details from GitHub API and processing the results. We will be building this flow in a way so that it can be called asynchronously. One way to achieve this is to implement a VM Listener endpoint so that we can call this flow asynchronously from other flows. For this flow to work, we only need the page number (which needs to be fetched) to be passed as a payload. ![GetRepoDetailsAsync flow with a VM Listener setting variables and calling InvokeGithubAPI and Append flows](../../assets/blog/implementing-pagination-with-mulesoft-4.png) Let’s discuss this flow in couple of steps: Step 1: We will be retrieving details from the Object Store regarding the GitHub API call, like page size, which is required to tell the GitHub API how many records it’s going to send as a response for the page. Step 2: We will be setting up two variables named pageToBeVisited and pageSize. First variable will get the value from the payload that was passed to the VM endpoint which indicates the specific page number that we are going to fetch while later suggests the number of records for that page to be returned which we can get from Step 1. Step 3: We will call the InvokeGitHubAPI flow that essentially handles the complexity of calling the GitHub API and returning the results. This will be discussed in the following section, “Invoking GitHub API.” Step 4: Next step is appending the result of the current page with previous pages. That can be done by retrieving the result from the Object Store key entry (which is initially created when we fetched the result of Page 1, appending the current page result and storing it back to the same Object Store key. ![AppendGithubResults sub-flow with a Try retrieving the response, then appending and storing it](../../assets/blog/implementing-pagination-with-mulesoft-5.png) Step 5: Finally, we need to update the processing status to true in the Object Store key created for that specific page (discussed in “Step 2: Processing Subsequent Pages"). This indicates that the processing for this specific page is completed. ![UpdateProcessingStatus sub-flow with a Try retrieving the entry then storing the updated status](../../assets/blog/implementing-pagination-with-mulesoft-6.png) ## Invoking GitHub API GitHub REST API offers many functionalities - from getting details of repositories to push/pull the commits. You can read more about the GitHub API here to know what it has to offer. At the time of the writing this article, GitHub offered v3 of its implementation. By default, all requests to [https://api.github.com](https://api.github.com/) receive the v3 [version](https://docs.github.com/en/free-pro-team@latest/developers/overview/about-githubs-apis) of the REST API. However, you can also request a previous version if you need. For our implementation, we will be using the latest version. All API access is over HTTPS and accessed from [https://api.github.com](https://api.github.com/). All data is sent and received as JSON. There are many resource endpoints available but for this implementation, we will only be focusing on GET /user/repos resource to get repositories that the authenticated user has explicit permission (:read, :write, or :admin) to access. The authenticated user can get information on repositories they own, repositories where they are a collaborator, and repositories that they can access through an organization membership. Since this endpoint accepts GET HTTP method, it does not need request payload rather it accepts few query parameters and response comes in application/json format. You will find more details about this endpoint [here](https://docs.github.com/en/free-pro-team@latest/rest/reference/repos#list-repositories-for-the-authenticated-user). To implement this in Mule, we will be building a flow in the following steps: ![InvokeGithubAPI sub-flow setting the authToken variable and making an HTTP Request to GitHub](../../assets/blog/implementing-pagination-with-mulesoft-7.png) Step 1: We setup a variable named authToken that gets the encrypted token from configuration file appended by string “token”. For more details about how to obtain this token, please see section “Authentication for GitHub API.” Step 2: Next step is making a HTTP request to the GitHub API endpoint we discussed earlier. There are a couple of headers and query parameters which are required. **Headers**: Authorization: This header will be used to authenticate the user with the GitHub API. We will be using the value from variable authToken that we created in Step 1. There are number of ways to authorize any user via GitHub API to access protected resources. For this implementation, we have used OAuth with Personal Access Token. The details to implement this in GitHub can be found in section “Authentication for GitHub API.” **Query Parameters**: per_page: This query parameter indicates how many records we want to fetch per page. Default is 30 records per page. However, for some endpoints, we can go up to 100 records per page. If you have more items than the ones you have requested using this query parameter, your results will be paginated. For more information about how pagination works with GitHub API, you can click [here](https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#pagination). page: Page number that needs to fetch. ## Conclusion Thank you for taking the time to read this article. We have learnt how to effectively and efficiently retrieve large datasets by iterating over a number of pages returned from the backend API. This implementation solely focuses on the GitHub REST API for demonstration, but I believe it can be tailored to your technical needs. Here are few references that may help you to get started- [https://docs.github.com/en/free-pro-team@latest/rest/reference](https://docs.github.com/en/free-pro-team@latest/rest/reference) [https://docs.mulesoft.com/vm-connector/2.0/](https://docs.mulesoft.com/vm-connector/2.0/) [https://docs.mulesoft.com/vm-connector/2.0/vm-examples](https://docs.mulesoft.com/vm-connector/2.0/vm-examples) --- ## Understanding APIs (Part 5): Intro to Postman and Query Parameters Source: https://prostdev.com/post/understanding-apis-part-5-intro-to-postman-and-query-parameters | Published: Dec 8, 2020 | Category: Tutorials Part 5 already! What a trip it’s been. :) Let’s recap what we have learned so far: - [Understanding APIs (Part 1): What is an API?](https://www.prostdev.com/post/understanding-apis-part-1-what-is-an-api) - We defined the initial diagram. - [Understanding APIs (Part 2): API Analogies and Examples](https://www.prostdev.com/post/understanding-apis-part-2-api-analogies-and-examples) - We talked about the Restaurant and Calculator analogies and looked at the Human Resources API example. - [Understanding APIs (Part 3): What are HTTP Methods?](https://www.prostdev.com/post/understanding-apis-part-3-what-are-http-methods) - We learned the 5 most popular HTTP methods: GET, POST, PUT, PATCH, and DELETE. - [Understanding APIs (Part 4): What is a URI?](https://www.prostdev.com/post/understanding-apis-part-4-what-is-a-uri) - We learned that a URL is just a form of URI. We broke down a URI into protocol (HTTP/HTTPS), host, and path. ![Diagram defined in Part 1, which contains the 4 aspects of the API + the Implementation.](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-1.png) Continuing with the last topic, URIs, we will now get started using Postman and learning what Query Parameters are. ## Getting started with Postman You can download Postman for free from their [website](https://www.postman.com/downloads/). Once you have it, open it, and it will look something like this: ![Postman app home screen with an empty Collections sidebar and the workspace launchpad](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-2.png) Now let’s click on the plus (+) button as indicated below. ![Postman toolbar with the plus button for a new request highlighted by a red box](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-3.png) This will open a new Untitled Request. Let’s paste the following URI into the text box that says “Enter request URL” - ``` https://official-joke-api.appspot.com/random_joke ``` Now click on “Send.” You should be able to see a formatted Response similar to this: ![Postman GET request to the Joke API showing the HTTP method, URI, and JSON response labeled](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-4.png) > [!NOTE] > If the Response appears at the bottom of the screen and not on the right, you can change Postman’s layout by going to Postman preferences (or settings) > General tab > Check that the User Interface is set as Two-pane view. Congratulations! You just made a call to the [Joke API](https://github.com/15Dkatz/official_joke_api) using Postman. ## What are Query Parameters? Query Parameters (also known as Query Params), which are appended to the URI, are fields that are used to send additional information or data to the API. They are usually used to filter or search for data. For example, if you want to search for “cats” from Google, you can simply use this URI: [https://www.google.com/search?q=cats](https://www.google.com/search?q=cats). Let’s break down this URL like we did in the last post. ![Table breaking the Google search URL into protocol, host, path /search, and query param q=cats](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-5.png) In this case, there’s only one Query Param, which is named “q,” and the value we’re sending is “cats.” For Google, when you call the path “/search” and attach the “q” Query Parameter, it knows that you want to perform a Google search with the word “cats.” So what are the elements of a Query Param? - **(?)** - We can identify the Query Parameters in a URI because they follow the question mark (?). - **(=)** - The Query Param name will be located before the equal sign (=), and the value will go after it. In this case, the value “cats” is being assigned to the Query Param called “q.” - **(&)** - If there were more Query Params in the same URI, they would be attached using the ampersand character (&). E.g. ``` myhost.com/mypath?queryparam1=cats&queryparam2=dogs&queryparam3=squirrels ``` ## Example: Expedia Let’s now get into a more complex example. In your browser, go to [Expedia.com](https://www.expedia.com/) and search for a hotel. ![Expedia search form with Toronto as destination and Nov 23–24 check-in and check-out dates](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-6.png) After you click “Search,” take a look at the URL in your browser. ![Browser address bar showing the long Expedia Hotel-Search URL full of encoded query parameters](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-7.png) For me, it looks like this: ``` https://www.expedia.com/Hotel-Search?adults=2&d1=2020-11-23&d2=2020-11-24&destination=Toronto%20%28and%20vicinity%29%2C%20Ontario%2C%20Canada&endDate=2020-11-24®ionId=178314&rooms=1&semdtl=&sort=RECOMMENDED&startDate=2020-11-23&theme=&useRewards=false&userIntent ``` Let’s put this URI in Postman and see what happens. ![Postman Query Params table listing the Expedia parameters such as adults, dates, and destination](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-8.png) A bunch of Query Params just appeared! Postman is smart enough to know that everything that comes after the question mark (?) are Query Parameters. They are shown in this neat way to read/modify/add/delete them more easily. If you click “Send,” you will see some of the HTML code behind the Expedia website. This is the data that your browser sees and then translates into visuals for you. ![Postman showing the Expedia response body as raw HTML markup after sending the request](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-9.png) Take a look at the “destination” Query Param. Its value will look something like this: ``` Toronto%20%28and%20vicinity%29%2C%20Ontario%2C%20Canada ``` Notice how there are some numbers after the percentages (%) like “%20”, “%28”, “%29”. This is because the URIs can’t contain some characters such as a space, parentheses or commas. So such characters are translated into these “%20” encodings. Here are some examples: ![W3Schools.com - HTML URL Encoding Reference.](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-10.png) So, if we translate “Toronto%20%28and%20vicinity%29%2C%20Ontario%2C%20Canada” back into the original string, this is what it says: “Toronto (and vicinity), Ontario, Canada”. What would happen if we paste the original string directly into the “destination” Query Param in Postman? Let’s do it and resend the Request to see if it works. ![Postman destination param set to the plain text "Toronto (and vicinity), Ontario, Canada" working fine](../../assets/blog/understanding-apis-part-5-intro-to-postman-and-query-parameters-11.png) It worked! Postman is smart enough to know that these characters will need to be translated into the ASCII Encoding and does it in the background without us noticing. If you try to call this URL from your browser, it will also work, but your browser will show you the translated URL just as before. Notice how all these Query Params are information that the backend will use to filter the results that it will show you. For example, adults=2, rooms=1, endDate=2020-11-24, startDate=2020-11-23. With these Params, the server knows not to show you hotels that won’t be available for two adults or that won’t be available from November 23 to November 24. ## Recap - Postman is a tool that can help us call our APIs with a user-friendly interface. You can download the application from their website. - Postman can automatically format the JSON Responses you get from an API, so it’s more human-readable. - Postman shows your Query Params so you can conveniently add/read/modify/delete them as you wish. - Query Parameters (also known as Query Params), which are appended to the URI, are fields that are used to send additional information or data to the API. - There is a special HTML URL Encoding that needs to be used to read special characters like spaces, commas, or parentheses. Postman translates this automatically for you. Even though we didn’t use the Expedia API, we were able to call the Expedia website. We learned how the Query Parameters work in a real-life example and how to identify them in a URI. In the next post, we will be learning about a component that is found in the Response of our APIs: the HTTP Status Codes. Subscribe now to receive a notification as soon as this post is published! *Prost!* -Alex --- ## Anypoint Platform Chrome Extension Source: https://prostdev.com/post/anypoint-platform-chrome-extension | Published: Dec 1, 2020 | Category: Guides As developers we always want to have information at first hand whenever we need to access information. One of the main reasons this extension was developed is that currently whenever we access Anypoint Platform we need to navigate with many clicks in the portal to see information about the organization or about a specific application. For example, what alerts does this application have? Or, does this application have any schedulers? My motivation for developing this extension is to know what the MuleSoft community is looking for with tools like this and to know how we can create something useful for everybody. We started really basic on this one but definitely the idea would be to make it as robust and helpful as we can. In this post, I’ll show you some of the features that you can find in the Anypoint Platform Chrome Extension so far. ## See Organization name, Organization id and Environment id These parameters are important for you, as a developer. Whenever you need to make an HTTP call using the CloudHub or Hybrid API, or when you are about to use Anypoint CLI. This information is visible at the top of the extension. ![Chrome extension header showing the Organization name, Organization id, and Environment id.](../../assets/blog/anypoint-platform-chrome-extension-1.png) ## This only works so far on the Runtime Manager page There is still a lot of work to do. For now, the extension only works on the Runtime Manager page and diving into the Runtime Manager applications. I will try to add some information for the main pages. *Please don't judge so bad. :P* ## Runtime Manager Application Once you select a specific application on Runtime Manager, you should be able to see more information in the extension. For example, on the top you should be able to see the name of the application, the target system where it’s deployed, and its current status. ![Extension view of an app named configuration-mapper, target CloudHub, status Started, with a Download Artifact button.](../../assets/blog/anypoint-platform-chrome-extension-2.png) As you can see above, there is a button allowing you to download the artifact (directly from your browser). If the application deployed is a Mule 4 application, you will get a .jar file. If the application was deployed in Mule 3.9.X, the artifact will be downloaded as a .zip file. ## See schedulers created in the application Right now to be able to see if there are schedule processes in our application, we need to go into the application, click on “schedulers,” and there we are able to enable or disable the schedulers. In this case, in the application (and using the extension) you should be able to see the schedulers created in CloudHub and Hybrid. ![Image from a CloudHub application](../../assets/blog/anypoint-platform-chrome-extension-3.png) ![Hybrid application view](../../assets/blog/anypoint-platform-chrome-extension-4.png) ## See alerts related to the application You are able to see related alerts in the application just by clicking on the "Alerts" tab. The good thing about this is that you’re able to click on the alert and be redirected to make the changes that you need. ![Extension Alerts tab listing a critical CPU alert that is enabled for the application.](../../assets/blog/anypoint-platform-chrome-extension-5.png) Finally, you can get the extension from [this link](https://chrome.google.com/webstore/detail/anypoint-platform-chrome/gofndnflkobgljnjjalmehnlamoifmhc?hl=en&authuser=0) (there are a few bug fixes on version 1.5, so wait for it). Feel free to use the support page on the Chrome Extension page to give me any feedback, I'm really looking forward to hearing from the community about this. Link to the original post can be found here: [Anypoint Platform Chrome Extension at yucelmoran.com](https://yucelmoran.com/anypoint-platform-chrome-extension/). --- ## Understanding APIs (Part 4): What is a URI? Source: https://prostdev.com/post/understanding-apis-part-4-what-is-a-uri | Published: Nov 24, 2020 | Category: Guides We are slowly starting to get a sense of what an API is. So far, we’ve been learning, piece by piece, some of the components that define an API. Let’s take a look back at what we’ve learned: - [Understanding APIs (Part 1): What is an API?](https://www.prostdev.com/post/understanding-apis-part-1-what-is-an-api) - We defined the 4 aspects: Inputs, Operations, Outputs, and Data Types. We also explained what an API’s Request and Response is. Finally, we put all this together in a diagram. - [Understanding APIs (Part 2): API Analogies and Examples](https://www.prostdev.com/post/understanding-apis-part-2-api-analogies-and-examples) - To get a better grasp, we talked about the Restaurant and Calculator analogies. Then, we defined the Human Resources API example using the CSV and JSON Data Types. - [Understanding APIs (Part 3): What are HTTP Methods?](https://www.prostdev.com/post/understanding-apis-part-3-what-are-http-methods) - We started getting technical and learned about the 5 most popular HTTP methods used by RESTful APIs: GET, POST, PUT, PATCH, and DELETE. ![API diagram: Request input with data type and operation, flowing through Implementation to a Response output](../../assets/blog/understanding-apis-part-4-what-is-a-uri-1.png) *Diagram defined in Part 1, which contains the 4 aspects of the API + the Implementation.* For this post, we will learn what a URL and a URI really are and how this fits in our previously defined diagram. ## What is a URL? In the previous post, we learned that RESTful APIs or RESTful Web Services are calls that are made through [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol) (Hypertext Transfer Protocol). When you’re using a telephone to make a call, you need to know the telephone number to dial. When you want to make a call using HTTP, you need to know the web address instead of the telephone number. This web address is known as a [URL](https://en.wikipedia.org/wiki/URL#:~:text=A%20typical%20URL%20could%20have,and%20a%20file%20name%20(%20index.) (Uniform Resource Locator). E.g. [https://www.google.com](https://www.google.com/). URLs are mostly used to retrieve websites or web pages from the internet (using [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol)) such as Google or Facebook. But URLs can also be used for File Transfer (through [FTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)), database access (using [JDBC](https://en.wikipedia.org/wiki/Java_Database_Connectivity)), and many others. ## What is a URI? From Wikipedia: “The most common form of URI is the Uniform Resource Locator (URL), frequently referred to informally as a web address.” Actually, a [URL](https://en.wikipedia.org/wiki/URL#:~:text=A%20typical%20URL%20could%20have,and%20a%20file%20name%20(%20index.) is just a form of [URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) (Uniform Resource Identifier). A lot of people use both terms to refer to the same thing in the API world. For the sake of this post, let’s just say that a URL is mostly used for visual websites, and we’ll be using URIs to call our APIs. E.g. [https://api.twitter.com/2/tweets/:id](https://api.twitter.com/2/tweets/:id). ## Example: Joke API I went ahead and googled some public APIs that I could use as examples, and I found this [Joke API](https://github.com/15Dkatz/official_joke_api) that, when called, returns with a joke in JSON format. ![Official Joke API documentation listing endpoints for random jokes, ten jokes, and jokes by type](../../assets/blog/understanding-apis-part-4-what-is-a-uri-2.png) *Official Joke API documentation is taken from the* [README.md](https://github.com/15Dkatz/official_joke_api/blob/master/README.md) *file.* Since these are all using GET as the HTTP method, you can open these links directly from your browser (or just click on them). Let’s check out the first option, which is to grab a random joke: [https://official-joke-api.appspot.com/random_joke](https://official-joke-api.appspot.com/random_joke) This is what I get in my browser after clicking or inputting the link: ```json {"id":131,"type":"general","setup":"How do you organize a space party?","punchline":"You planet."} ``` It may be a bit hard to read if you’re not familiar with the JSON syntax, but it basically returned a setup that says, “How do you organize a space party?” and the punchline is “You planet.” Let’s try now to retrieve a random programming joke: [https://official-joke-api.appspot.com/jokes/programming/random](https://official-joke-api.appspot.com/jokes/programming/random) Now I got this as the Response: ```json [{"id":377,"type":"programming","setup":"Knock-knock.","punchline":"A race condition. Who is there?"}] ``` Let me show you a trick to be able to format a JSON into something more human-readable. Go to [this JSON formatter website](https://jsonformatter.curiousconcept.com/) and paste your JSON data into the text box. ![JSON Formatter & Validator website with the raw programming-joke JSON pasted, ready to Process](../../assets/blog/understanding-apis-part-4-what-is-a-uri-3.png) Then just click on “Process,” and you will get something like this: ![JSON formatter showing the indented, human-readable programming joke marked valid per RFC 8259](../../assets/blog/understanding-apis-part-4-what-is-a-uri-4.png) Isn’t it more human-readable now? P.S. I didn’t get the joke. :( ## Breaking down a URI Let’s break down the two URIs we just called: the random joke and the random programming joke. **Random joke** ![Table breaking the random_joke URI into protocol HTTPS, host, and path /random_joke](../../assets/blog/understanding-apis-part-4-what-is-a-uri-5.png) **Random programming joke** ![Table breaking the URI into protocol HTTPS, host, and path /jokes/programming/random](../../assets/blog/understanding-apis-part-4-what-is-a-uri-6.png) As we can see here, the HTTP or HTTPS protocols will be used when using RESTful APIs. HTTPS is just a secure way to call HTTP. The host is the URI part that remains the same even when you change the path you’re using to retrieve different information. Just like how “google.com” doesn’t change even after searching for something. Those two were kind of straightforward to break down, but in the following post, we will learn about Query Parameters and start using a great tool called Postman. With it, you will be able to call your APIs and see the responses in a more human-readable way without using the browser or the JSON formatter tool. ## Recap - A **URL** (Uniform Resource Locator) is the web address that you use in your browser to retrieve a website. It’s also a form of URI. - A **URI** (Uniform Resource Identifier) is what we will be referring to when we call our API. - An API’s URI uses **HTTP** (Hypertext Transfer Protocol) or **HTTPS** (Hypertext Transfer Protocol Secure). - The **host** is the part of the URI that remains the same even when you change the path you’re using to retrieve information. APIs are slowly taking form for us. We already called an API from our browser today! Stay tuned to learn more about Postman and more exciting real-life examples of APIs. *Prost!* -Alex --- ## Understanding APIs (Part 3): What are HTTP Methods? Source: https://prostdev.com/post/understanding-apis-part-3-what-are-http-methods | Published: Nov 3, 2020 | Category: Guides In [part 1](https://www.prostdev.com/post/understanding-apis-part-1-what-is-an-api) of this series, we learned what an API is and the 4 aspects that we can use to define it, which are Inputs, Operations, Outputs, and Data Types. We also outlined this nice diagram to have a visual representation: ![Diagram of an API: Request input plus operation, an Implementation box, and a Response output](../../assets/blog/understanding-apis-part-3-what-are-http-methods-1.png) In the [previous post (part 2)](https://www.prostdev.com/post/understanding-apis-part-2-api-analogies-and-examples), we described some analogies and examples to understand better how APIs can be applied in the real world. There was also a quiz for the second analogy (the calculator). The question was: *What would be the Data Types for the RQ and RS in this scenario?* The answer will be revealed throughout this post! So far, we’ve been talking about the theory behind APIs, but we have to get our hands dirty at some point! Today we will be learning more about these HTTP methods and how they fit into our previously defined diagram. ## What exactly is an API? API can mean a lot of things in the software development world, but I will be talking about the APIs that have been popular lately: the RESTful APIs or RESTful Web Services. These APIs are “calls” (the combination of Request and Response sent to and received from the API) that are made through [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol) (Hypertext Transfer Protocol). To explain HTTP more simply, when you type “www.google.com” or “facebook.com” into your browser, and you press Enter, you’re communicating with a remote server via HTTP. You are sending a Request to a specific web address, and you are getting back a Response, which your browser shows visually (Google or Facebook’s website, in this case). ## Understanding the Operation One of the 4 aspects that we previously defined was the Operation that is sent to the API in the Request. When we send a Request to an API, we have to specify what kind of action we want the API to perform with the data we are sending. In our [previous post](https://www.prostdev.com/post/understanding-apis-part-2-api-analogies-and-examples), we explained the calculator analogy and the Human Resources API example. Let me refresh your memory a bit. **Calculator analogy** In this analogy, we selected the calculator to act as our “API body” or API Implementation. We sent two numbers (2 and 2), and the Operation, in this case, was a sum. We got a result of 4 as the Output or Response. The calculator knows what to do with the numbers because we specify an Operation in the Request. ![Calculator analogy: 2 plus 2 request into a calculator returning 4 as the response](../../assets/blog/understanding-apis-part-3-what-are-http-methods-2.png) Breaking down the 4 aspects: ![Table breaking the calculator into Request and Response with Number data types and an addition operation](../../assets/blog/understanding-apis-part-3-what-are-http-methods-3.png) **Human Resources API** In this more practical example, we wanted to retrieve the first day of employment from certain employees. The list of employees was sent inside a CSV file containing the Employee ID, First Name, and Last Name. The Operation we specified for the API was a “GET.” This is what the Request looked like: ![Input CSV of two employees with the operation to GET their first day of employment](../../assets/blog/understanding-apis-part-3-what-are-http-methods-4.png) The Human Resources API then returned a new CSV file in which there was a new column called “First Day.” This contains the information we asked for in the Request, which was the employees’ first day of employment. ![Response CSV with an added First Day column highlighting each employee's start date](../../assets/blog/understanding-apis-part-3-what-are-http-methods-5.png) Breaking down the 4 aspects: ![Table of the HR API: CSV input and output with a GET operation](../../assets/blog/understanding-apis-part-3-what-are-http-methods-6.png) Now I’m sure you’re wondering why the Operation for this Human Resources API is just “GET.” Well, you’re about to figure it out! ## HTTP Methods As I mentioned earlier, a RESTful API communicates through HTTP. To communicate through HTTP, you need to specify what Operation you want the server to perform. When you write “google.com” in your browser and press Enter, you’re actually performing a GET method when trying to call Google - you want to get or retrieve the main Google website. The server then returns some data as the Response. Your browser is able to translate that information into a visual, which is the Google website with images and animations. And this is exactly what you were expecting. There are 9 HTTP methods: GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, and PATCH. However, the most popular ones are GET, POST, PUT, PATCH, and DELETE. > [!NOTE] > There is A LOT of debate around the POST, PUT, and PATCH methods since they *can* be used to do “the same” kind of operations (create or update data). However, I will be based on the best practices shown by [Mozilla](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods) and [REST API Tutorial](https://www.restapitutorial.com/lessons/httpmethods.html). **GET** This method is used to read or retrieve data from the server or the API. We can see this in the Human Resources API example, when we wanted to retrieve the employees’ first day of employment. The information we sent in the Request was merely used to perform a search of the data, but no information was changed in the server. **POST** This method is used when you want to create new data. In the Human Resources API example, you would use POST to add a new employee. POST is **not** an [idempotent](https://en.wikipedia.org/wiki/Idempotence) method, which means that sending the same information more than once can result in duplicated data. This is why it’s used for creation only. **PUT** This method is mostly used to update or replace existing data, but it can also be used to create new data. The difference with POST is that PUT **is** an [idempotent](https://en.wikipedia.org/wiki/Idempotence) method, which means that sending the same information more than once will have the same effect as sending it just once. We’ll talk more about these differences in the following posts. **PATCH** This method is used to update or modify *parts* of the existing data. An example of this would be updating only the first name of an employee because there was a typo. PATCH *can* be an [idempotent](https://en.wikipedia.org/wiki/Idempotence) method, depending on how the API Implementation works. We’ll talk more about the differences between PUT and PATCH when we discuss updating data in the next posts. **DELETE** This is the easiest method to understand. This method is used to delete data, such as deleting an employee in the HR analogy. ## Recap - The answer to the quiz from the previous post (*What would be the Data Types for the RQ and RS in this scenario?)* is “Number.” - RESTful APIs or RESTful Web Services are calls that are made through HTTP (Hypertext Transfer Protocol). - There are 9 HTTP methods: GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, and PATCH. However, the most popular ones are GET, POST, PUT, PATCH, and DELETE. - GET is used to retrieve data. - POST is used to create new data and is not an idempotent method (or unsafe). - PUT is mostly used to update or replace existing data but can also be used to create new data. It is an idempotent (safe) method. - PATCH is used to update or modify parts of the existing data and *can* be an idempotent method. - DELETE is used to delete data. As I’ve been saying throughout these posts, we’re absorbing information piece by piece. It’s ok if you’re still confused about what it all means or how it all fits together. We’ll get there; I promise. Just don’t give up! *Prost!* -Alex --- ## How to connect your Philips Hue Smart Lights with Google Calendar Source: https://prostdev.com/post/how-to-connect-your-philips-hue-smart-lights-with-google-calendar | Published: Oct 27, 2020 | Category: Tutorials If you’ve been working from home recently, you probably have had one of those dreaded WFH nightmares. You know the one I’m talking about. The one where you’re giving a presentation or you’re in the middle of a meeting with your webcam on, and your roommate or family suddenly enters the room. Wouldn’t it be great to be able to use your Philips Hue Smart Lights and synchronize them with your Google Calendar so that they show a red light when you’re busy? This way, they’ll know when you’re in the middle of a meeting and they won’t enter the room. For this post, I’ll guide you through some simple steps to synchronize your Philips Hue and Google account. ## Before starting If you're new to Philips Hue or you're thinking of purchasing, you should read this post first: [Things to consider before buying a Philips Hue Smart Light](https://www.prostdev.com/post/things-to-consider-before-buying-a-philips-hue-smart-light). I will be assuming that you already have your Google account (to use the Google Calendar) and your Smart Lights all set up with the [Hue application](https://www.philips-hue.com/en-ca/philips-hue-app). > [!NOTE] > This post will not guide you through the synchronization or installation of your Philips Hue Lights. You should also create a [Philips Hue account](https://account.meethue.com/) where you will be storing all the data from Hue - your lights, rooms, scenes, zones, etc. ## 1. Sign in to your Philips Hue account Go to [account.meethue.com](https://account.meethue.com/) and sign in with your credentials. If you go into the [Bridge tab](https://account.meethue.com/bridge), you should be able to see an online status for your bridge. ![Philips Hue account Bridge page showing the Hue Bridge status as online](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-1.png) You can also go into the [Apps tab](https://account.meethue.com/apps) to check some of the other applications that you may already have synced up with your Hue account. ## 2. Connecting IFTTT to your Google account Go into [ifttt.com](https://ifttt.com/home) (If This Then That) and create a new account using the Google account connected to your Google Calendar. You can simply click on [Sign Up](https://ifttt.com/join) and then choose the “Continue with Google” button. This will open a new browser window (or tab), so you can sign in to your Google account. This will automatically link your Google account to your brand-new IFTTT account. If you want to use a different email address from your Google account, you can create an IFTTT account with any email. Go into your [IFTTT account’s settings](https://ifttt.com/settings) and click on “Linked accounts” for Google. ![IFTTT Linked accounts settings showing Apple, Google, and Facebook all not linked](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-2.png) This will also open a new browser window (or tab), so you can sign in to your Google account. After doing this, your Google account will be linked. ![IFTTT Linked accounts now showing the Google account is linked with an Unlink option](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-3.png) ## 3. Configuring the trigger (If This) Click on the [Create button](https://ifttt.com/create/) on the top right side of the screen. This will open the screen where you will be able to program the integration (If This Then That). ![IFTTT "Create your own" screen with the "If This" Add button above "Then That"](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-4.png) Click on “If This” and search for Google Calendar. Click on the Google Calendar icon. ![IFTTT "Choose a service" search for "google calendar" showing the Google Calendar tile](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-5.png) Now select the “Any event starts” trigger. ![Google Calendar trigger options including "New event from search added" and "Any event starts"](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-6.png) Select the calendar you want to use from your Google account. You can also select how much time before the event starts that you want this integration to trigger. ![Trigger fields for "Any event starts" with a calendar picker and "Time before event starts" set to 0 minutes](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-7.png) Click on “Create trigger,” which will return you to the create screen so you can set up the “Then That” part. ## 4. Configuring the behavior (Then That) Click on this second button to continue with the configuration. ![IFTTT applet with the "If Any event starts" trigger set and "Then That" still to add](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-8.png) Now search for “Philips Hue” and click on the icon. ![IFTTT "Choose a service" search for "philips hue" showing the Philips Hue tile](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-9.png) You can choose any action you want your lights to perform. I’m going to choose “Change color.” ![Philips Hue action choices including Turn on/off lights, Blink, Dim, and Change color](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-10.png) Click on “Connect” to select your Philips Hue account. ![IFTTT "Connect service" screen for Philips Hue with a Connect button](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-11.png) This will open a new browser window (or tab), so you can sign in to your Philips Hue account. Select “Yes” to finish the setup. ![Philips Hue "Grant permission" page asking to trust IFTTT, with Yes and No buttons](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-12.png) Now you can select the light, room, or zone from your Philips Hue account that will change its color, and you can write the color name (ignore the “Add ingredient” button). Click on “Create action” when you’re done. ![Change color action fields with the Meeting Zone light selected and color set to Red](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-13.png) It’s all set up now! Just click on “Continue” to finish the configuration. ![Completed applet showing "If Any event starts, Then Change color" with a Continue button](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-14.png) You can also set a name so you can remember what this integration will do. You can choose whether you want to receive a notification when the Applet runs or leave it turned off. Click on “Finish.” ![Review and finish screen with the applet titled "Set Meeting Zone to Red when a new event starts"](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-15.png) ## 5. Did it work? Try it out! Create an event on your Google Calendar and wait for the lights to change. :) ![Google Calendar showing a test event from 8 to 9pm used to trigger the applet](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-16.png) Worked for me! ![A Philips Hue wall light glowing red after the calendar event triggered the applet](../../assets/blog/how-to-connect-your-philips-hue-smart-lights-with-google-calendar-17.png) You can check out all the different integrations or Applets that you can create in IFTTT - you’ll have a lot of fun! Let me know what else you create or if you have any questions. *Prost!* -Alex --- ## Things to consider before buying a Philips Hue Smart Light Source: https://prostdev.com/post/things-to-consider-before-buying-a-philips-hue-smart-light | Published: Oct 22, 2020 | Category: Opinion If you own an [Amazon Alexa](https://amzn.to/3JFoPwc) device or any of the [Google Assistant](https://assistant.google.com/) devices to create a smart home, you may already be looking into some smart lights to go with that! You may even be thinking about buying some [Philips Hue](https://www.philips-hue.com/) products such as smart bulbs, or maybe some lamps or smart switches. My first Philips Hue product was a [lightstrip](https://amzn.to/3mWJSk1)! I still have it, and I love it. But there are some things I wish I knew before buying it. In this post, I’ll show you some of the pros and cons of using Philips Hue for your smart home. ## What exactly are smart lights? There is one main difference between a regular light bulb and a smart bulb: you can turn on and off your regular bulb with only a wall switch. Smart bulbs can be turned on and off with a variety of smart switches, with a mobile app, with a voice command, or even with a synched schedule. Not just that, but you can also dim your smart bulbs and change their colors/tones. Sounding pretty cool so far, right? ## Choosing the right product - Bulbs You can choose to simply change your current regular bulbs with [Philips Hue Smart Bulbs](https://www.philips-hue.com/en-us/bulbs). You can choose the size you need, or if they’re indoor or outdoor bulbs too! When researching bulbs, something important to notice is that some are “white and color ambiance,” and some are just “white.” This small detail can be easy to miss at first, so make sure to choose the product you want. As you may have guessed, one bulb can change into different colors or tones, and the other one is just a white light (which can still be dimmed). ![Product cards comparing Hue White and color ambiance bulbs versus white-only Hue White bulbs](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-1.png) Both are still smart lights, and both offer a range of functions. Just the colors of the light can’t be personalized for the Hue White bulbs. You can also choose some vintage bulbs, like this one: ![Hue White Filament vintage-style bulb product card with spec details](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-2.png) Or some smaller “candle” bulbs, like these: ![Two small E12 candle bulb product cards, white ambiance and color ambiance versions](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-3.png) These are just the most common bulbs to give you an example of the variety of products that you can find. You can go to the [product overview page](https://www.philips-hue.com/en-us/product-overview) and check out all the different lights they have to offer. The options are endless! ![Philips Hue product-overview grid of light types: bulbs, lightstrips, lamps, outdoor and more](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-4.png) As you can see, you can buy just the bulbs, or you can also choose between different lamps or outdoor lightings. My favorite is, by far, the lightstrips. ## Lightstrips [Lightstrips](https://amzn.to/3mWJSk1) have some adhesive on the back, so you can paste them wherever you want! Check out these COOL examples from the internet: **Behind your desk** ![Source: Babbling Boolean YouTube channel - Philips Hue LightStrip Review](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-5.jpg) **Over furniture / near the ceiling** ![Source: New Home Tricks - Philips Hue Lightstrip Plus](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-6.jpg) **Light-up the halls!** ![Source: Trend Hunter - Customizable Connected Light Strips](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-7.jpg) You can also use them behind the couch/furniture, behind a TV/screen, under the stairs, or under the table. You choose! ## Accessories So what are these [accessories](https://www.philips-hue.com/en-us/products#filters=CONTROLS_SU&page=1)? There are some smart switches that you can program from a mobile application to do different things - turn several lights on/off, dim the lights, choose different colors. You can do all these directly from the Hue app, but some people prefer to touch a switch. :) ![Hue accessory cards for the dimmer switch, smart button, and motion sensor](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-8.png) There are also smart plugs, cable extensions, power supplies, and more. But wait! Don’t buy anything just yet, you still need to learn about some **very important** things. ## Connecting over Wi-Fi - Philips Hue Bridge All these lights can be used over Bluetooth, and for that, you only need your smartphone. But honestly, why use only Bluetooth when you can connect everything to the internet? If you connect your lights to your Wi-Fi, you can even turn on/off your lights when you’re away on vacation! But to do this, you need to make an additional purchase: the [Philips Hue Bridge](https://amzn.to/3FUbu0P). ![The white square Philips Hue Bridge hub with its status lights glowing blue](../../assets/blog/things-to-consider-before-buying-a-philips-hue-smart-light-9.png) > “The brains of the Philips Hue smart lighting system, the Hue Bridge allows you to connect and control up to 50 lights and accessories.” With this little box, you’ll be able to control up to 50 lights/bulbs/lamps and accessories. But you need to know this: **you need to connect your Bridge directly to your modem using an Ethernet cable**. You can’t just connect the Bridge to the Wi-Fi directly. Once the Bridge is connected with the Ethernet cable, *then* you can use all your products over Wi-Fi. *Sneaky marketing...* So, if your modem is not accessible to you for any reason, you may want to think twice before committing to Philips Hue. Here’s a quick video to get you started with your first Philips Hue Bridge installation and to get it connected to your modem: [Signify - How to install a Philips Hue starter kit with Jonathan Morrison](https://youtu.be/aTa26eNlI54). You can opt to buy the Bridge alone or choose one of the [Starter Kits](https://amzn.to/32Io5pF), which will provide you with the Bridge, plus some lights or switches. ## My favorite part - syncing your lights to your screen This is my absolute favorite thing about Philips Hue. You can sync the color of the lights in a given room based on whatever colors are on the screen. This gives you an immersive experience, especially when playing video games or watching movies! However… To be able to sync your lights to your screen, it is only free when using a PC/laptop through the [Hue Sync app](https://www.philips-hue.com/en-hk/entertainment/hue-sync), which is compatible with both a Mac OS and Windows. **If you want to do the same with a regular TV, you’ll need a** [Play HDMI Sync Box](https://amzn.to/3ztwLfw). I’ll end this post by providing this amazing video, so you can see this beautiful functionality in action (the more lights you use, the more detailed colors you’ll get): I hope this post was useful to help you choose your next smart lights. Let me know if you set them up! *Prost!* -Alex ## Links - [Amazon - Echo Dot](https://amzn.to/3JFoPwc) - [Google Assistant](https://assistant.google.com/) - [Philips Hue](https://www.philips-hue.com/) - [Amazon - Lightstrips](https://amzn.to/3mWJSk1) - [Amazon - Bulb](https://amzn.to/3pT9Irf) - [Philips Hue - Product Overview](https://www.philips-hue.com/en-us/product-overview) - [Amazon - Bridge (Hub)](https://amzn.to/3FUbu0P) - [Amazon - Starter Kit](https://amzn.to/32Io5pF) - [Philips Hue - Hue Sync app for PC/Laptop](https://www.philips-hue.com/en-hk/entertainment/hue-sync) - [Amazon - Play HDMI Sync Box](https://amzn.to/3ztwLfw) --- ## 3 Simple Steps to Convert a Flat File into JSON / CSV / XML Source: https://prostdev.com/post/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml | Published: Oct 20, 2020 | Category: Tutorials Hello Muleys, This article is all about how simple it is to convert a Flat File into JSON, CSV, or XML using the all-time powerful weapon: DataWeave 2.0. Please do not forget to read the “Things to Remember” at the end of the article so that you don’t commit any mistake while coding. For this post, I used Anypoint Studio 7.4 and Mule Runtime version 4.2.2. Before starting, let's get familiar with a few terms. Flat File: (Also known as “FlatFile,” “FFD File,” or Flat-File”) A Flat File is data in a plain text format. Usually we get this kind of data from either mainframe systems, SFTP, or Files. Copybook: A copybook is a selection of code that defines data structures. If a particular data structure is used in many programs, then instead of writing the same data structure again, we can use copybooks. In general terms, to define the fields Data Types/Length and their structure, we define them in a copybook which in turn helps us in generating FFD files using a Transform Message. ## Step 1: Creating the copybook As previously said, a Flat File is a plain text string and the rows are separated by a new line, e.g. ``` 1234SRAVAN 27-09-1992TELANGANA HYDERABAD 11111 5678LINGAM 07-02-1996USA CHICAGO 77777 ``` This Flat File contains 98 characters with spaces (will discuss why spaces are also considered). Each row has 49 characters defined and they are divided by a new line. In my example, the data I am getting from SFTP or Mainframe basically contains the following information: ``` 'BIRTHID' 'BIRTHNAME' 'BIRTHDATE', 'BIRTHSTATE','BIRTHCITY', 'BIRTHZIP' ``` > [!IMPORTANT] > Remember the Flat File always has FIXED length! If you have BIRTHNAME defined for the length of 10 characters and if your name has only 6 characters, then the remaining 4 characters MUST be 4 empty spaces. Below is the copybook that I have created to define my incoming fields: ``` 000000 01 BIRTH-DETAILS. 000100 05 BIRTHID PIC X(4). 000200 05 BIRTHNAME PIC X(10). 000300 05 BIRTHDATE PIC X(10). 000400 05 BIRTHSTATE PIC X(10). 000500 05 BIRTHCITY PIC X(10). 000600 05 BIRTHZIP PIC X(5). ``` Please follow the indentation in this copybook. The type is nothing but your datatype of that particular field, length is the fixed length allocated to that field. Save this file with extension .cpy, e.g., BirthDetails.cpy. ## Step 2: Auto-generating the FFD file using DataWeave This is a pretty simple step with no coding involved! We basically need an FFD file to convert the Flat File to JSON or CSV. - Drag-and-Drop the Transform Message component in an empty flow. - Go to the Input section and click on Define Metadata. - Click on Add and give some Type Id. - Select Copybook from the Type drop-down. - Click Import and select the copybook that you have created in Step 1. - Finally, click Select. ![Anypoint Studio Select metadata type dialog importing BirthDetaiks.cpy as a Copybook type.](../../assets/blog/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml-1.png) After clicking on Select, an FFD file will be created automatically and placed under `src/main/resources` > schemas > BirthDetails.ffd Please copy the FFD from the schemas folder and paste it inside `src/main/resources` as shown in below screenshot: ![Package Explorer with the generated BirthDetaiks.ffd file copied from schemas into src/main/resources.](../../assets/blog/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml-2.png) FFD file generated: ```yaml form: COPYBOOK id: 'BIRTH-DETAILS' values: - { name: 'BIRTHID', type: String, length: 4 } - { name: 'BIRTHNAME', type: String, length: 10 } - { name: 'BIRTHDATE', type: String, length: 10 } - { name: 'BIRTHSTATE', type: String, length: 10 } - { name: 'BIRTHCITY', type: String, length: 10 } - { name: 'BIRTHZIP', type: String, length: 5 } ``` ## Step 3: Final Transformation No matter what your data source is; whether it’s a File Read connector or an HTTP Listener, the first thing that you need to place after the source is a Set Payload component. Drag-and-Drop a Set Payload component and configure as below. It needs to have a value of #[payload] Now go to the MIME Type tab and configure recordTerminator and schemaPath as below: ![Set Payload MIME Type tab set to application/flatfile with recordTerminator and schemaPath parameters.](../../assets/blog/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml-3.png) Place a Transform Message just after the Set Payload and code it as below: ![Transform Message mapping each copybook field to a JSON details object with default "" values.](../../assets/blog/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml-4.png) If you want your output in a CSV format, just replace application/json with application/csv If you want your output in XML format, we need another Transform Message after application/json output because this doesn't have a root tag. If you try to map directly to application/xml, it will throw an error! That's it finished :) ## Things to Remember From the following image you can see that the name is defined as 10 characters in your FFD file. But "sravan' has only 6 characters. So the input name must have 10 characters, i.e., ``` "sravan " ``` We have to have 4 spaces appended at the end. This is because for each row, we defined the FFD file with the following rules: - First 4 characters belong to Birth ID - Next 10 characters belong to Birth Name - Next 10 characters belong to Birth Date - And so on... Each special character is treated as 1 character. The rows must be divided using a new line break (highlighted in yellow). If you can see the input, the State name for the second row is USA (3 characters). So we have appended 7 spaces to it to make it a perfect and valid Flat File. One of the validations mentioned above failed in input, we will get Transformation errors! ![Flat file input with padding spaces highlighted, transformed into two JSON records on the right.](../../assets/blog/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml-5.png) Try it out! Pass the Flat File data through Postman using a POST Method and select the type as Text. Send it as input for your Application. ![Postman POST request with the flat file pasted in a raw Text body, ready to send to the app.](../../assets/blog/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml-6.png) Here’s your final Mule application: ![Final Mule flow: HTTP Listener, then Set Payload, then Transform Message.](../../assets/blog/3-simple-steps-to-convert-a-flat-file-into-json-csv-xml-7.png) Get back to me if you have any doubts! Happy Learning! Yours, Sravan Lingam --- ## 7 tips to start writing your technical blog post Source: https://prostdev.com/post/7-tips-to-start-writing-your-technical-blog-post | Published: Oct 13, 2020 | Category: Opinion In the previous post, [Things to consider before writing a technical blog post](https://www.prostdev.com/post/things-to-consider-before-writing-a-technical-blog-post), we asked ourselves some questions to help us choose the topic that we want to write about. We also learned how to visualize our target audience before writing our post. For this post, we will start writing our article or blog post! Yes, just like that. Take your laptop, your PC and your keyboard, your tablet, or whatever you’re most comfortable using. Just take it and follow these simple steps to start writing! ## 1. Pick a title You can change this title later, but pick something that’ll keep you focused on what you’re trying to explain. This way, if you find yourself over-explaining something, just read your title once more and ask yourself if you actually need that long paragraph, or if you can keep it shorter since it’s not the focus of your article right now. You can create a title that explains exactly what you will be talking about. Look at this example: “How to get started on Java programming.” You can also create lists of something related to the technology you chose, for example: - “5 ways to improve your application’s performance” - “10 tips for improving your time management skills” - “5 reasons why you should be using Service Mesh” - “The 10 biggest mistakes when developing Python scripts” - “The 5 biggest misconceptions of RESTful APIs” Some authors choose titles that are catchy or end with a question to grab the reader’s attention, like this one: [Software Quality is not cheap, but would you risk your company’s future for the lack of it?](https://www.prostdev.com/post/software-quality-is-not-cheap-but-would-you-risk-your-company-s-future-for-the-lack-of-it) Ultimately, you want a title that will help your audience understand the topic of the post, but you also want something memorable. ## 2. Write an introduction Most people will only read the first(s) paragraph(s) before they decide if your post is worth reading or not. The introduction can be just one or two paragraphs; try to keep it simple. Write a summary of what you’re going to cover with your article and try to get the reader’s attention. A popular way of ending an introduction is with one or two questions that you’re going to answer throughout the post, which will also pique the reader’s curiosity. - “Can this function be used with objects?” - “How can you achieve high performance with complex code?” - “Is it really worth using APIs for your project?” ## 3. Write some definitions or pre-requisites This step is optional. It depends on what kind of audience is reading your post. I personally try to explain the basic concepts that the reader needs to know so that they can follow along with me. How long the explanation is, depends on the definition’s complexity and how important it is for the audience to know this term before they read the rest of the post. Sometimes I use two or three sentences to define a term or concept. Sometimes I use one or two paragraphs. And sometimes, I even create a whole separate article to explain it, and then I just reference that post (with a link) before starting my current one. ## 4. Develop the article Now you can start writing the actual article’s content. Explain all you want, or talk to the reader just as if you were face to face. Use formal or informal wording. In this part, try to be yourself! This is what will differentiate your post from others. Even if you write about a very well-known subject, maybe you’re using a new perspective that no one has done before, or maybe you’re giving easy-to-read explanations. This is your essence as a writer. Own it! Make use of titles and subtitles, or bold and underlined formatting, in order to have a readable and user-friendly format. Structure your post in sections. If your audience has to pause reading the article for a meeting, when they return, they’ll easily want to know where to continue reading afterwards. Having subtitles or sections will help them maintain some order. Otherwise, they’ll get lost in between paragraphs and may lose interest in your post. > [!NOTE] > people tend to read articles that are shorter (2-5 minutes, or 800-1,300 words), but sometimes posts can become longer when there are a lot of explanations involved. You can always separate your information in a series of posts (Part 1, Part 2…) instead of creating a huge article. ## 5. Add visual content Most people nowadays want to read articles because they are way quicker than reading a book. We don’t have the time, nor the mind, to be able to sit for hours and read a programming language book. Instead, we read articles or watch videos that can easily break down the information. For this same reason, the average reader today may lose interest if the post is only paragraphs with no visual content. Adding media (pictures, gifs, pieces of code) will definitely get the reader’s attention, and people may focus on these visuals if they don’t want to read everything. Or the readers can skip the explanations because they can understand your post just by looking at the pictures or even the subtitles. Be careful not to add a ton of media everywhere; doing this will also lose the reader’s attention because they can lose the sense of the section they’re in and feel lost. You do *not* want your target audience to feel lost while reading your article. ## 6. Write a conclusion or a summary Some readers will skip your article altogether because they don’t feel like reading. It’s not your fault; that's just the way it is. So, be ready for your readers to skip the whole explanation. Create one or two paragraphs of a *very quick* summary of the complete article - maybe one or two sentences per definition. If they don’t understand the summary completely, they’ll know that it’s because they skipped the detailed explanation. Another way to look at it is: Imagine that your reader is about to get to an important meeting and they need to sound like they know everything about your article, so you have 30 seconds to explain it to them; otherwise, they’re going to get fired! ## 7. Say goodbye This, just like the body of the post, has to sound like you. This is where you get to be friendly and yourself. You can use one or two sentences to say goodbye and thank them for reading. You can even sign with your name or a nickname. I personally prefer to sign with “-Alex.” ## Recap - **Pick a title** that will catch the reader’s attention. You can choose to explain exactly what the article is about, make a list of things, or create a catchy title. You can even add a question to the title to spark your audience’s curiosity. - **Write an introduction** to give the reader a better idea of the post’s topic. This is where they’ll decide if they want to read your content or not! - **Write some definitions or pre-requisites**, so your reader is not confused when they read the rest of the post. Depending on who the article is for (your target audience), you can give a quick 2-sentence summary, or you can redirect the reader to a previous article. - **Develop the article**! Remember to be yourself and try to keep it short, unless it’s a very technical post. You can also break it down into a series of articles instead of having one long post. - **Add visual content** to keep your reader’s attention, but don’t overdo it! - **Write a conclusion or a summary**. Remember to imagine that you have 30 seconds to summarize your article, or your colleague will get fired! - **Say goodbye** to the reader in your own special way. I hope this was helpful to get you started on creating some blog posts! Leave me a comment if you have any questions or if you’d like to see more content about writing technical articles. *Prost!* -Alex --- ## Implementing NetSuite Saved Search with MuleSoft Source: https://prostdev.com/post/implementing-netsuite-saved-search-with-mulesoft | Published: Oct 6, 2020 | Category: Tutorials ## Introduction NetSuite is a SaaS-based ERP which allows companies to manage important business using a single tool. NetSuite provides a suite of cloud-based financials / Enterprise Resource Planning (ERP) and omnichannel commerce software. NetSuite has capabilities of ERP, CRM, Manufacturing, e-commerce, retail. MuleSoft provides a NetSuite connector, which enables to automate the business process and synchronize the data between NetSuite and third-party tools. NetSuite Connector uses SuiteTalk WSDL to provide SOAP-based integration to generate NetSuite business objects, make use of different authentication levels, and support error handling. NetSuite connector provides various features and capabilities: - SOAP-Based integration. - Connecting NetSuite using REST calls to RESTlets that expose APIs created with SuiteScript. - Multiple types of Authentication. - Error Handling support. ## How to Connect NetSuite with MuleSoft MuleSoft NetSuite connector provides various ways of Authentication to connect NetSuite. - Login authentication. - Request based authentication. - SSO authentication. - Token authentication. ![NetSuite connector config dropdown listing Login, Request, SSO, and Token authentication](../../assets/blog/implementing-netsuite-saved-search-with-mulesoft-1.png) One of the most common ways of connecting NetSuite is using Token authentication. For this, we will need Consumer key, Consumer secret, Token ID, and Token secret. ![NetSuite token authentication form with consumer key, secret, token, and account fields](../../assets/blog/implementing-netsuite-saved-search-with-mulesoft-2.png) Now, we are going to see a few operations related to NetSuite. ## Saved Search Saved Search allows us to search the data in NetSuite and it can be performed on any record type in NetSuite. MuleSoft provides the operation Search which can be used to perform the search on the record type. ## Use Case 1 - Basic Employee Search (lastName starts with J) Let's consider, we want to retrieve all the Employee details from NetSuite whose lastName starts with "J". MuleSoft’s search operation in the NetSuite connector can be used to retrieve all employee details whose lastName starts with "J". We will be adding the following script in MuleSoft Transform Message to pass as Payload to NetSuite connector for the search operation and record type as `EMPLOYEE_BASIC`. ```dataweave %dw 2.0 output application/java --- { lastName: { operator: "STARTS_WITH", searchValue: "J" } } as Object { class: "org.mule.module.netsuite.extension.api.EmployeeSearchBasic" } ``` **NetSuite Connector Configuration** ![NetSuite Search operation config with key EMPLOYEE_BASIC and search record set to payload](../../assets/blog/implementing-netsuite-saved-search-with-mulesoft-3.png) ## Use Case 2 - Advanced Employee Search Let's consider, we want to retrieve all the employee details from NetSuite. NetSuite has created Saved Search which can be utilized in the MuleSoft application to fetch all employee details. NetSuite provided us savedSearchId as "8656" and savedSearchScriptId as "EmployeeSearch8656". We will be adding the following script in MuleSoft Transform Message to pass as Payload to NetSuite connector for the search operation and record type as `EMPLOYEE_ADVANCED`. ```dataweave %dw 2.0 output application/java --- { savedSearchId: "8656", savedSearchScriptId: "EmployeeSearch8656" } as Object { class: "org.mule.module.netsuite.extension.api.EmployeeSearch" } as Object { class: "org.mule.module.netsuite.extension.api.EmployeeSearchAdvanced" } ``` **NetSuite Connector Configuration** ![NetSuite Search operation config with key EMPLOYEE_ADVANCED and search record set to payload](../../assets/blog/implementing-netsuite-saved-search-with-mulesoft-4.png) ## Use Case 3 - Advanced Employee Search (lastModifiedDate is after 1st August 2020) Let's consider, we want to retrieve all the Employees details from NetSuite whose lastModifiedDate is after 1st August 2020. NetSuite has created Saved Search which can be utilized in the MuleSoft application to fetch all employee details whose lastModifiedDate is after 1st August 2020. NetSuite provided us savedSearchId as "8657" and savedSearchScriptId as "EmployeeSearch8657". We will be adding the following script in MuleSoft Transform Message to pass as Payload to NetSuite connector for the search operation and record type as `EMPLOYEE_ADVANCED`. ```dataweave %dw 2.0 output application/java --- { savedSearchId: "EmployeeSearch8657", savedSearchScriptId: "EmployeeSearch8657", criteria: { basic: { lastModifiedDate: { operator: "AFTER", searchValue: "2020-08-01T12:00:00" as LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss"} } as Object { class: "org.mule.module.netsuite.extension.api.SearchDateField" } } as Object { class: "org.mule.module.netsuite.extension.api.EmployeeSearchBasic" } } as Object { class: "org.mule.module.netsuite.extension.api.EmployeeSearch" } } as Object { class: "org.mule.module.netsuite.extension.api.EmployeeSearchAdvanced" } ``` ## Use Case 4 - Advanced Employee Search (isInactive is false and retrieve firstName, middleName and lastName) Let's consider, we want to retrieve all the Employees details from NetSuite who are active and retrieve only firstName, middleName, and lastName. We will be adding the following script in MuleSoft Transform Message to pass as Payload to NetSuite connector for the search operation and record type as `EMPLOYEE_ADVANCED`. ```dataweave %dw 2.0 output application/java --- { columns: { basic: { firstName: [{ customLabel: "First Name" }], middleName: [{ customLabel: "Middle Name" }], lastName: [{ customLabel: "Last Name" }] } }, criteria: { basic: { isInactive: { searchValue: false } } } } as Object { class: "org.mule.module.netsuite.extension.api.CustomerSearchAdvanced" } ``` ## Use Case 5 - Basic employee search (lastModifiedDate is after 1st August 2020 and isInactive is false) Let's consider, we want to retrieve all the Employees details from the NetSuite who are active and lastModifiedDate is after 1st August 2020. We will be adding the following script in MuleSoft Transform Message to pass as Payload to NetSuite connector for the search operation and record type as `EMPLOYEE_BASIC`. ```dataweave %dw 2.0 output application/java --- { lastModifiedDate: { operator: "AFTER", searchValue: "2020-08-01T12:00:00" as LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss"} }, isInactive{ searchValue: false }, } as Object { class: "org.mule.module.netsuite.extension.api.EmployeeSearchBasic" } ``` Now, you know how to perform search operations with the MuleSoft NetSuite connector. --- ## What is CIDR (Classless Inter-Domain Routing) in MuleSoft VPC Source: https://prostdev.com/post/what-is-cidr-classless-inter-domain-routing-in-mulesoft-vpc | Published: Sep 29, 2020 | Category: Guides ## Introduction CIDR stands for Classless Inter-Domain Routing and it’s a way of allocating IP addresses or hosts in a more efficient manner. It replaces the old way of allocating IP addresses based on the class system. This method allocates the IP Addresses or hosts in a more efficient way and avoids the waste of IP Addresses. - Class A: 16 million host identifiers or IP Addresses. - Class B: 65,535 host identifiers or IP Addresses. - Class C: 254 host identifiers or IP Addresses. Let's consider the Organization requires around 500 IP Addresses or Hosts. In such cases, organizations have to go with a Class B IP distribution system where almost more than 60,000 IP addresses are wasted. ## What is an IP Address? IP Addresses consist of two groups of bits in the address, the most significant bits are network the prefix, which identifies the network or subnet; and the least significant bits are the host identifier, which specifies a particular interface of the host on that network. IP Addresses have 2 components: - Network Address - Host Address Each IP Address (IPv4) is 32 bit or 4 Octet. Below is the representation of an IP Address in Binary: ![Binary breakdown of the IP address 192.168.0.1 across its four 8-bit octets.](../../assets/blog/what-is-cidr-classless-inter-domain-routing-in-mulesoft-vpc-1.png) CIDR Block Notation: ``` xxx.xxx.xxx.xxx/n ``` where n is the number of bits used for the subnet mask. The subnet mask is made up of setting up all network bits to all ones and host bits to all zeros. Let's consider this scenario: if you provide this CIDR Block 192.168.0.0/24, it will give 255 hosts or IP addresses. ![Table mapping CIDR notation to total hosts: /24 gives 256, /23 gives 512, /22 gives 1024.](../../assets/blog/what-is-cidr-classless-inter-domain-routing-in-mulesoft-vpc-2.png) ## What is a Subnet? A subnetwork or subnet is a logical subdivision of an IP network. The practice of dividing a network into two or more networks is called subnetting. Computers that belong to a subnet are addressed with an identical most-significant bit-group in their IP addresses. Now, we will see how to Calculate the total number of hosts using Subnet Mask. ## Use Case 1 Subnet Mask 192.168.0.0/24 will equate to IP Range 192.168.0.0 — 192.168.0.255. ![Logical AND of IP 192.168.0.1 with the /24 subnet mask, showing 24 network bits and 8 host bits.](../../assets/blog/what-is-cidr-classless-inter-domain-routing-in-mulesoft-vpc-3.png) N represents Network and H represents Host. In the above example, we made 24 bits to ones and the remaining 8 bits to zeros because the Subnet Mask end range is 24. Total zeros are 8 for Host (2*2*2*2*2*2*2*2=256). This will give an IP range of 192.168.0.0 — 192.168.0.255 (256 Hosts in total). ## Use Case 2 Subnet Mask 192.168.0.0/23 will equate to IP Range 192.168.0.0 — 192.168.0.511. ![Logical AND of IP 192.168.0.1 with the /23 subnet mask, showing 23 network bits and 9 host bits.](../../assets/blog/what-is-cidr-classless-inter-domain-routing-in-mulesoft-vpc-4.png) N represents Network and H represents Host. In the above example, we made 23 bits to ones and the remaining 9 bits to zeros because the Subnet Mask end range is 23. Total zeros are 9 for Host (2*2*2*2*2*2*2*2*2=512). This will give an IP range of 192.168.0.0 — 192.168.0.511 (512 Hosts in total). ## Use Case 3 Subnet Mask 192.168.0.0/27 will equate to IP Range 192.168.0.0 — 192.168.0.31. ![Logical AND of IP 192.168.0.1 with the /27 subnet mask, showing 27 network bits and 5 host bits.](../../assets/blog/what-is-cidr-classless-inter-domain-routing-in-mulesoft-vpc-5.png) N represents Network and H represents Host. In the above example, we made 27 bits to ones and the remaining 5 bits to zeros because the Subnet Mask end range is 27. Total zeros are 5 for Host (2*2*2*2*2=32). We have borrowed 3 bits from the host to make a total of 27 bits. Subnet will be (2*2*2=8) and the Host will be 32. So we can get a total of 8 subnets. Subnetworks will be 192.168.0.0/27, 192.168.0.31/27, 192.168.0.63/27, 192.168.0.95/27, 192.168.0.127/27, 192.168.0.159/27, 192.168.0.191/27, 192.168.0.223/27 Here we are dividing the subnet mask into smaller subnetworks. Whenever you are creating MuleSoft VPC, you need to make sure whatever CIDR Mask you are providing doesn't conflict with your on-premise or any other networks. The smallest network subnet block you can assign for your Anypoint VPC is /24 and the largest is /16. For each worker deployed to CloudHub, the following IP assignation takes place: - For better fault tolerance, the VPC subnet may be divided into up to four Availability Zones. - A few IP addresses are reserved for infrastructure. - At least two IP addresses per worker to perform at zero-downtime. ## MuleSoft VPC Sizing Now, we learn how we can do the VPC sizing. Below are some requirements. - You have four environments: dev, test, sit and prod. - Application on dev and sit must run on 1 Worker. - Application on test must run on 2 Workers. - Application on prod must run on 2 Workers. - Total Application = 100 (Near Future). - The organization will have 2 VPCs: one for PROD and another for NON-PROD. The problem is that we need to decide the minimum CIDR block that will be needed for PROD and NON-PROD VPCs. ![VPC sizing table per environment totaling 300 IPs for Production and 600 for Non-Production.](../../assets/blog/what-is-cidr-classless-inter-domain-routing-in-mulesoft-vpc-6.png) There will be 2 IPs reserved for each VPC for infrastructure. For Production VPC, we require around 300 IPs and it will be provided by a subnet mask of /23 (e.g. 192.168.0.0/23). This subnet mask will provide 512 IPs. For Non-Production VPC, we require around 600 IPs and it will be provided by a subnet mask of /22 (e.g. 192.168.0.0/22). This subnet mask will provide 1024 IPs. You should know how you can make use of the CIDR range efficiently and perform MuleSoft VPC sizing. --- ## Mule 4 Continuous Integration using Azure DevOps Source: https://prostdev.com/post/mule-4-continuous-integration-using-azure-devops | Published: Sep 24, 2020 | Category: Tutorials *GitHub repository with the project can be found at the end of the post.* Once we start developing applications in MuleSoft and we keep our code stored in any source control platform like Github, Bitbucket, GitLab, or Azure; as a developer and as a best practice, we look into automating the process to deploy our applications to either CloudHub or an On-Premise server. In this post I will explain how a MuleSoft application can be automatically deployed into CloudHub or an On-Premise server from Azure DevOps as our main Continuous Integration (CI) and source control platform. ## Creating a new project in Azure The first step would be to setup our project in Azure DevOps, for this you need a Microsoft account. You can get one [here](https://dev.azure.com/). Then, we can create a new project, provide a name and a description, as well as set the privacy settings. By default you can set it as "private". ![Azure DevOps Create a project form with name mulesoft-pipeline and Private visibility selected](../../assets/blog/mule-4-continuous-integration-using-azure-devops-1.png) Once it is created, we can go into the main page and locate the "Repos" section. Once we see the page, we can use the information to send our code directly from our local environment. ![Empty Azure Repos page showing clone URL and commands to push an existing repository](../../assets/blog/mule-4-continuous-integration-using-azure-devops-2.png) Just before actually pushing any code we can create a pretty simple endpoint that we can hit later to verify it’s all working. Let's create a simple RAML definition in Anypoint Platform: ```yaml #%RAML 1.0 title: Sample API /persons: get: responses: 200: body: application/json: example: | [{ "id": 1, "name": "Edgar Moran", "username": "emoran", "email": "yucel.moran@some.test", "address": { "street": "Kulas Light", "suite": "Apt. 556", "city": "Gwenborough", "zipcode": "92998-3874", "geo": { "lat": "-37.3159", "lng": "81.1496" } }}] ``` Then we can generate the flows in our project. Right click on the RAML file > Mule > Generate Flows from Local REST API. ![Anypoint Studio right-click menu choosing Mule then Generate Flows from Local REST API](../../assets/blog/mule-4-continuous-integration-using-azure-devops-3.png) Once we have our flow generated, we can proceed to create our `test.yaml` file in order to identify any value we want to attach to our environment. It will look just like this: ```yaml http: port: "8081" environment: "[TEST]" ``` I will create a `global.xml` configuration file to keep my configurations. I will add a Configuration Properties element: ![Configuration properties dialog with the File field set to an env-based YAML placeholder](../../assets/blog/mule-4-continuous-integration-using-azure-devops-4.png) After that, we can create a Global Property, in this case called env which will allow us to know which environment we are working on. ![Global Property dialog defining a property named env with a placeholder value](../../assets/blog/mule-4-continuous-integration-using-azure-devops-5.png) I know, a lot of steps, right? But let's try to make it right, just before our final check to verify everything is working. Let's add an environment variable called env in our Run Configuration. ![Anypoint Studio Run As submenu with Run Configurations option highlighted](../../assets/blog/mule-4-continuous-integration-using-azure-devops-6.png) ![Run Configurations Environment tab adding an env variable set to test](../../assets/blog/mule-4-continuous-integration-using-azure-devops-7.png) Now let's run the project. If everything was correctly set up, you'll see your project deployed! ![Anypoint Studio flow with APIkit Router and console showing the app deployed successfully](../../assets/blog/mule-4-continuous-integration-using-azure-devops-8.png) We can start pushing our first version of this working code into our Azure repository, so let's do that. ## Generating a personal access token to push our code to Azure Let's go back into the Azure DevOps main project page: ![Azure DevOps mulesoft-pipeline project overview page with summary and stats](../../assets/blog/mule-4-continuous-integration-using-azure-devops-9.png) And let's create a personal access token: ![Azure DevOps user menu open with Personal access tokens option highlighted](../../assets/blog/mule-4-continuous-integration-using-azure-devops-10.png) Click con Create a new token. You'll see a bunch of options, select the actions and permissions you consider are the best depending on your needs. ![Create a new personal access token panel with name, expiration, and scope checkboxes](../../assets/blog/mule-4-continuous-integration-using-azure-devops-11.png) Once created, you'll have a new token which becomes your project password, so keep it safe! Now we can push our code. If the terminal asks you for a password, use the token you just generated and your code should be on master. ![Azure Repos Files view on master listing src, .gitignore, mule-artifact.json and pom.xml](../../assets/blog/mule-4-continuous-integration-using-azure-devops-12.png) All right, so we have our code already in Azure. Now let's create a new branch for our repository, just let's call it test based from master and let's create another one as a developer branch called emoran. It's important to mention that during this sample on emoran branch we will review the build and in the test branch we will deploy to CloudHub. ## Creating an Artifact Before creating our pipeline we need to make sure we have the right information in our `settings.xml` and our `pom.xml` files. This artifact will allow us to get the Distribution Management piece we need to set in our project. For this, we will need to go into our main project page, and locate the artifact option on the left, then we need to click on "Connect to feed". We select maven and then it will show the information we need to put in place in our `settings.xml` and `pom.xml` files. ![Azure Artifacts Connect to feed page showing Maven setup XML for pom and settings files](../../assets/blog/mule-4-continuous-integration-using-azure-devops-13.png) ## Preparing the settings.xml and pom.xml files The next task will be to prepare our files in order to make sure we will be able to build and deploy our application into CloudHub. Here's how my `settings.xml` file looks like: ```xml yucelmoran-azure-pipeline yucelmorans ${azure.password} MuleRepository ${nexus.username} ${nexus.password} Mule true yucelmoran-azure-pipeline https://pkgs.dev.azure.com/yucelmorans/yucelmoran-azure-pipeline/_packaging/yucelmoran-azure-pipeline/maven/v1 true true MuleRepository MuleRepository https://repository.mulesoft.org/nexus-ee/content/repositories/releases-ee/ default true true ``` and here's the `pom.xml`: ```xml 4.0.0 com.emoran yucelmoran-azure-pipeline 1.0.0-SNAPSHOT mule-application yucelmoran-azure-pipeline UTF-8 UTF-8 4.3.0 3.3.5 1.7 scm:git:git@ssh.dev.azure.com:v3/yucelmorans/yucelmoran-azure-pipeline/yucelmoran-azure-pipeline scm:git@ssh.dev.azure.com:v3/yucelmorans/yucelmoran-azure-pipeline/yucelmoran-azure-pipeline https://yucelmorans@dev.azure.com/yucelmorans/yucelmoran-azure-pipeline/_git/yucelmoran-azure-pipeline yucelmoran-azure-pipeline https://pkgs.dev.azure.com/yucelmorans/yucelmoran-azure-pipeline/_packaging/yucelmoran-azure-pipeline/maven/v1 org.apache.maven.plugins maven-clean-plugin 3.0.0 org.mule.tools.maven mule-maven-plugin ${mule.maven.plugin.version} true https://anypoint.mulesoft.com 4.3.0 ${anypoint.username} ${anypoint.password} Sandbox yucelmoran-azure-pipeline Micro true ${env} deploy deploy deploy org.codehaus.mojo buildnumber-maven-plugin 1.1 buildnumber validate create {0,number} buildNumber false false unknownbuild org.mule.tools.maven mule-app-maven-plugin ${mule.tools.version} true org.apache.maven.plugins maven-install-plugin 2.5.2 org.apache.maven.plugins maven-deploy-plugin 2.8.2 org.mule.connectors mule-http-connector 1.5.17 mule-plugin org.mule.connectors mule-sockets-connector 1.1.6 mule-plugin org.mule.modules mule-soapkit-module 1.2.6 mule-plugin org.mule.modules mule-apikit-module 1.3.13 mule-plugin yucelmoran-azure-pipeline https://pkgs.dev.azure.com/yucelmorans/yucelmoran-azure-pipeline/_packaging/yucelmoran-azure-pipeline/maven/v1 true true MuleRepository MuleRepository https://repository.mulesoft.org/nexus-ee/content/repositories/releases-ee/ default true true mulesoft-release mulesoft release repository default http://repository.mulesoft.org/releases/ false mulesoft-release mulesoft release repository default http://repository.mulesoft.org/releases/ false MuleRepository MuleRepository https://repository.mulesoft.org/nexus-ee/content/repositories/releases-ee/ default ``` Basically, we are specifying where is our repository for source control, and then we specify where we want to deploy our application with the cloudHubDeployment configuration into the mule-maven-pluging tag. We can verify everything works using this command: ```bash mvn clean install -s settings.xml -e ``` ## Creating a pipeline Now we can create our pipeline configuration. Find the Pipelines options and select "Create Pipeline." ![Azure Pipelines section with the Create your first Pipeline prompt and button](../../assets/blog/mule-4-continuous-integration-using-azure-devops-14.png) Select the Azure Repos Git option and select your repository from your project. ![New pipeline Where is your code step with the Azure Repos Git YAML option](../../assets/blog/mule-4-continuous-integration-using-azure-devops-15.png) In the next step we can select Maven and then a template yaml file would be generated (we will change this eventually). ```yaml # Maven # Build your Java project and run tests with Apache Maven. # Add steps that analyze code, save build artifacts, deploy, and more: # https://docs.microsoft.com/azure/devops/pipelines/languages/java trigger: - master pool: vmImage: 'ubuntu-latest' steps: - task: Maven@3 inputs: mavenPomFile: 'pom.xml' mavenOptions: '-Xmx3072m' javaHomeOption: 'JDKVersion' jdkVersionOption: '1.8' jdkArchitectureOption: 'x64' publishJUnitResults: true testResultsFiles: '**/surefire-reports/TEST-*.xml' goals: 'package' ``` When you finish, the new yaml file will appear as part of your code. We need to modify the script, leaving it this way: ```yaml # Maven # Aurthor: Edgar Moran # Build your Java project and run tests with Apache Maven. # Add steps that analyze code, save build artifacts, deploy, and more: # https://docs.microsoft.com/azure/devops/pipelines/languages/java trigger: - test - emoran pool: vmImage: 'ubuntu-latest' steps: - task: Maven@3 condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/test')) inputs: mavenPomFile: '$(system.defaultWorkingDirectory)/pom.xml' mavenOptions: '-Xmx3072m' javaHomeOption: 'JDKVersion' jdkVersionOption: '1.8' jdkArchitectureOption: 'x64' publishJUnitResults: true testResultsFiles: '**/surefire-reports/TEST-*.xml' goals: 'clean deploy -Danypoint.username=$(ANYPOINT_USERNAME) -Danypoint.password=$(ANYPOINT_PASSWORD) -Denv=test -Dazure.password=$(AZURE_TOKEN) -Dnexus.username=$(NEXUS_UERNAME) -Dnexus.password=$(NEXUS_PASSWORD) -s settings.xml -e' - task: Maven@3 inputs: mavenPomFile: '$(system.defaultWorkingDirectory)/pom.xml' mavenOptions: '-Xmx3072m' javaHomeOption: 'JDKVersion' jdkVersionOption: '1.8' jdkArchitectureOption: 'x64' publishJUnitResults: true testResultsFiles: '**/surefire-reports/TEST-*.xml' goals: 'clean install -Dazure.password=$(AZURE_TOKEN) -Dnexus.username=$(NEXUS_UERNAME) -Dnexus.password=$(NEXUS_PASSWORD) -s settings.xml -e' ``` It’s important to set the variables we will use during deployment, for example: credentials to login to nexus repositories in case we need to, credentials for Anypoint Platform and the personal token we created in Azure. Once we create our pipeline we can select the option Variables. ![Pipeline YAML editor showing the generated azure-pipelines.yml with Maven trigger config](../../assets/blog/mule-4-continuous-integration-using-azure-devops-16.png) Finally, let's create as many as we need, according to what we need from our yaml file. ![Pipeline Variables panel listing Anypoint, Azure token, and Nexus credential secrets](../../assets/blog/mule-4-continuous-integration-using-azure-devops-17.png) Now the last piece of the puzzle is basically to send a change from source control to Azure on the first step (sending a change to the emoran branch, for example). ![Terminal git status on the emoran branch showing the modified azure-pipeline.xml file](../../assets/blog/mule-4-continuous-integration-using-azure-devops-18.png) ![Terminal output of git push origin emoran creating the new branch on Azure](../../assets/blog/mule-4-continuous-integration-using-azure-devops-19.png) We'll see our pipeline running for emoran. ![Azure pipeline run summary for the emoran branch with a job still running](../../assets/blog/mule-4-continuous-integration-using-azure-devops-20.png) When it finishes, we'll see a *clean install build* command for this branch. ![Pipeline Maven job log for the emoran branch finishing a clean install build](../../assets/blog/mule-4-continuous-integration-using-azure-devops-21.png) Now all happens in source control, so if we want to deploy our application to CloudHub, then, we need to promote our changes from emoran to the test branch. Let's create a Pull Request in Azure, from emoran to test: ![Azure DevOps New pull request form merging the emoran branch into test](../../assets/blog/mule-4-continuous-integration-using-azure-devops-22.png) Once we click on Create, then we need to complete the Pull Request and Merge Request in order to trigger the pipeline. When the pipeline that we ran for test finishes, we can see the deployment happened this time: ![Pipeline Maven job log for the test branch uploading the artifact and deploying](../../assets/blog/mule-4-continuous-integration-using-azure-devops-23.png) So now we can go to Anypoint Platform and see our app deploying. ![Anypoint Runtime Manager listing the yucelmoran-azure-pipeline app with Updating status](../../assets/blog/mule-4-continuous-integration-using-azure-devops-24.png) ![Runtime Manager detail panel showing the deployed CloudHub app in Started status](../../assets/blog/mule-4-continuous-integration-using-azure-devops-25.png) Finally, with this you can set as many branches you need and try other processes along. ## GitHub repository Hope this helps others to set their process. You can find the complete code [here](https://github.com/emoran/mule4-yucelmoran-azure-pipeline). Hope to hear from you on comments and let me know if this was helpful. --- ## Implementing Mapping Rules With MuleSoft Dedicated Load Balancer Source: https://prostdev.com/post/implementing-mapping-rules-with-mulesoft-dedicated-load-balancer | Published: Sep 22, 2020 | Category: Guides ## Introduction Dedicated Load Balancer is an optional component within the Anypoint Platform and it is used to route the HTTP and HTTPS traffic to multiple applications deployed to CloudHub workers in the VPC. To create a Dedicated Load Balancer, you must first create the Anypoint VPC which can be mapped to multiple environments and the same Dedicated Load Balancer can be used for different environments. You can use multiple DNSs for the same Dedicated Load Balancer (i.e. `api-dev.example.com` and `api-test.example.com`) ## Applying Mapping Rules on a Dedicated Load Balancer (DLB) Mapping rules are used on dedicated load balancers to translate input URI to call applications deployed on CloudHub. A pattern is a string that defines a template for matching an input text. Whatever value is placed within curly brackets ({ }) is treated as a variable. Variable names can contain only lowercase letters (a-z) and no other characters, including slashes. Let's consider that we have 2 DNS (i.e. `api-dev.example.com` and `api-test.example.com`) setup on a dedicated load balancer: `api-dev.example.com` is for the Dev environment whereas `api-test.example.com` is for the Test environment. ## Use Case 1 We are receiving requests on the DLB `https://api-dev.example.com/ecommerce/v1.0/invoice` and need to redirect them to `http://org-ecommerce-api.cloudhub.io/v1.0/invoice` (the CloudHub application name will be `org-ecommerce-api`) We can use this mapping rule to achieve this: ![DLB mapping rule table with an app path variable mapped to an org-app-api target, output /v1.0, protocol http](../../assets/blog/implementing-mapping-rules-with-mulesoft-dedicated-load-balancer-1.png) This previous rule will be applied when requests come on DLB and route to the CloudHub application in the VPC. ``` https://api-dev.example.com/ecommerce/v1.0/invoice (DLB) ``` is redirected to: ``` http://org-ecommerce-api.cloudhub.io/v1.0/invoice (CloudHub) ``` ![Use Case 1](../../assets/blog/implementing-mapping-rules-with-mulesoft-dedicated-load-balancer-2.png) But here we have some problems that on our DLB, we have set up 2 DNSs, one for Dev and another for Test. Now, how will the DLB know this is a request that needs to route to either the Dev or Test application because the same rule will be applied for both? To avoid this, we will be using a subdomain in the next use case. ## Use Case 2 In this case, we will be using a subdomain for routing the request to the correct environment from DLB. Our application name format must be org-app-subdomain (e.g. org-ecommerce-api-dev for dev environment and org-ecommerce-api-test for test environment) when deploying to CloudHub workers in VPC. So, our mapping rule will look like this: ![DLB mapping rule using app and subdomain variables in the target, output /v1.0](../../assets/blog/implementing-mapping-rules-with-mulesoft-dedicated-load-balancer-3.png) subdomain is a variable to map any subdomain. **1) dev environment** ``` https://api-dev.example.com/ecommerce/v1.0/invoice (DLB) ``` is redirected to: ``` http://org-ecommerce-api-dev.cloudhub.io/v1.0/invoice (CloudHub Dev Environment) ``` **2) test environment** ``` https://api-test.example.com/ecommerce/v1.0/invoice (DLB) ``` is redirected to: ``` http://org-ecommerce-api-test.cloudhub.io/v1.0/invoice (CloudHub Test Environment) ``` ![Use Case 2](../../assets/blog/implementing-mapping-rules-with-mulesoft-dedicated-load-balancer-4.png) In this use case, we solve the issue of routing the request from DLB to the correct environment. Let's consider another scenario where you want to route the request to CloudHub on the basis of the application version. We will see this in the next use case. ## Use Case 3 In this case, we will deploy an application to CloudHub and it will be in format org-app-subdomain-version (e.g. org-ecommerce-api-dev-v1-0 for Dev environment and org-ecommerce-api-test-v1-0 for Test environment). Whenever we get a request on the DLB, then the version in the URL will be v1.0 and v2.0 but when you deploy an application on CloudHub it doesn't allow you to use "." in the application name. That is the reason we are using "-" in the version of the application deploying to CloudHub. So, our mapping rule will look like this: ![DLB mapping rule with subdomain and version variables in the input and target paths](../../assets/blog/implementing-mapping-rules-with-mulesoft-dedicated-load-balancer-5.png) **1) dev environment** ``` https://api-dev.example.com/ecommerce/v1.0/invoice (DLB) ``` is redirected to: ``` http://org-ecommerce-api-dev-v1-0.cloudhub.io/v1.0/invoice (CloudHub Dev Environment) ``` **2) test environment** ``` https://api-test.example.com/ecommerce/v1.0/invoice (DLB) ``` is redirected to: ``` http://org-ecommerce-api-test-v1-0.cloudhub.io/v1.0/invoice (CloudHub Test Environment) ``` ![Use Case 3](../../assets/blog/implementing-mapping-rules-with-mulesoft-dedicated-load-balancer-6.png) ## DLB Mapping Rules Priority DLB will apply the first matching rule regardless of more exact matching rules available. A rule defined first, at index 0 has higher priority against other rules defined after it. The higher the index assigned, the less priority the mapping rule has. This is how you can apply mapping rules on MuleSoft's Dedicated Load Balancer. --- ## Securing APIs: Two ways to apply policies for CloudHub applications Source: https://prostdev.com/post/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications | Published: Sep 15, 2020 | Category: Tutorials > [!NOTE] > Anypoint Platform’s API Manager policies can only be applied to the applications which are deployed to CloudHub. For other deployment modes you need to have other ways of applying policies. This article helps you to apply policies via Anypoint Platform’s API Manager. We have 2 ways to apply policies: - Creating Proxy APIs - Without Creating Proxy APIs The main difference between the two of them is: - Using API ID in Auto-Discovery of your “actual” application, - Or Mule itself will create a Proxy application on top of your “actual” application which has API Auto-Discovery. ## 1) Without Creating Proxy API We will need these two things: - API ID generated in API Manager and used in the global element “Auto-Discovery” of your application. - Anypoint Platform’s Client ID and Client Secret. We can get them from “Access Management” Under Environments Tab > Choose the environment in which your application is getting deployed. Use them in Runtime properties while deploying the application in below way: ``` anypoint.platform.client_id : xxxxxxxxxxxx anypoint.platform.client_secret : xxxxxxxxxxxx ``` With the help of these two steps, we can now see in API Manager that the status of our API will turn from “Unregistered” to “Active”. **Step 1**: Design a RAML, publish the asset to Exchange, import the RAML to Anypoint Studio, and generate the flows. **Step 2**: Go to API Manager, Click on Manage API and choose “Manage API from Exchange”. ![API Manager "Manage API" dropdown with the "Manage API from Exchange" option highlighted](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-1.png) **Step 3**: Select your API, and select the managing type as “Basic Endpoint”. Don’t forget to check the box “Mule version: Check this box if you are managing this API in Mule 4 or above.” Click on Save. ![Manage API from Exchange form for BankDetails with managing type set to Basic Endpoint](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-2.png) ![BankDetails API instance page with status Unregistered and the API Instance ID highlighted](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-3.png) **Step 4**: Go to the Application in Anypoint Studio, Create a Global Element Auto-Discovery and Add API id and choose the flow name (usually Main flow where the request listens). ![Anypoint Studio API Autodiscovery global element with the API Id and Flow Name filled in](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-4.png) **Step 5**: Go to Access Management (the owner of the account can have access and can view this option. If you are not seeing this, that means you need to request access to the owner of the Anypoint Platform account). Choose the Environments Tab and select your environment. Get your Client ID and Client Secret. ![Anypoint Platform navigation menu with the Access Management option highlighted](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-5.png) ![Access Management Environments with the Sandbox "Edit environment" panel showing Client ID and Secret](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-6.png) **Step 6**: Go to Runtime Manager and add the Runtime properties like below and deploy. ![Runtime Manager Properties tab with anypoint.client_secret and anypoint.client_id set as text properties](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-7.png) Now your API in API manager status will turn to Active and you can start applying policies by going to Policies > Apply New Policy. > [!NOTE] > There’s no need to restart the application once the policy is applied. It will reflect within seconds automatically. Even if you add one or more policies at a later point of time, no restart of the application is required. ## 2) Creating Proxy API In the previous method there were a lot of manual processes required. To avoid all these, we can go for this second approach to create a Proxy application. This Proxy application is nothing but a kind of wrapper to your main application. Now for each application, we will have 2 apps in Runtime Manager: - Your Actual Application - Your Proxy Application, which has API ID, Client ID, and Client Secret within it. There’s no manual effort required to deploy. It’s deployed from the API manager. **Step 1**: Design a RAML, publish the asset to Exchange, import the RAML to Anypoint Studio, and generate the flows. **Step 2**: Go to API Manager, Click on Manage API and choose “Manage API from Exchange”. **Step 3**: Select your API, and select the managing type as “Endpoint with Proxy”. Don’t forget to check the box “Mule version: Check this box if you are managing this API in Mule 4 or above.” Click on Save. **Step 4**: Give the Implementation URL of your actual application that is deployed. Make sure you provide your complete path, up until /api (all RAML-generated flows have a common base path as /api) E.g. ``` http://myfirstapp.cloudhub.io/api ``` Once done, click on Save. ![Proxy setup form with deployment target CloudHub selected and the Implementation URI field filled in](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-8.png) **Step 5**: Now you can see the Deployment Configuration. Choose the runtime and name of the Proxy app you want to give and click on deploy. The application will be deployed in the Runtime Manager. ![API Manager Deployment Configuration with runtime version 4.2.0 and a proxy application name set](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-9.png) **Step 6**: Now edit the Consumer endpoint in API Manager with your Proxy endpoint. So that all the requests will hit the proxy application. Basically your implementation URL is your actual endpoint and Consumer endpoint is the endpoint for the Proxy. In other words, the client hits the Proxy endpoint, which in-turn is routed to the implementation URL. ![Active API instance showing the implementation URL, the consumer endpoint, and the deployed proxy URL](../../assets/blog/securing-apis-two-ways-to-apply-policies-for-cloudhub-applications-10.png) You can see that the Client ID and Client Secret values are picked automatically and placed in Runtime properties. You can also download the jar of the Proxy application and import it in Anypoint Studio, where you can see the API ID is automatically configured and the HTTP request will route to your main app. You have to provide the endpoint of the Proxy application to consumers/clients so that they don’t have access to your main app. The policies are applied to your Proxy application. But the thing here is, we need additional vCore and workers to deploy our Proxy app. According to your requirements, you can choose which method you want to follow to apply policies for your CloudHub apps. ## Pros and Cons Without Proxy Application: - You will save a vCore and worker. - As a developer, you need to know the Client ID and Client Secret - unless the deployment is managed by CI/CD process. - You need to manually create a global element to invoke Auto-Discovery. - You will expose your actual endpoint to the Client/Consumer. - Use case: If your APIs are used within the organization, within intranet, you can choose this option where we can share the endpoints directly. This saves vCores. With Proxy Application: - You will need additional vCore and worker for each Proxy app you deploy. - As a developer, you don’t need to know the Client ID since it’s managed internally. - No need for the manual process of using API ID or using Client ID and Client Secret. - You will expose your Proxy endpoint over the actual endpoint to the Client/Consumer. - Use case: If your APIs are used outside of your organization, you can choose this option where we can share the endpoints of the Proxy app instead of the actual endpoint. This gives you more security. These are 2 simple ways to apply policies! Happy learning Sravan Lingam --- ## Things to consider before writing a technical blog post Source: https://prostdev.com/post/things-to-consider-before-writing-a-technical-blog-post | Published: Sep 8, 2020 | Category: Guides Congratulations on taking the first step towards writing! You may have chosen to do this because you’re curious about trying something new, or maybe you feel relaxed when writing and you want to take up a new hobby. Maybe you noticed that it could be a good thing on your resume, or maybe it’s just a great opportunity for you to acquire new knowledge and skills for your professional career. Whatever your reason for writing may be, I just want to let you know that you’re not alone. A lot of us have felt like writing, but were not sure where to start or what to do exactly. For this post, I will give you some tips on how you can get started and get your first article out! ## Choosing what to write about > [!NOTE] > When I say “technology,” it can mean a programming language, a software development tool, a mobile application, a computer program, etc. Basically, it is anything related to Software. (At least in ProstDev) you don’t need to be an absolute expert on a technical topic to be able to write about it. It would help if you had more information about it, in case you need to explain certain details for the article, but you can also write about your own personal experience with a technology. Think about any technology that makes you happy, or that you really enjoy using. Maybe quite the opposite, think about a technology that was hard to use and understand and create a blog post about everything you learned through your experience to make someone else’s life easier. You can start by asking yourself some questions to get ideas about a technology or an experience that you’d like to share with a wider audience: - Was there a technology in your professional career that you always wanted to learn more about, but you never did? - Was there a technology that was exceptionally fun to learn? - Was there a technology that you struggled *a* *lot* to learn, but in the end, it proved to be super useful? - Was there a technology that you truly didn’t want to learn, but then you kept seeing it as a requirement for other job positions? - Was there a technology that you were really terrible at, but you still thought it was interesting and wanted to learn more about it? - Is there a technology that you thought wasn’t going to be that important when you first heard about it, but now it’s imperative to know it for your current role? - Is there a technology that you want to keep using for your software development projects because it’s very useful? - Is there a technology that you thought was *nearly* perfect, but you stopped using it because of those small flaws? - Is there a technology that you thought would be revolutionary for the industry, but then everyone stopped using it, and it became “unimportant”? - Is there a technology that you keep suggesting to use in your project, but your company or the client won’t let you make this change? - Do you excel in a specific type of technology? - Was there a technology in your university/college days that you absolutely enjoyed and were good at, but then you never got to use it in the professional world? - Was there a technology you learned during your internship(s) or your first job(s)? - Is there a specific topic from any technology that you really struggled to understand and had to do a lot of research to master it? - Is there a specific problem you encountered using any technology, and you couldn’t find any documentation online to solve it, but in the end, you were able to troubleshoot it on your own? I hope that you have a list of technologies or features in your mind now! If not, just keep asking yourself more questions and you’ll get to a technology or a topic that would make a great blog post. If that doesn’t work, you can also just start by writing about the technology you’re currently using, whether for software development, architecture, design, management, or even some methodologies that you use for your team like Scrum, Kanban, or Waterfall. Choose the first topic you want to write about, and let’s go to the next step! ## Who to write to (a.k.a. your target audience) Now you need to start thinking about who will be reading your article. It’s good to create a blog post just for the sake of writing, but it’s way better if it’s actually useful for other people. One way to create better content is to define the target audience; this way, you will have a specific group of people interested in reading your post. Ask yourself, who will read this article? Try to be as specific as you can, so you can also write your content accordingly. Here are some more questions you can ask yourself to define your audience: - Is this for developers? What kind of developers? (Front-end, back-end, integration, data scientists, etc.) - Is this for architects? - Is this for business people? Are they project managers? - Are these people new to the technology? Are they just starting to learn it? - Do these people hold senior positions, or have a lot of years of experience in their position? You’re not going to write complex code or technical details if your target audience is someone from the business side. You’re not going to give an introduction to a programming language if you’re writing for senior developers - they will be expecting lines of code to see the solution! One trick that I do to stay focused on writing for my target audience is to actually imagine someone that I know from that area and write as if I was face to face with them explaining my article. This helps me add some definitions that they may not be familiar with, so they can follow the rest of the post (for basic posts). Or this keeps me focused on *not* explaining the basics and just get straight to the point (for advanced topics). Now we know what to write about and who to write the blog post for. In my next post, I will give you more tips to actually start writing your article. So, stay tuned for things like picking a title, organizing the post’s format, or adding additional content. Subscribe to ProstDev to receive an email as soon as this new content is published! *Prost!* -Alex --- ## Understanding APIs (Part 2): API Analogies and Examples Source: https://prostdev.com/post/understanding-apis-part-2-api-analogies-and-examples | Published: Sep 3, 2020 | Category: Guides In Part 1 of this series, “[Understanding APIs (Part 1): What is an API?](https://www.prostdev.com/post/understanding-apis-part-1-what-is-an-api),” we learned about the acronym, the definition, the four aspects of an API, and how everything works together. We also defined this cute little diagram below. If you feel like you forgot some pieces, you can always go back and read the recap at the end for a quick summary. ![Diagram: a Request input flows through an Implementation box to a Response output](../../assets/blog/understanding-apis-part-2-api-analogies-and-examples-1.png) When I started learning what an API was, it took me some time to really start understanding this concept. It can become so abstract! When I look at practical examples of applied technologies, it can really open my eyes to understanding a theory. In the case of APIs, looking at different examples helped me get a better idea of this concept. This post will complement what you learned in Part 1. You will learn about some analogies and real-life examples of APIs to understand better what they are. ### API Analogies Before I begin, I just want to get something clear: when you are using or *consuming* an API, you don’t really care about how the Implementation works. It doesn’t matter if the Implementation is built on Java, Python, or MuleSoft; it’s still going to give you the Response that you are expecting according to your Request. You don’t care about what systems the API needs to call, if any, or if the API is using JSON or CSV as the internal Data Type. None of this is really visible to you anyways. You just want your Request to be processed and your Response to be returned. **1. Restaurant** Imagine you arrive at a restaurant. You sit in a chair and start looking at the menu. Then the server comes to ask for your order. You say that you want a hamburger because it was one of the choices that you selected from the menu. The employee goes to the kitchen, and someone cooks your meal. Sometime later, the server comes back with your hamburger. *“How is this an analogy for an API?”* You, the client, give a Request to the server: you want to receive a hamburger. The server does whatever needs to be done to deliver the order. Then, you receive a Response with what you asked for: a hamburger. ![Restaurant analogy: a client sends a menu request and a server returns a dish response](../../assets/blog/understanding-apis-part-2-api-analogies-and-examples-2.png) Notice how I didn’t add the part of the server going to the kitchen, or the chef cooking your meal. This is because we only care about what our order was and what was delivered to us. **2. Calculator** Now, let’s say that a regular calculator is an API. You send a “2+2” Operation as a Request, hit the equal button, and receive a “4” as a Response. ![Calculator analogy: a 2+2 request goes into a calculator and returns 4 as the response](../../assets/blog/understanding-apis-part-2-api-analogies-and-examples-3.png) Once more, you’re not interested in how the calculator processed this information; you just want an accurate response to the Inputs and Operation that you sent in the Request. **Quiz**: What would be the Data Types in the RQ and RS in this scenario? *The answer will be revealed in the next post.* ### API Examples Now let’s see some examples of how APIs can be used in real-life scenarios. **1. Human Resources API** You started by being a customer at a restaurant, then a calculator user, and now you are the head of the Human Resources (HR) department at FictionalCorp. Hooray! As the HR head manager, you obviously know how to use the HR API to process the company’s information regarding the employees. You want to check what the employees’ first day of employment at FictionalCorp was, and you already know that the HR API accepts Requests in the format of a CSV file. Your Request would look like this: ![CSV request with two employees and an operation to get their first day of employment](../../assets/blog/understanding-apis-part-2-api-analogies-and-examples-4.png) There’s a CSV Data Type, the Input information of the two employees (*we’re a startup, ok?*), and the Operation telling the API what to do with this data. You click “Send,” and off it goes! Within seconds the API returns the Response with the information you requested: ![CSV response adding a highlighted First Day column with each employee's start date](../../assets/blog/understanding-apis-part-2-api-analogies-and-examples-5.png) It returns the same Data Type (CSV format) and an Output file that contains the same information you sent in the Request, PLUS, a new field called “First Day,” which contains each of the employees’ first day of employment at FictionalCorp. You may not know if the API is connected to a Database or to an external system where this information is kept, but it returned what you requested in the first place, which is all you need. What would happen if the RQ/RS was in a different format, you say? Well, this HR API also accepts the [JSON](https://www.json.org/json-en.html) Data Type. Here’s how that would look: ![Same HR request and response shown as JSON, the response adding a FirstDay field](../../assets/blog/understanding-apis-part-2-api-analogies-and-examples-6.png) We have the same Inputs, same Operation, and same Output, but the Data Types for both the Request and the Response are different. **2. Real-life APIs** Do you want to see some actual APIs? Follow these links to see their documentation: - [Twitter API](https://developer.twitter.com/) - [Slack API](https://api.slack.com/) - [Google Maps API](https://developers.google.com/maps/documentation) - [Facebook APIs](https://developers.facebook.com/) It’s ok if you don’t understand everything that you see in these pages. This is just to give you an idea of what real APIs look like. ## Recap To recap, when you’re calling or *consuming* an API, you don’t really care about how the Implementation works; you just want your Request to be processed and a Response to be returned according to the Inputs and Operations that you sent in the RQ. Stay tuned for Part 3 of this series if you want to know the answer to the quiz in the second analogy: the calculator. The question was: *What would be the Data Types for the RQ and RS in this scenario?* I hope you are starting to get a better picture of what APIs are. Contact me or leave me some comments if you have any additional questions - I’m happy to help. Keep learning and don’t give up! *Prost!* -Alex --- ## Creating a connector and consuming it from a Mule Application Source: https://prostdev.com/post/creating-a-connector-and-consuming-it-from-a-mule-application | Published: Sep 1, 2020 | Category: Tutorials ## What is a Connector? An Anypoint Connector is a reusable component that interacts with Mule runtime and Anypoint Studio. A connector communicates with a target resource and conveys information between a resource and Mule, and transforms the data into a Mule message. [Anypoint Connector DevKit](https://docs.mulesoft.com/connector-devkit/3.9/) abstracts the communication that happens between Mule and the external service or API, and generates a user interface to help simplify usage of the connector by the developer who would eventually use it in their application. A well-developed connector makes the development of a Mule application much easier for users when handling tasks like pagination, session expirations, and input and output metadata. This tutorial shows you how to create a well-designed connector. ## Why create a Custom Connector? In our MuleSoft journey we, as developers, may have used different connectors when creating Mule applications. We have a good number of connectors available out-of-the-box for us to use. There could be a scenario where no out-of-the-box connector is available and we need to follow a series of complex steps (i.e. authorizing the client, getting access token, implement caching, calling database, etc.) to actually call a third party API. In this scenario, it is recommended to use a custom connector which will be responsible for calling the third party API and hiding the complexity altogether. Consumer API would otherwise have no idea how that particular third party API is called by the connector. ## Creating a Custom Connector In Mule 3, we had a separate API Dev Kit to create an Anypoint Connector Project directly from Anypoint Studio. However, with Mule 4, that is no longer required. Creating a connector project has become simpler than ever. **Prerequisites**: - Java JDK 1.8 - Maven Apache Plugin Run the following command with maven. This should create a basic template for the connector project with test cases. This will create a project with sample operations and basic structure with required files. However, you can change the implementation anytime according to your needs. ```bash mvn org.mule.extensions:mule-extensions-archetype-maven-plugin:1.2.0:generate ``` This will ask for some basic parameters like name and version of the project. If you don’t want to specify anything just press Enter or the Return key (for Mac users) leaving it empty. Maven will take the default values for that. Some important parameters are: - **Name of the extension**: specify the name you want to set for the connector. - **GroupId**: If you are planning to deploy it to Exchange, you need to provide Organization Id as groupId. If your extension is certified by Mule, then you can use `org.mule.modules` which is the default value in case you do not provide anything. - **Version**: you can provide a version number here. If you leave it empty, it will take 1.0.0 - SNAPSHOT as the default value. - **ArtifactId**: Provide a name for the artifact. You can keep the same as the extension name (the first point). - **Package**: You can specify the Java package namespace here. If you leave it empty, it will take the default one: `org.mule.extensions.project.name.internal` for Java implementation and `org.mule.extensions.project.name.test` for testing. ![Maven archetype command output prompting for connector name, groupId, and version, ending in BUILD SUCCESS](../../assets/blog/creating-a-connector-and-consuming-it-from-a-mule-application-1.png) ## How to Obtain Organization Id To get the Organization Id for your Anypoint Platform instance; log-in to your Anypoint Platform Account, navigate to Access Management >> Organization, and click on the name of the business group. A new pop-up window will open where you can find the Organization Id. ![Anypoint Access Management Organization info dialog with the Organization Id highlighted](../../assets/blog/creating-a-connector-and-consuming-it-from-a-mule-application-2.png) ## Configure the POM File for the Connector This step is crucial. We will make some changes in the connector’s POM file to make it ready for publishing to Anypoint Exchange. First add a mule-maven-plugin like below: ```xml org.mule.tools.maven mule-maven-plugin ${mule.maven.plugin.version} true mule-application-example ``` Then create a profile with the exchange distribution repository information. ```xml exchange ExchangeRepository Corporate Repository https://maven.anypoint.mulesoft.com/api/v1/organizations/${groupId}/maven default ``` Please note, `groupId` is a variable which is extracted from the groupId properties from this section of the POM file. ```xml 4.0.0 c5f216cc-80a0-45ee-9721-8a0809356e37 mule-test-connector 1.0.5 mule-extension mule-test-connector 3.3.5 ``` Finally, you need to add the server section in the `settings.xml` file from your `./m2` folder. You can use your username and password. Please note that the password is encrypted, because I am using `settings-security.xml` that contains an encrypted master password which is used for decryption of server password. The details for encrypting your password can be found in this amazing blog post: [How to Encrypt your Maven Password](https://dev.to/scottshipp/how-to-encrypt-your-maven-password-408d). However, for testing, you can stick with a plain password if you want to. ```xml ExchangeRepository YourUserName {PCdHLVg1NrFdHY+ighZBQF4/DlJYCjxYxA=} ``` Please note that `server.id` should be the same as the `repository.id` mentioned in the profile section above. ## Java Project Structure Once you create your Mule connector project which is basically a Java project and can be opened in any Java [IDE](https://www.prostdev.com/post/5-reasons-why-you-need-an-ide) (i.e. IntelliJ Idea). Once you open the project in a Java IDE, you should see the project structure as shown in the picture. ![IDE project tree for mule-test-connector showing the Configuration, Extension, and Operations classes](../../assets/blog/creating-a-connector-and-consuming-it-from-a-mule-application-3.png) Please note that `org.mule.extension.mule.test.internal` is your Java package namespace that was mentioned during the creation of the connector project. Any connector project has the 3 following class files: - **<project-name>Configuration**: It contains the configuration parameters. Whatever parameters we mention here will be used to create the connector’s configuration in Mule. - **<project-name>Extension**: This is the main extension file and starting point of the connector. - **<project-name>Operations**: This file contains the methods for the connector’s operations to do the specific task. This contains the actual connector operation’s implementation. Each connector operation is actually a method in this class file. ![Java code for the connector's HelloWorld and Calculate operations annotated with @MediaType and @DisplayName](../../assets/blog/creating-a-connector-and-consuming-it-from-a-mule-application-4.png) If you want unit tests designed for a connector, this project will contain some additional files for that purpose. ## Publish and Consume the Connector from MuleSoft Once we have everything in place in our Java Connector project, we need to deploy this to Anypoint Exchange. ```bash mvn deploy -Pexchange -DskipTests ``` Here with the -P parameter, we need to specify the profile name as mentioned in the POM file. ![Command prompt running mvn deploy -Pexchange to publish the connector](../../assets/blog/creating-a-connector-and-consuming-it-from-a-mule-application-5.png) This should deploy the connector in Anypoint Exchange. You can see the asset in Anypoint Exchange as shown in the screenshot. ![Anypoint Exchange Assets view showing the published Mule-test-Extension connector by Jyoti Choudhury](../../assets/blog/creating-a-connector-and-consuming-it-from-a-mule-application-6.png) Now it’s time to consume that connector from the Mule project. We simply start by adding dependencies in the POM file. ```xml c5f216cc-80a0-45ee-9721-8a0809356e37 mule-test-connector 1.0.5 mule-plugin ``` Please change the groupId, artifactId, and version based on your connector project POM file. If everything is in place, maven dependencies will be resolved and the connector will be added as a component. Finally, we can use this custom connector like any other out-of-the-box connector that’s available in the market. ![Studio flow with Listener, the custom Hello world connector operation, and Transform Message](../../assets/blog/creating-a-connector-and-consuming-it-from-a-mule-application-7.png) ## Conclusion We have learnt so far from this post: - What is Connector and why do we use Custom Connector? - Learn to create Custom Connector project. - Learn to publish and consume Custom Connector from MuleSoft API. Hope this has helped you. If you have any questions, please feel free to post your questions. ## References - [https://docs.mulesoft.com/connector-devkit/3.9/devkit-tutorial](https://docs.mulesoft.com/connector-devkit/3.9/devkit-tutorial) - [https://dzone.com/articles/how-to-publish-mule-custom-connector-to-anypoint-e](https://dzone.com/articles/how-to-publish-mule-custom-connector-to-anypoint-e) - [https://javastreets.com/blog/publish-connectors-to-anypoint-exchange.html](https://javastreets.com/blog/publish-connectors-to-anypoint-exchange.html) Thanks. Jyoti --- ## Understanding APIs (Part 1): What is an API? Source: https://prostdev.com/post/understanding-apis-part-1-what-is-an-api | Published: Aug 27, 2020 | Category: Guides I know that the concept of an “API” can be a bit hard to understand. It’s such an abstract thing! If you’re in technology, you probably have heard already about these APIs, but you may not fully understand what it means or what it is just yet. Hey, don’t worry. The same thing happened to me! It took me some time to really figure out what these are. For this post, I’ll give you a simple explanation and a cool diagram for you to refer to whenever you get confused with the terminology. ## What *is* an API? First of all, API is an acronym for Application Programming Interface. You can read the complete wiki definition [here](https://en.wikipedia.org/wiki/Application_programming_interface). Now that the textbook definition is out of the way, let me start by stating that an API can be created in different programming languages or different technology tools. Let’s call the “body” of the API, the *Implementation*. ![Examples of programming languages and tools that can be used to create an API.](../../assets/blog/understanding-apis-part-1-what-is-an-api-1.png) An API is composed of 4 specific aspects (plus the Implementation of the *actual* API). - Inputs - Operations - Outputs - Data Types **1. Inputs** The Inputs for the API are pieces of data or information that you can send to the Implementation. **2. Operations** This is where you define what you expect the API to do with the Inputs you will be sending to the Implementation. **3. Outputs** These are also pieces of data or information, but instead of *sending* these to the API (or Implementation), you will be *receiving* these *from* the API (or Implementation). **4. Data Types** The data or information that you send to the API and that you receive from it must be of some type. It can be numbers, words, an Excel file, a Java Object, a CSV file, a JSON or XML format, etc. You can send, for example, some input numbers and tell the API which operation you expect it to process, and get some more numbers in return as the output. Maybe you send your name as an input, and you receive a “Hello” plus your name as the output. All these formats are the Data Types. ## Defining an API's Request and Response **Request** The combination of Inputs (along with their Data Types) and Operations is what we call a “Request.” In other words, everything that you send to the API or the Implementation is a Request (RQ). **Response** The Outputs from the API or Implementation, as well as their Data Types, are what we call a “Response.” Put simply, everything sent back from the API to you, the caller, is a Response (RS). ## Putting it all together ![Diagram: Request (Input with Data Type + Operation) flows into Implementation, returning a Response (Output with Data Type).](../../assets/blog/understanding-apis-part-1-what-is-an-api-2.png) The Request is formed by the Inputs we want to send to the API. This includes their specified Data Types and the specific Operation that we want the API to process and return to us. This Request is sent to the Implementation of the API, which is processed by the code, and then the Outputs with the specified Data Types are returned to us. ## Um… Ok… Not quite sure what an API is yet? APIs are a very abstract concept, as I mentioned before, and I wouldn’t blame you if you went through this complete post and still felt confused as to what an API is. Not to worry, my friend! I thought of that too. I wanted to have this explanation clarified first, so you could at least keep in mind this last diagram. Save this picture somewhere because you may need to re-reference it as you keep learning. As you may have guessed from the title, this is only Part 1 of this series! We will continue to go on a journey through the wonders and philosophies of APIs. [Part 2](https://www.prostdev.com/post/understanding-apis-part-2-api-analogies-and-examples) will give you some analogies and examples of APIs. This way, you’ll end up with a better picture of this weird, but needed, concept. Let me just finish up with a quick recap. You are allowed to forget what you learned; it happens to everyone! The important thing is to keep trying and don’t give up. ## Recap - API is an acronym for Application Programming Interface. - The “body” of the API is also called the *Implementation*. - APIs are composed of Inputs, Operations, Outputs, and Data Types. - The Request is what we *send* to the API: Inputs with their Data Types and Operations. - The Response is what the API *returns* to us: Outputs with their Data types. That’s it for now! You can subscribe to this blog to receive an email as soon as new content is published. You can also follow ProstDev on social networks to see more of our content. :) *Prost!* -Alex --- ## Reviewing Sorting Algorithms: Merge Sort Source: https://prostdev.com/post/reviewing-sorting-algorithms-merge-sort | Published: Aug 25, 2020 | Category: Guides Let’s sort it out! In a series of posts, we will be discussing some of the sorting algorithms listed in the below order: - [Bubble Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-bubble-sort) - [Selection Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-selection-sort) - [Insertion Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-insertion-sort) - **Merge Sort** - Quick Sort - Heap Sort As promised, we are back to explore the next in a series of sorting algorithms, the Merge Sort. Till now, we have reviewed three sorting algorithms as listed above, to conclude that the common traits they possess are inefficiency and slowness. Bubble Sort, Selection Sort, and Insertion Sort algorithms have minor differences among themselves with the same quadratic running time; meaning the time complexity of these three algorithms is O(n^2). In this post, we shall determine if Merge Sort is any better than its already discussed peers, and does it fit the bill to be chosen as "The One"? Let’s find out. ## Why is it called Merge Sort? Merge Sort is a[divide and conquer algorithm](https://en.wikipedia.org/wiki/Divide_and_conquer_algorithm) that was invented in 1945 by [John von Neumann](https://en.wikipedia.org/wiki/John_von_Neumann), who was a founding figure in computing. The Merge Sort algorithm sorts an input collection by breaking it into two halves. It then sorts those two halves and merges them together as one sorted array. Most of the merge sort implementations use divide and conquer, a common algorithm design paradigm based on [recursion](https://en.wikipedia.org/wiki/Recursion_(computer_science)). The idea behind the merge sort is, it’s easier to sort sublists or smaller sets and combine them rather than sorting the long list of an array’s items. In our [previous post for Insertion Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-insertion-sort), we have clearly identified the common issue in all the discussed sorting techniques as sorting a long list of values that had slower and inefficient runtimes, with a quadratic time complexity O(n^2). Let’s first understand the divide and conquer algorithm. > “*A divide-and-conquer algorithm works by recursively breaking down a problem into two or more sub-problems of the same or related type until these become simple enough to be solved directly. The solutions to the sub-problems are then combined to give a solution to the original problem.*” [[Ref. 1](https://en.wikipedia.org/wiki/Divide-and-conquer_algorithm)] A divide-and-conquer algorithm divides the problem into the simplest form. The smaller problems are the ones most easier to solve. The solution is then applied to the bigger chunks of the complex problem. Hence, the larger problem is conquered by using the same solution recursively. The three parts of the divide and conquer algorithm are as below: **Divide**: The problem is divided into the smallest possible number of the same problem; meaning the problem is divided into a subproblem, which indeed is divided into another subproblem and so forth until the smallest set is achieved. **Conquer**: The smallest subproblem is conquered first by finding the solution as a base case. Once the solution is achieved, the same technique can be used to tackle bigger subproblems recursively. **Combine**: The solution to the small problem is combined and build-up to solve the bigger problem. The below pictures demonstrate the divide-and-conquer algorithm, wherein a problem is divided into two subproblems and the algorithm makes multiple recursive calls. ![Fig1.1 Divide-and-Conquer demonstration - Khan academy [Ref. 2]](../../assets/blog/reviewing-sorting-algorithms-merge-sort-1.png) Further simplification of the recursive steps is demonstrated below: ![Fig1.2 Divide-and-Conquer demonstration - Khan academy [Ref. 2]](../../assets/blog/reviewing-sorting-algorithms-merge-sort-2.png) At this point, we have better clarity on how the Divide-and-conquer algorithm works in theory, it’s time to illustrate this design paradigm implementation in the Merge Sort. The idea is to discuss the Merge Sort in a very simple way so that it sticks well in our memory, very much like solving small problems to achieve the original big problem solution. We have now cemented in theory how the Merge Sort will split the collections in a number of sublists to find the basic solution. Let’s break it down with our post series sample array [10,8,4,6,1,3,2,5]. Additional element number [3,2,5] is added to the array to make it an even array for easier demonstration purposes. **Divide**: The array is divided into two halves initially and each half is again divided into halves until the smallest subset is attained. Below is the illustration for the divide step and we achieve a sorted smallest sub list at the end. You may ask, how is it sorted? Well, the final eight sublists are considered sorted as there is only one value and nothing else to compare. In short, the smallest sublist is always a sorted list by itself and considered as a base case. Now that we have solved the smallest problem, the next step would be to solve the next biggest subproblem. How do we do that? This is the time we introduce the “conquer” part of the algorithm. ![Merge sort divide step splitting an 8-element array into eight single-element sublists](../../assets/blog/reviewing-sorting-algorithms-merge-sort-3.png) **Conquer**: The sorted sublists are merged together maintaining the sorted list. Two sorted lists are required to merge together and create one single sorted list. The sorted lists are merged together as a part of the “*combine*” step recursively building towards the final single sorted list. The base case is the first single element sorted list. If you observe closely, the sorting has taken place recursively for each of the below sublists. ![Merge sort conquer step merging sublists back into one fully sorted array](../../assets/blog/reviewing-sorting-algorithms-merge-sort-4.png) We discussed the Divide-and-conquer algorithm on how it’s implemented in the Merge Sort and tried to simplify the workings of the Merge Sort, in theory. Let’s dive into code snippets to understand how it’s implemented in real-time. *Okay, but what about recursion?* We have used the reference multiple times in our discussion. [Recursion](https://en.wikipedia.org/wiki/Recursion_(computer_science)) is a method to solve the same problem by breaking it down into smaller problems until a base condition (ex: if n<= 1) is satisfied to stop the recursion. A recursive function is one that calls itself during execution directly or indirectly to achieve the desired solution. ## Pseudocode logic: - The recursive approach is followed to implement the Merge Sort. The first step is to find the mid-position of the array to split the array into two halves. Mid position for an array of ‘n’ size is found as ‘n/2’. Alternatively, (L+R)/2 would also yield the mid-position of the array where L is the left index, and R is the right index value. - If the input array length is less than 2 then no action will be performed, as this will be considered as a sorted single element sublist. - The array is split into two sublists of size ‘n/2’ until no further division can be performed. The split function ‘mergesort’ is invoked recursively to get the left and right sublist until a single element list is formed. - For each split function, ‘mergesort’ will invoke the function ‘merge’ each time recursively. For example, array [10,8,4,6,1,3,2,5] can be divided into two, four, and eight sublists, making it *mergesort(left), mergesort(right)* function call. - Once the single sublists are formed, the next step is to invoke the function ‘merge’ passing the input parameters left, right, and array. The single sublist is considered as a sorted list as there are no other values to compare, making it as our base case. - The logic of function *“merge“* implements the sorting and merging of the sublists. For example, single arrays [10] [8] are sorted and merged together as [8,10]. Similarly, backtracking other array values we attain the final sorted list. - The elements from the left array and right array indexes will be compared for sorting. In our example, the left array [8,10] and right array [4,6] the index[0] in both lists are compared to find the sorted position for the new sublist as [4,6,8,10]. ## Code Implementation: ```python #function for merging the left and right array. Sorting will be performed def merge(left, right, array): print(" ") print(" Merge -> array : {} Left : {} Right : {} ".format(array,left,right)) # Iterator i for traversing through the Left Half of the array i = 0 # Iterator j for traversing through the Right Half of the array j = 0 # Iterator k for the main list k = 0 while (i> Merged array is : {} ".format(array)); print(" ") return array ###################################################### ### Function for dividing and calling merge function # ###################################################### def mergesort(array): arrlen = len(array) if(arrlen<2): print(" array has only one element {} - no further action ".format(array)) return # derive the midpoint of the array mid = int(arrlen/2) # divide the array into left Half and Right Half based on the midpoint left = array[:mid] right = array[mid:] print(" Left Half :{} Right Half : {}".format(left,right)) print(" ") print(" Recursive call mergesort for left array {} ".format(left)) print(" ---------------------------------------------------------") mergesort(left) print(" ----------------------------------------------------------") print(" ") print(" Recursive call mergesort for Right array {} ".format(right)) print(" ----------------------------------------------------------") mergesort(right) print(" ----------------------------------------------------------") # print( " After MergeSort Right :{} ".format(right)) merge(left,right,array) return array # test driver code if __name__ == '__main__': # Test Long unsorted list # arrlist = [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] # arrlist = [] # Test short unsorted list arrlist = [10,4,8,6,1,3,2,5] #Test sorted list # arrlist = [1,2,3,4,5] #Test n= 100 unsorted list #arrlist = [52,68,84,73,3,81,91,58,1,19,8,78,32,94,4,98,23,6,21,20,43,69,31,87,33,62,36,41,70,51,48,38,72,24,47,57,13,7,64,49,9,16,14,99,90,92,35,25,37,89,50,85,76,67,88,131,39,56,74,42,77,26,93,97,80,82,96,60,53,46,18,65,30,27,2,83,66,29,61,10,55,22,11,75,95,45,59,86,12,15,17,79,40,34,5,71,44,63,28,54] print(" Input array : {}".format(arrlist)) print(" Output array :{} ".format(mergesort(arrlist))) ``` **Log Output:** ```text ############# Test Case 1 ############# Input array : [10, 4, 8, 6, 1, 3, 2, 5] Left Half :[10, 4, 8, 6] Rigth Half : [1, 3, 2, 5] Recursive call mergesort for left array [10, 4, 8, 6] --------------------------------------------------------- Left Half :[10, 4] Rigth Half : [8, 6] Recursive call mergesort for left array [10, 4] --------------------------------------------------------- Left Half :[10] Rigth Half : [4] Recursive call mergesort for left array [10] --------------------------------------------------------- array has only one element [10] - no further action ---------------------------------------------------------- Recursive call mergesort for Right array [4] ---------------------------------------------------------- array has only one element [4] - no further action ---------------------------------------------------------- Merge -> array : [10, 4] Left : [10] Right : [4] *-*-* End of merge function >> Merged array is : [4, 10] ---------------------------------------------------------- Recursive call mergesort for Right array [8, 6] ---------------------------------------------------------- Left Half :[8] Rigth Half : [6] Recursive call mergesort for left array [8] --------------------------------------------------------- array has only one element [8] - no further action ---------------------------------------------------------- Recursive call mergesort for Right array [6] ---------------------------------------------------------- array has only one element [6] - no further action ---------------------------------------------------------- Merge -> array : [8, 6] Left : [8] Right : [6] *-*-* End of merge function >> Merged array is : [6, 8] ---------------------------------------------------------- Merge -> array : [10, 4, 8, 6] Left : [4, 10] Right : [6, 8] *-*-* End of merge function >> Merged array is : [4, 6, 8, 10] ---------------------------------------------------------- Recursive call mergesort for Right array [1, 3, 2, 5] ---------------------------------------------------------- Left Half :[1, 3] Rigth Half : [2, 5] Recursive call mergesort for left array [1, 3] --------------------------------------------------------- Left Half :[1] Rigth Half : [3] Recursive call mergesort for left array [1] --------------------------------------------------------- array has only one element [1] - no further action ---------------------------------------------------------- Recursive call mergesort for Right array [3] ---------------------------------------------------------- array has only one element [3] - no further action ---------------------------------------------------------- Merge -> array : [1, 3] Left : [1] Right : [3] *-*-* End of merge function >> Merged array is : [1, 3] ---------------------------------------------------------- Recursive call mergesort for Right array [2, 5] ---------------------------------------------------------- Left Half :[2] Rigth Half : [5] Recursive call mergesort for left array [2] --------------------------------------------------------- array has only one element [2] - no further action ---------------------------------------------------------- Recursive call mergesort for Right array [5] ---------------------------------------------------------- array has only one element [5] - no further action ---------------------------------------------------------- Merge -> array : [2, 5] Left : [2] Right : [5] *-*-* End of merge function >> Merged array is : [2, 5] ---------------------------------------------------------- Merge -> array : [1, 3, 2, 5] Left : [1, 3] Right : [2, 5] *-*-* End of merge function >> Merged array is : [1, 2, 3, 5] ---------------------------------------------------------- Merge -> array : [10, 4, 8, 6, 1, 3, 2, 5] Left : [4, 6, 8, 10] Right : [1, 2, 3, 5] *-*-* End of merge function >> Merged array is : [1, 2, 3, 4, 5, 6, 8, 10] Output array :[1, 2, 3, 4, 5, 6, 8, 10] ############# Test Case 2 ############# Input array : [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] Left Half :[3, 56, 2, 58, 79, 59] Right Half : [34, 23, 4, 78, 8, 123, 45] >>> Recursive call mergesort for left array [3, 56, 2, 58, 79, 59] Left Half :[3, 56, 2] Right Half : [58, 79, 59] >>> Recursive call mergesort for left array [3, 56, 2] Left Half :[3] Right Half : [56, 2] >>> Recursive call mergesort for left array [3] Array has only one element [3] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [56, 2] Left Half :[56] Right Half : [2] >>> Recursive call mergesort for left array [56] Array has only one element [56] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [2] Array has only one element [2] - no further action ---------------------------------------------------------- Merge -> Array : [56, 2] Left : [56] Right : [2] *-*-* End of merge function ** array is : [2, 56] ---------------------------------------------------------- Merge -> Array : [3, 56, 2] Left : [3] Right : [2, 56] *-*-* End of merge function ** array is : [2, 3, 56] ---------------------------------------------------------- >>> Recursive call mergesort for Right array [58, 79, 59] Left Half :[58] Right Half : [79, 59] >>> Recursive call mergesort for left array [58] Array has only one element [58] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [79, 59] Left Half :[79] Right Half : [59] >>> Recursive call mergesort for left array [79] Array has only one element [79] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [59] Array has only one element [59] - no further action ---------------------------------------------------------- Merge -> Array : [79, 59] Left : [79] Right : [59] *-*-* End of merge function ** array is : [59, 79] ---------------------------------------------------------- Merge -> Array : [58, 79, 59] Left : [58] Right : [59, 79] *-*-* End of merge function ** array is : [58, 59, 79] ---------------------------------------------------------- Merge -> Array : [3, 56, 2, 58, 79, 59] Left : [2, 3, 56] Right : [58, 59, 79] *-*-* End of merge function ** array is : [2, 3, 56, 58, 59, 79] ---------------------------------------------------------- >>> Recursive call mergesort for Right array [34, 23, 4, 78, 8, 123, 45] Left Half :[34, 23, 4] Right Half : [78, 8, 123, 45] >>> Recursive call mergesort for left array [34, 23, 4] Left Half :[34] Right Half : [23, 4] >>> Recursive call mergesort for left array [34] Array has only one element [34] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [23, 4] Left Half :[23] Right Half : [4] >>> Recursive call mergesort for left array [23] Array has only one element [23] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [4] Array has only one element [4] - no further action ---------------------------------------------------------- Merge -> Array : [23, 4] Left : [23] Right : [4] *-*-* End of merge function ** array is : [4, 23] ---------------------------------------------------------- Merge -> Array : [34, 23, 4] Left : [34] Right : [4, 23] *-*-* End of merge function ** array is : [4, 23, 34] ---------------------------------------------------------- >>> Recursive call mergesort for Right array [78, 8, 123, 45] Left Half :[78, 8] Right Half : [123, 45] >>> Recursive call mergesort for left array [78, 8] Left Half :[78] Right Half : [8] >>> Recursive call mergesort for left array [78] Array has only one element [78] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [8] Array has only one element [8] - no further action ---------------------------------------------------------- Merge -> Array : [78, 8] Left : [78] Right : [8] *-*-* End of merge function ** array is : [8, 78] ---------------------------------------------------------- >>> Recursive call mergesort for Right array [123, 45] Left Half :[123] Right Half : [45] >>> Recursive call mergesort for left array [123] Array has only one element [123] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [45] Array has only one element [45] - no further action ---------------------------------------------------------- Merge -> Array : [123, 45] Left : [123] Right : [45] *-*-* End of merge function ** array is : [45, 123] ---------------------------------------------------------- Merge -> Array : [78, 8, 123, 45] Left : [8, 78] Right : [45, 123] *-*-* End of merge function ** array is : [8, 45, 78, 123] ---------------------------------------------------------- Merge -> Array : [34, 23, 4, 78, 8, 123, 45] Left : [4, 23, 34] Right : [8, 45, 78, 123] *-*-* End of merge function ** array is : [4, 8, 23, 34, 45, 78, 123] ---------------------------------------------------------- Merge -> Array : [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] Left : [2, 3, 56, 58, 59, 79] Right : [4, 8, 23, 34, 45, 78, 123] *-*-* End of merge function ** array is : [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 79, 123] Output array :[2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 79, 123] ############# Test Case 3 ############# Input array : [1, 2, 3, 4, 5] Left Half :[1, 2] Right Half : [3, 4, 5] >>> Recursive call mergesort for left array [1, 2] Left Half :[1] Right Half : [2] >>> Recursive call mergesort for left array [1] Array has only one element [1] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [2] Array has only one element [2] - no further action ---------------------------------------------------------- Merge -> Array : [1, 2] Left : [1] Right : [2] *-*-* End of merge function ** array is : [1, 2] ---------------------------------------------------------- >>> Recursive call mergesort for Right array [3, 4, 5] Left Half :[3] Right Half : [4, 5] >>> Recursive call mergesort for left array [3] Array has only one element [3] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [4, 5] Left Half :[4] Right Half : [5] >>> Recursive call mergesort for left array [4] Array has only one element [4] - no further action ---------------------------------------------------------- >>> Recursive call mergesort for Right array [5] Array has only one element [5] - no further action ---------------------------------------------------------- Merge -> Array : [4, 5] Left : [4] Right : [5] *-*-* End of merge function ** array is : [4, 5] ---------------------------------------------------------- Merge -> Array : [3, 4, 5] Left : [3] Right : [4, 5] *-*-* End of merge function ** array is : [3, 4, 5] ---------------------------------------------------------- Merge -> Array : [1, 2, 3, 4, 5] Left : [1, 2] Right : [3, 4, 5] *-*-* End of merge function ** array is : [1, 2, 3, 4, 5] Output array :[1, 2, 3, 4, 5] ``` **Observations based on code implementation:** - We defined two functions *“merge”* and *“mergesort”* to implement the Divide-and-conquer algorithm. From the logs, we see that the array [10,4,8,6,1,3,2,5] is divided into Left Half: [10, 4, 8,6] Right Half: [1,3,2,5]. - To further divide the arrays, we see the recursive algorithm in action by invoking function “mergesort” multiple times. As shown in the logs, the first set of [10,4,8,6] is divided until the single element, further sorting and merging it as [4,6,8,10]. In the same way, the right array is also sorted and built-up to merge and form a new array [1,2,3,5]. - Merge Sort performs three steps: dividing the collection, sorting the values, and merging it back together. Multiple sublists are sorted at a time in the Merge Sort due to recursion, whereas in case of iteration we sort one item at a time. This is a clear distinction from its peers Bubble Sort, Selection Sort, and Insertion Sort. The recursive action is more efficient than iterations, as the work is constantly divided into halves. - The function *“merge”* is the one that does the heavy lifting by sorting and merging back the arrays. In short, we are appending the items from a single element to form the complete array as a new array structure in sorted order. The step *“merge”* actively compares two sublists and then appends the smaller value as the first index and larger value at the second index position. - As shown below, in total we must perform n= 8 and 3 recursion level operations for merge and sorting. In each level, we will be performing sorting and merge of the elements to store it in a new temporary sublist array. As there are 3 levels we have performed a total of 24 operations of comparing the elements and inserting them in a new array. Remember, the length of our array is 8, how many recursion levels will we have for the size n = 16 array? It would be 4 recursion levels, similarly, for a size ‘n’ number of an array, the recursion level will be the logarithmic value of Log2(n). - From the above abstraction, in case n = 8 (array size) multiplying with Log2(8) value (i.e 3) will give 24 as the number of operations to be performed. For time complexity, it is read as O(n*log(n)) time taken for the operations to be performed for the Merge Sort. ![Recursion tree and operation counts illustrating merge sort's O(n log n) complexity](../../assets/blog/reviewing-sorting-algorithms-merge-sort-5.png) - The code implementation log has three test cases, out of which one case is for a completely sorted array. The sorted array will also go through the discussed Divide-and-conquer algorithm process and comparisons are performed to find the rightful sorting position. This indicates there is no distinction in runtimes for both the best-case scenarios of sorted arrays and an unsorted array. ## Time and space complexity - As we see from the above illustration, as the size of ‘n’ increases, so does the number of merge operations required. if we recollect from our previous posts, Bubble Sort, Insertion Sort, and Selection Sort all have worst-case time complexity is O(n^2) as the size of ‘n’ increases, the runtime is quadratic (n(n-1)/2) times. Whereas in the case of the Merge Sort runtime is not quadratic. - Merge Sort’s time complexity follows both linear and logarithmic runtime, where O(n) is linear and O(log N) is logarithmic. The division of the “mergesort” function indicates logarithmic time complexity, whereas the time complexity of the “merge” function and recursion is linear indicating O(n) time complexity. The merge sort combining both linear and logarithmic runtimes follows a time complexity called “linearithmic time” which is O(n log n). - In the case of Merge Sort, the time complexity for the best case, worst case, and average case scenarios are the same as O(*n* log *n*). - It’s time to refer to the Big-O Complexity chart we referred back in our [first post series of the Bubble Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-bubble-sort). From the below chat we can see that Merge Sort performs a lot better than its peers with quadratic runtimes O(n^2). ![Fig3.1 Big-O Algorithm Complexity Chart [Ref.4]](../../assets/blog/reviewing-sorting-algorithms-merge-sort-6.png) - We have discussed stability in our [previous post](https://www.prostdev.com/post/reviewing-sorting-algorithms-insertion-sort), the Merge Sort is a stable algorithm as it maintains the stability of the input data set. - As the Merge Sort requires a temporary array to store the sorted merge values, it requires an O(n) or constant amount of space. This is the memory needed for the temporary buffer array storage. The Merge Sort is an out-of-place sorting algorithm, which requires additional memory as its dataset grows. The Bubble Sort, Insertion Sort, and Selection Sort are in-place algorithms. > “*An in-place algorithm is an* [algorithm](https://en.wikipedia.org/wiki/Algorithm) *that transforms input using no auxiliary* [data structure](https://en.wikipedia.org/wiki/Data_structure)*. However, a small amount of extra storage space is allowed for auxiliary variables. The input is usually overwritten by the output as the algorithm executes. In-place algorithm updates the input sequence only through replacement or swapping of elements. An algorithm that is not in-place is sometimes called not-in-place or out-of-place."* [[Ref. 3](https://en.wikipedia.org/wiki/In-place_algorithm)] **Time and space complexity table:** ![Table comparing time and space complexity of merge, insertion, selection, and bubble sort](../../assets/blog/reviewing-sorting-algorithms-merge-sort-7.png) ## Final thoughts The question is, did we find “The One”? We certainly see some potential in the Merge Sort as a faster and more efficient sorting algorithm with minor drawbacks, compared to the most popular kid Bubble Sort, followed by Selection and Insertion Sort. - In the case of large data sets, the Merge Sort is definitely way faster than its peers O(n^2) with a linearithmic time complexity of O(*n* log *n*) in best, average, and worst case scenarios. - However, the time complexity of the Insertion Sort’s best case scenario is O(n), which is better than the Merge Sort in case of a fully sorted array. - The space complexity of the Insertion Sort O(1) is better than the Merge Sort’s O(n) as it requires additional space for a temporary buffer array in case of large data sets. This is one of the drawbacks of the Merge Sort being an out-of-place algorithm requiring more memory as the dataset grows. Considering the above drawbacks, for small datasets it will be beneficial to use the Insertion Sort and for large datasets, the best suited so far is the Merge Sort. In fact, in Python and Java, the sort function is implemented as a hybrid of the Merge Sort and Insertion Sort called [Tim Sort](https://en.wikipedia.org/wiki/Timsort). For hybrid models, the Insertion Sort is used for the sublist of the Merge Sort as it runs faster for small data sets and has efficient space complexity. The hybrid model can be considered as “The One”, but hey, there are few other candidates auditioning for the title. The next candidate up for auditioning is known to quickly sort the data set using the “Quick Sort” algorithm. Stay tuned!! I hope you enjoyed the post. Please subscribe to[ProstDev](https://www.prostdev.com/) for more exciting topics. Hasta luego, amigos! ## References - [Divide-and-conquer algorithm - Wikipedia](https://en.wikipedia.org/wiki/Divide-and-conquer_algorithm) - [Divide-and-conquer algorithm - Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/merge-sort/a/divide-and-conquer-algorithms) - [In-place algorithm - Wikipedia](https://en.wikipedia.org/wiki/In-place_algorithm) - [Big-O Algorithm Complexity Chart](https://www.bigocheatsheet.com/) ## Study Resources - [Recursion - Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/recursive-algorithms/a/recursion) - [Video] [Recursion - CS50 Stanford](https://www.youtube.com/watch?v=mz6tAJMVmfM) - [Differentiating Logarthamic and linearithmic time complexity](https://levelup.gitconnected.com/differentiating-logarithmic-and-linearithmic-time-complexity-976cd49c351b) - [Analysis of Merge Sort - Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/merge-sort/a/analysis-of-merge-sort) - [Logarithm Calculator - RapidTable](https://www.rapidtables.com/calc/math/Log_Calculator.html) - [Video] [Merge Sort with Transylvanian German Folk Dance](https://www.youtube.com/watch?v=HzUrYDdXY-8) - [Video] [CS50 - Merge Sort](https://www.youtube.com/watch?v=Pr2Jf83_kG0) - [Merge Sort - GeekforGeeks](https://www.geeksforgeeks.org/merge-sort/) --- ## IoT with MuleSoft: Implementing a Temperature Sensor using LED lights, Twilio, and a Raspberry PI Source: https://prostdev.com/post/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi | Published: Aug 20, 2020 | Category: Tutorials Hey Muleys, In this post, I will explain how to implement IoT with a MuleSoft use case. I would like you to try out some new use cases and publish them. I hope this article helps you to implement them! First of all, why should you use MuleSoft to implement IoT? Why not use other integration platforms? Quick answer: Ease of Integration and Speed of Delivery. Let's dive in. Make sure to watch the videos posted at the end of the article! You will get a full idea on how easy it is to implement IoT with MuleSoft. ## What is IoT? The Internet of Things (IoT) is the inter-network of physical devices, vehicles, buildings, and other items embedded with electronics, software, sensors, actuators, and network connectivity that enable these objects to collect and exchange data. Simplified: IoT is about connecting software with hardware! **MuleSoft + IoT:** - The Mule engine can be embedded directly into IoT devices, which enables data exchange for the devices by connecting to IoT cloud services and backend apps in the cloud. - The Mule Runtime engine can be used to expose APIs on any IoT device. Mule APIs can be deployed on IoT devices and turn them on and off. - In this article we will discuss about IoT and how it can be used with MuleSoft, and how Mule APIs can be deployed on IoT devices. ## Use Case When a user passes a receiver’s number in the URL, the receiver should read the current local temperature details to his mobile and on successful receiving of details on his mobile, a green LED light should be blinked. In case of any issue in receiving the details, the red LED light should be blinked instead. **How does it work internally?** When you hit this endpoint: ``` http://localhost:8081/test?toNumber=919999999999 ``` - The request is sent to the Mule application. - The receiver’s number is stored in a variable. - The Mule application connects to Raspberry PI (IoT device) and senses the temperature using a temperature sensor. - The temperature is stored in a variable. - The Mule application sends the details to the receiver’s mobile number using the Twilio Connector. - If the data is received as expected, a green LED light is blinked, otherwise it will be a red LED light. ## Let's cook the recipe **Required Software:** - [Raspberry Pi OS (previously called Raspbian OS)](https://www.raspberrypi.org/downloads/raspbian/) - [SD Memory Card Formatter](https://www.sdcard.org/downloads/formatter/) - [Xming Display Server](https://xming.en.softonic.com/) - [Win32 Disk Imager](https://win32diskimager.download/) - [PuTTY](https://www.putty.org/) - [WinSCP](https://winscp.net/eng/download.php) - [Mule Standalone Server](https://www.mulesoft.com/lp/dl/mule-esb-enterprise) **Required Hardware:** - Raspberry Pi 3 - Micro SD Card of 16GB - Ethernet Cable or HDMI cable (in this use case we are using ethernet cable) - Adapter Charger for Raspberry Pi 3: Available in electronic stores or any e-commerce websites as a combo. I was able to find everything [here](https://www.amazon.in/gp/product/B07C6SN8PL/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1). **Resistors:** - 4k7 ohms (4.7k) - 1 (used for temp sensor) - 22k ohms: 2 in total (used for RED and GREEN LED lights) - Jumper wires: Male – Female (10 for safe side) - 1 Bread Board - Temperature Sensor: DS18B20 model - 1 red LED light - 1 green LED light ## Part 1 - Setting up your environment Before starting with the Raspberry PI setup, we have to follow these steps. **Step 1: Download and Install Raspberry Pi OS / Raspbian OS** Download Raspbian OS (URL can be found in Required Software above). Select “Raspbian Buster with desktop and recommended software”. It’s almost 2.5 GB. ![Raspberry Pi download page for Raspbian Buster with desktop and recommended software](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-1.png) While it’s downloading, insert your micro SD card into your system and format it using the SD Card formatter. Now extract the downloaded Raspbian OS into your local. You can see only one file of the type Disk Image. We can’t unzip them normally. That’s the reason we use Win32 disk imager: to extract and copy them to the Micro SD card. Open the Win32 disk imager and you will see that the destination folder will be automatically detected (the SD card). Just select the file where you have extracted Raspbian and click on write. It takes around 9 mins to finish. Once it’s done it will say that the writing was successful. Now your OS is copied successfully in your SD card. **Step 2: Enable SSH** The SSH command provides a secure encrypted connection between two hosts over an insecure network. This connection can also be used for terminal access, file transfers, and for tunneling other applications. Since we need to see what’s happening in the Raspberry PI, we will need a UI to see this. In order to set this up, we need to have SSH enabled. Go to the SD card folder where you can see the extracted files (they were extracted using Win32 disk imager). Now just create a text file and name it as ssh. After creating this, you can remove the SD Card from your PC. **Step 3: Network and Sharing** Now we need to connect our Raspberry device to our system. As I said before, we have 2 ways. One is using an HDMI cable which requires WiFi sharing, a monitor, a keyboard and a mouse to perform operations. To go for the alternative, it’s better to use an ethernet cable which helps us to get connected with our PC itself. For this article, we are using ethernet connectivity. After ejecting the SD card from the PC, mount the SD card to your Raspberry PI. See the slot where it needs to be inserted (usually it will be on the downside of the device). Plug the adaptor and switch it on. Now connect the ethernet cable: one side should be connected to the Raspberry PI and the other to your PC. Make sure your PC is already connected to WiFi or internet before plugging the ethernet cable. Once the ethernet cable is plugged to your PC. Go to the Network and Sharing center. You will see an unidentified network. You can refer to the next screenshot. ![Windows Network and Sharing Center with an Unidentified Ethernet network circled](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-2.png) Click on the internet connection which you have already connected to and click on WiFi – properties. Go to Sharing and make sure the 2 options in Internet Connection Sharing are checked. Also see that Home networking connection is automatically generated with the name “Ethernet” (you can go through the video I shared at the end to verify these steps). ![Wi-Fi Properties Sharing tab with Internet Connection Sharing enabled for Ethernet](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-3.png) Click on unidentified network (ethernet). Go to Properties and double click on Internet Protocol Version 4 (TCP/IPv4). Under “Use the following IP address” see that an IP address is automatically shown. Copy that IP address as we will be using the same IP to connect to the Raspberry PI. ![IPv4 Properties showing the auto-assigned IP address 192.168.137.1](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-4.png) **Step 4: Connect to the Raspberry PI using PuTTY** Before connecting using PuTTY, make sure you have open Xming (assuming it's already been downloaded and installed). You can open PuTTY after this. Include the following configuration: Hostname: `raspberrypi.mshome.net` (this is the default hostname for Raspberry Pi. You can also use the IP address generated above, but it’s better to use the mentioned hostname) Port: 22 Go to SSH and include the following configuration: ![PuTTY SSH X11 settings with Enable X11 forwarding checked and display set to localhost:0](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-5.png) Save with any name. Now you’re ready to connect to the device. Click on Open. You will get a security alert warning. Click on yes to proceed. A cmd prompt is popped out asking to login. You can select the default login details like the following: Login as: pi Password: raspberry Now you are connected to the “pi” user of your Raspberry device. To see a graphical view of your device (generally another system which runs on Raspbian OS) you can use the below command in PuTTY command: ```bash pi@raspberrypi : $ startlxde ``` Now your Xming opens! ![Raspberry Pi LXDE desktop shown through Xming with an LXTerminal icon](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-6.png) Leave your LXTerminal pinned on your desktop. Open the Terminal and type the below command: ```bash pi@raspberrypi : $ sudo raspi-config ``` A blue prompt appears. It has option1 to change password. Make sure you change the password and finish off. **Step 5: Install Java 8** We need to install Java version 8. By default, Java and Python are already installed when we download Raspbian OS. If you check the version (using the java -version command), we can see the Java 11.x version is already installed. We have some permission issues with this version. This is why we need to install Java 8 for this use case. Before installing, you will need to uninstall Java 11. Use below commands for uninstalling java 11 and then installing java 8 accordingly. To uninstall Java 11: ```bash pi@raspberrypi : $ sudo apt-get purge openjdk* ``` To install Java 8: ```bash pi@raspberrypi : $ sudo apt-get install openjdk-8-jdk ``` Now check the Java version again. It will show Java 8. We have set up our Raspberry OS. All 5 steps are related to Raspberry and have nothing to do with Mule installation. These are common steps for setting up a Raspberry PI device. ## Part 2 - Setting up Raspberry PI with Mule **Step 1: Installing Mule Standalone Server** Open a browser in Xming (in the Raspberry PI). Go to the link I shared in the Required Software to download Mule Standalone Server. While it’s downloading, we shall create a new user with the name “mule”. All MuleSoft operations are carried out by this user (this is not mandatory but it’s good to create one). To create a Mule user type the following commands: ```bash pi@raspberrypi : $ sudo su – root@raspberrypi : # useradd -s /bin/bash -d /home/mule -U -G sudo mule root@raspberrypi : # passwd mule New Password: Retype New Password: ``` Your “mule” user is now created successfully. Now you need to create a director and give all necessary permissions. For this, use the following commands: ```bash root@raspberrypi : # mkdir /home/mule /opt/mule root@raspberrypi : # chown mule:mule /opt/mule root@raspberrypi : # exit logout ``` Now it's time to look at the Mule Standalone Server. Your Mule Standalone will be located in the Downloads folder of the PI user. Follow these commands next: ```bash pi@raspberrypi : $ cd /home/pi/Downloads pi@raspberrypi :~/ Downloads$ chmod 777 * pi@raspberrypi :~/ Downloads$ su -mule Password: mule@raspberrypi :~$ cd /home/pi/Downloads mule@raspberrypi : /home/pi/Downloads$ cp mule-ee-distribution-standalone-4.2.2.zip /opt/mule mule@raspberrypi : /home/pi/Downloads$ cd /opt/mule mule@raspberrypi : /opt/mule$ unzip mule-ee-distribution-standalone-4.2.2.zip mule@raspberrypi : /opt/mule$ cd mule-ee-distribution-standalone-4.2.2 mule@raspberrypi : /opt/mule/mule-ee-distribution-standalone-4.2.2$ cd /opt/mule ``` Mule runtime uses the Tanuki Service Wrapper, which allows a Java-based application (that’s right, such as Mule runtime) to be started as a Windows Service or UNIX daemon. However, out-of-the-box, the bundled Service Wrapper is not optimized for Raspberry Pi’s ARM architecture. Therefore, the next step is to download the Armhf port of the Java Service Wrapper and patch the bundled Service Wrapper by copying a few required files to the Mule runtime directory. Additional config files needed: ```bash mule@raspberrypi:/opt/mule $ wget https://download.tanukisoftware.com/wrapper/3.5.34/wrapper-linux-armhf-32-3.5.34.tar.gz tar zxf wrapper-linux-armhf-32-3.5.34.tar.gz mule@raspberrypi:/opt/mule $ cp ./wrapper-linux-armhf-32-3.5.34/lib/libwrapper.so ./mule-standalone-4.2.2/lib/boot/libwrapper-linux-armhf-32.so mule@raspberrypi:/opt/mule $ cp ./wrapper-linux-armhf-32-3.5.34/lib/wrapper.jar ./mule-standalone-4.2.2/lib/boot/wrapper-3.2.3.jar mule@raspberrypi:/opt/mule $ cp ./wrapper-linux-armhf-32-3.5.34/bin/wrapper ./mule-standalone-4.2.2/lib/boot/exec/wrapper-linux-armhf-32 ``` Exit current terminal and re-open it. It comes with pi user ```bash mule@raspberrypi:/opt/mule $ cd mule-enterprise-standalone-4.2.2 mule@raspberrypi:/opt/mule/mule-enterprise-standalone-4.2.2$ cd conf mule@raspberrypi:/opt/mule/mule-enterprise-standalone-4.2.2/conf $ vi wrapper.conf ``` The mule file comes in edited format. Change below lines as mentioned and save it: ``` ## Initial Java Heap Size (in MB) wrapper.java.initmemory=256 ## Maximum Java Heap Size (in MB) wrapper.java.maxmemory=512 ``` Final step: ```bash mule@raspberrypi:/opt/mule/mule-enterprise-standalone-4.2.2 $ cd bin mule@raspberrypi:/opt/mule/mule-enterprise-standalone-4.2.2/bin $ ./mule ``` The server is up and running now: ![Mule standalone server console log showing the temperature app deployed and Mule running](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-7.png) **Step 2: Developing the Mule Application** Now we shall develop a RESTful application in your local (Windows) system. And then run the application which generates a snapshot jar file. ![Anypoint Studio flow: Listener, transforms, Groovy scripts, Twilio Send message, and a try/error-handler](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-8.png) Connectors used: - Groovy script: This component is used to run python scripts. Use operation groovy. - Twilio: This component is used to send a message to the user. - Transform Message: Setting variables and structuring the message before sending to twilio. > [!NOTE] > You are not writing any Python scripts in the Mule application. The Python scripts to light the LEDs and sense temperature are already written and placed in a specified location. The location path is given in the groovy scripts. Make sure to give a proper message structure before sending to Twilio, since it expects a specific format containing these fields: body, to, from, message. Also make sure you have a developer account with Twilio and make sure whatever number you are using is already registered with Twilio. Messages are sent only to registered numbers. If you are not okay with using Twilio, you can build your own use case. Like printing the temperature instead. Code for the developed app is pasted at the end of this article. Run your application. Once it’s deployed locally, copy the jar file generated inside the target folder and paste it into your Raspberry device folder using WinSCP. To do this, open WinSCP, use the same host and port details from the ones you used for PuTTY. It will ask for username and password. Enter username as “pi” and the previously specified password. Copy the file from your Windows PC into the /home/pi/Downloads folder of your Raspberry PI. **Step 3: Connections in Raspberry PI** ![GPIO pinout chart beside a Raspberry Pi wired to a breadboard](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-9.png) Remember – Physical numbering is different from GPIO pin number. Watch the video for the details on how to create these connections. You can refer to the following pictures. ![Raspberry Pi GPIO pinout diagram mapping physical pin numbers to GPIO labels](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-10.png) ![Breadboard with a temperature sensor and green and red LEDs wired with jumper cables](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-11.png) Connection with Jumper wires: > [!NOTE] > physical numbering is normal 1,2,3…42 GPIO is different. For the connections please go with physical numbering. In the Python script you can see GPIO pin numbers. Do not get confused. **Green LED light:** GPIO Numbering: - Voltage: GPIO17 - Ground: Any nearest ground pin Physical Number: - Voltage: 11th pin - Ground: 14th pin **Red LED light:** GPIO Numbering: - Voltage: GIPO26 - Ground: Any nearest ground pin Physical Number: - Voltage: 37th pin - Ground: 39th pin **Temperature sensor:** GPIO Numbering: - Voltage: 3V3 - Ground: Any nearest ground pin - Data: GPIO 4 Physical Number: - Voltage: 1st pin - Ground: 6th pin - Data: 7th pin > [!IMPORTANT] > Before executing the Python scripts via Mule app, make sure you run the Python scripts independently (Python is already installed when Raspbian OS is installed). ```bash $ python /home/pi/Downloads workingTemp.py ``` Open LxTerminal: ```bash pi@raspberrypi: ~ $ cd /home/pi/Downloads pi@raspberrypi: ~/Downloads $ chmod 777 raspberry-temperature-new-1.0.0-SNAPSHOT-mule-application.jar pi@raspberrypi: ~/Downloads $ chmod a+x raspberry-temperature-new-1.0.0-SNAPSHOT-mule-application.jar ``` Write Python Scripts: greenLight.py: ![Terminal showing the greenLight.py Python script that toggles GPIO 17 high then low](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-12.png) redLight.py: ![Terminal showing the redLight.py Python script that toggles GPIO 26 high then low](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-13.png) Temperature.py: ![Terminal showing the workingTemp.py script reading the DS18B20 sensor and converting to Fahrenheit](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-14.png) Give permissions to Python scripts: ```bash pi@raspberrypi: ~/Downloads $ chmod 777 greenLight.py pi@raspberrypi: ~/Downloads $ chmod a+x greenLight.py pi@raspberrypi: ~/Downloads $ chmod 777 redLight.py pi@raspberrypi: ~/Downloads $ chmod a+x redLight.py pi@raspberrypi: ~/Downloads $ chmod 777 temp.py pi@raspberrypi: ~/Downloads $ chmod a+x temp.py ``` Final step: ```bash pi@raspberrypi: ~/Downloads $ su -mule Password: mule@raspberrypi: ~ $ cd /home/pi/Downloads mule@raspberrypi:cd /home/pi/Downloads$ cp raspberry-temperature-new-1.0.0-SNAPSHOT-mule-application.jar /opt/mule/mule-enterprise-standalone-4.2.2/apps ``` As the server is already up and running, the app is successfully deployed. ## Videos - For success: [Blink green LED on success](https://youtu.be/pEvFYxv8Hq0) - For failure: [Blink red LED on failure](https://youtu.be/HJGBqjTURco) - For connections on breadboard: [Raspberry Pi connection to breadboard](https://youtu.be/qTg3k8hxG6k) - For full setup: [Mule 4 standalone server on Raspberry Pi](https://youtu.be/TLOL9hwjNsY) ## Hardware Pictures ![Raspberry Pi 3 board with a yellow Ethernet cable plugged in](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-15.png) ![Hands holding two jumper wires with male and female ends](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-16.png) ![Hand holding a small DS18B20 temperature sensor with three leads](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-17.png) ![Hand holding a small resistor with its leads bent outward](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-18.png) ![Green LED and a resistor inserted into a breadboard](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-19.png) ![Raspberry Pi 3 with colored jumper wires connected to its GPIO header](../../assets/blog/iot-with-mulesoft-implementing-a-temperature-sensor-using-led-lights-twilio-and-a-raspberry-pi-20.png) ## Mule App Code ```xml def command = "python /home/pi/Downloads/workingTemp.py" println "$command" def cmd=command.execute() def command = "python /home/pi/Downloads/greenLight.py" println "$command" def cmd=command.execute() def command = "python /home/pi/Downloads/redLight.py" println "$command" def cmd=command.execute() ``` Hope this article helps you to do some real time use cases! Happy Learning!! Yours, Sravan Lingam :) --- ## Cryptography Module Mule 4 - Part 1 (PGP Encryption/Decryption) Source: https://prostdev.com/post/cryptography-module-mule-4-part-1-pgp-encryption-decryption | Published: Aug 18, 2020 | Category: Tutorials *GitHub repository with the Mule project can be found at the end of the post.* Mule 4 has a Cryptography module which includes these 3 different strategies: - PGP - XML - JCE In this article, we will see the PGP technique. ## PGP Pretty Good Privacy (PGP) is a cryptographic way that allows secure communication between two entities. It uses the public and private key concepts to encrypt the data as shown in the below diagram. ![PGP diagram: A encrypts a message with B's public key and B decrypts it with the private key](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-1.png) ## Prerequisites **1. Install the Crypto Module from Exchange, located in the Mule palette.** > [!NOTE] > Here is the reference documentation on how to install new modules to your Mule Project: [Adding Modules to Your Project](https://docs.mulesoft.com/studio/7.5/add-modules-in-studio-to). ![Mule Palette Crypto module listing PGP, JCE, and XML operations like Pgp encrypt and Pgp decrypt](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-2.png) **2. Create private and public keys** Please follow the below steps to generate a public/private key pair: - Download and install the GnuPG from [this link](https://www.gnupg.org/download/index.html). - Install [Kleopatra](https://www.openpgp.org/software/kleopatra/) to use a Graphical User Interface (GUI). - To generate the keys, click on “New Key Pair” and follow the instructions on the screen. ![Kleopatra welcome screen with New Key Pair and Import buttons for managing PGP keys](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-3.png) - Once the keys are generated, export them to the file system. ![Kleopatra context menu noting Export gives the public key and Export Secret Keys the private key](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-4.png) - The generated files are of ASC format, which is not supported by Mule yet, so we need to [dearmor](https://www.gnupg.org/documentation/manuals/gnupg/Operational-GPG-Commands.html) the keys first. Run the following command: `./gpg --dearmor ` for each of the keys. This command will create new files alongside the ASC files that will have .gpg appended to their filename which are supported in Mule. ![File listing with private.asc and public.asc next to their dearmored .gpg versions](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-5.png) This is what you will get after following the previous steps: - Public/Private keys - Fingerprint - Passphrase - KeyId ## Mule Code Implementation We will limit our scope to PGP encrypt/decrypt operation in this article. ### Global configurations Create 2 global configurations: - Encryption – Configure public key, keyId, and fingerprint. - Decryption – Configure private key, keyId, and fingerprint. ![**For projects: you should have these 2 configurations in different projects and all fields should be read from property files. The passphrase should be treated as a secure property.](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-6.png) ### Usage **1. Encryption/Decryption of entire payload** **Encryption:** ![**To log or use payload in the next processor, convert the encrypted stream to a base64 data type.](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-7.png) **Output:** ![Postman POST to /encrypt returning a PGP-encrypted payload as a base64 string](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-8.png) **Decryption:** ![Mule XML for the decryption flow converting the base64 payload to a stream and running Pgp decrypt](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-9.png) **Output:** ![Postman POST to /decrypt returning the original JSON payload with name, city, and email](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-10.png) **2. Encryption/Decryption at field level** This is a very common non-functional requirement where sensitive fields should be encrypted. For this, we can still reuse the pgp-encryption-flow and pgp-decryption-flow flows. The only change would be the way we refer to these flows from the main flow, and for that, the DataWeave lookup function is very useful. **Encryption:** ![**Used lookup function to invoke the pgp-encryption-flow.](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-11.png) **Output:** ![Postman POST to /encrypt returning JSON where only the EMAIL field is PGP-encrypted](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-12.png) **Decryption:** ![Mule XML using a DataWeave decrypt function with lookup to decrypt only the email field](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-13.png) **Output:** ![Postman POST to /decrypt returning JSON with the email field decrypted back to its original value](../../assets/blog/cryptography-module-mule-4-part-1-pgp-encryption-decryption-14.png) ## Conclusion So far we learned Mule code implementation for PGP. You can set this Mule code as a common service that will help to achieve the encryption/decryption non-functional requirement in many APIs. Here are the 2 ways to set up this Mule code as a common service: - Externalize the flow and publish it on the Anypoint Exchange. - Create a common API, which will encrypt/decrypt the payload. ## References - [https://docs.mulesoft.com/mule-runtime/4.3/cryptography-pgp](https://docs.mulesoft.com/mule-runtime/4.3/cryptography-pgp) - [https://docs.mulesoft.com/mule-runtime/4.3/dw-mule-functions-lookup](https://docs.mulesoft.com/mule-runtime/4.3/dw-mule-functions-lookup) ## GitHub repository [ProstDev GitHub - Demo PGP](https://github.com/ProstDev/demo-pgp) --- ## 3 simple rules & 5 examples to understand the Error Handling in Mule 4 Source: https://prostdev.com/post/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4 | Published: Aug 13, 2020 | Category: Guides Hey Muleys, I believe that this article on Error Handling in Mule 4 will definitely help you to understand the core concept of On-Error Propagate and On-Error Continue. ## What is Error Handling? An exception occurs when an unexpected event happens while processing. Exception (or error) handling is the process of responding to exceptions when a computer program runs. In Mule, we can handle the message exception at different levels. - At project level using the **Default error handler** - At project level using the **Custom Global error handler** - At flow level in exception handling using the **Raise Error** component, **On-Error Continue** and **On-Error Propagate**. - At flow or at processor level using the **Try scope**. Whenever an error occurs in a flow, an error object is created. It contains many properties, like error.description or error.errorType. The errorType property is a combination of Namespace and Identifier. For example, HTTP:UNAUTHORIZED, where the Namespace is HTTP and the Identifier is UNAUTHORIZED. Mule identifies the error based on the errorType and then routes it to its respective block that is placed inside the Error Handler. ## What is On-Error Propagate and On-Error Continue? The difference between a flow, a sub-flow and a private flow, is that sub-flow doesn’t have an Error Handling scope. So, it's just flow and private flow the ones that have an Error Handling block and can contain the On-Error Propagate and On-Error Continue. > Whether it is Propagate or Continue, Mule Executes all the components within those blocks. **Remember:** The error will route to the Error Handling only if it identifies that the errorType from the error matches with what you have set in your Error Handling block. Mule 4 has one excellent feature that can identify the types of errors that can occur within that flow by looking at what kind of connectors are placed in that particular flow. ![Anypoint Studio error-type picker listing DB and HTTP error types available to the flow](../../assets/blog/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4-2.png) You can see above that, since we have an HTTP Request and Database components, the drop-down shows all the types of errors that can happen in different scenarios from these two components. It even shows the EXPRESSION error type (not in the picture) because there are some DataWeave syntaxes in the flow. It will be very easy to learn Error Handling by knowing how On-Error Propagate and On-Error Continue work. We shall follow some set of rules to identify the flow of process. That way we can determine what is the result payload statusCode. ## Rules Before learning about rules. We must not forget that a RAML-based generated HTTP Listener will have the Error Response Body set as “payload” by default, but a manually drag-and-drop HTTP Listener will have output text/plain - - - error.description set by default. So, remember to change this according to your business requirements. Here are the rules to remember when an error has occurred in any flow: **Rule 1:** - If anything is present in the Error Handling section. Even if there are On-Error Propagate and On-Error Continue blocks, make sure that your particular errorType is handled by those. - If not, then Mule will use the default Error Handling. If your flow is not called by any other flow, then it will display the default value that is set in the Error Response Body and will return a status code of 500 by default, if nothing is manually set. If your flow is called by any other flow, then the error will be raised to the calling flow. **Rule 2:** If Error Handling is present in that particular flow and errorType is handled (Rule 1.1), then check whether it is On-Error Continue or On-Error Propagate. In both the cases, Mule will execute all components within that block. **Rule 3:** After the execution of all the components, now: - If the error is handled using On-Error Propagate, it will raise an error back to the calling flow. - If the error is handled using On-Error Continue, it will not raise an error back to the calling flow and continue to the next processor after “flow-ref” and continues further process as it is. But it will not continue to other processors in the flow where the error is handled. - Suppose you have only a single flow. Then On-Error Propagate and On-Error Continue behave the same way, except that On-Error Continue gives 200 status and On-Error Propagate gives 500 status. Because, as explained, even if it’s On-Error Continue, it will not go to the next processor within the same flow. - Always remember, this point is very important. **On-Error Continue will continue only to the next processor of the calling flow but it will not continue within the same flow.** If you observe the flows below, suppose your error is handled in the second private flow and if there is On-Error Continue, it will not execute the logger in private flow 2, instead, it will go to the Set Payload component of private flow 1, which is the one calling private flow 2. ![Two flows: flow 1 with On Error Propagate calling flow 2, which has a Logger and On Error Continue](../../assets/blog/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4-3.png) If it is On-Error Propagate, the error is raised back to the calling flow. Now follow Rule 1 to Rule 3 again for that particular calling flow until the final process is completed. Whenever an error is raised back to the calling flow, just go by the three rules again. ## Examples Let’s see some examples and check what is the output and status code of each one. Consider that in all cases the default value for the Error Response in HTTP Listener is set to #[payload] and all cases below raise the same error for errorType: HTTP:CONNECTIVITY. All cases have the same input payload, which is: ``` {name : “sravan lingam”} ``` ### **Case 1** Let us assume we have only one flow. Let's follow the previous rules. ![Single flow with debugger showing error HTTP:CONNECTIVITY while On Error Continue handles HTTP:NOT_FOUND](../../assets/blog/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4-4.png) **Rule1:** There’s something in the Error Handler, but the errorType from the error does not match the errorType from the Error Handling. So it doesn’t matter whether it’s On-Error Continue or On-Error Propagate. When the errorType is not matched, it will use the default error handler by Mule. ``` Actual errorType = HTTP:CONNECTIVITY Handled errorType = HTTP:NOT_FOUND ``` Since none of them match and this is the main flow and it’s not called by any other flow, it uses Mule’s default error-handling. It will print the payload from before the HTTP Request component with a status code 500. Final Response: ``` Response Body: {name : “sravan lingam” } Response status code: 500 ``` ### **Case 2** Let us assume we have only one flow. Let's follow the previous rules. ![Single flow whose On Error Continue handles HTTP:CONNECTIVITY, matching the request error](../../assets/blog/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4-5.png) **Rule1:** There’s something in the Error Handler and errorType matches with the handled errorType. ``` Actual errorType = HTTP:CONNECTIVITY Handled errorType = HTTP:CONNECTIVITY ``` Since both match and this is the main flow and it’s not called by any other flow, we can continue with the other two rules. **Rule 2:** The flow is using On-Error Continue. It executes all the components in this block, in this case, we are setting the payload to “Error Handled in Main flow” with a Transform Message component. **Rule 3:** According to points 2 and 3 in Rule 3, since it is On-Error Continue, and the current flow is not called by any other flow, it will execute the processors inside the On-Error Continue and end the process with a 200 status. It will not execute the components after the HTTP Request since this is the main flow. It will print the payload that is set in the Transform Message component from the On-Error Continue. Final Response: ``` Response Body: “Error Handled in Main flow” Response status code: 200 ``` ### **Case 3** If we were to use On-Error Propagate in Case 2 instead of On-Error Continue, every rule is the same but since it’s On-Error Propagate, the error will be raised and sent back to the caller with a status code of 500. Final Response: ``` Response Body: “Error Handled in Main flow” Response status code: 500 ``` ### **Case 4** Let us assume we have 2 flows. Let's follow the previous rules. ![Main flow with On Error Continue and private flow 1 with a Request and On Error Propagate type ANY](../../assets/blog/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4-6.png) **Rule1:** There’s something in the Error Handler and the errorType matches with the handled errorType. ``` Actual errorType = HTTP:CONNECTIVITY Handled errorType = ANY ``` > [!NOTE] > ANY can handle any type of error. **Rule 2:** The flow that has the error (private flow 1 in this case) is using On-Error Propagate. It executes all the components in this block, in this case, we are setting the payload to “Error Handled in Private Flow 1” with a Transform message component. **Rule 3:** According to Rule 3, since it is On-Error Propagate, and the current flow is called by the main flow, it will execute the processors inside the On-Error Propagate and then raise the error back to the calling flow, which is the main flow. Now the ball is in the Main flow court. ![Both flows showing the error propagated from private flow 1 back to the main flow's On Error Continue](../../assets/blog/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4-7.png) This is how the error is raised back to the Main flow. Repeat the rules: **Rule 1:** The error matches the errorType from the Error Handler. **Rule 2:** On-Error Continue executes the Transform Message component with payload “Error Handled in Main flow”. It will not go further to the other components after the flow-ref. Since it is On-Error Continue, and no other flow is calling the current flow, it will display the latest payload with a 200 status. Final Response: ``` Response Body: “Error Handled in Main flow” Response status code: 200 ``` ### **Case 5** Same as Case 4 except that On-Error Continue is used in private flow 1 (the second flow). ![Main flow with On Error Continue and private flow 1 also using On Error Continue type ANY](../../assets/blog/3-simple-rules-5-examples-to-understand-the-error-handling-in-mule-4-8.png) **Rule 1:** There’s something in the Error Handler and the errorType matches with the handled errorType. ``` Actual errorType = HTTP:CONNECTIVITY Handled errorType = ANY ``` > [!NOTE] > ANY can handle any type of error. **Rule 2:** The flow that has the error (private flow 1 in this case) is using On-Error Continue. It executes all the components in this block, in this case, we are setting the payload to “Error Handled in Private Flow 1” with a Transform message component. **Rule 3:** According to Rule 3, since it is On-Error Continue, and the current flow is called by the main flow, it will execute the processors inside the On-Error Continue, doesn’t raise the error and goes back to the calling flow, which is the main flow. It executes all the components after flow-ref and the final payload will be displayed with a 200 status code. Final Response: ``` Response Body: “Success in Main flow” Response status code: 200 ``` Simple! If you follow this set of rules, then you can easily understand the way that On-Error Continue and On-Error Propagate work! Happy Learning! Sravan Lingam --- ## How to run locally the DataWeave Playground Docker Image Source: https://prostdev.com/post/how-to-run-locally-the-dataweave-playground-docker-image | Published: Aug 11, 2020 | Category: Tutorials You may already know about the DataWeave (DW) Playground that can be used for both versions of DW: 1.0 and 2.0. DataWeave version 1.0 is used in Mule 3, and DataWeave version 2.0 is used in Mule 4. If this is your first time hearing about the DW Playground, check out this post to see why you need to start using it now to test your DataWeave scripts or transformations. I will explain how to get this Docker Image running on your laptop (even if you don’t know how to use Docker). I’ll also show you how to install Docker Desktop and what exact commands you need to run to have it up and ready to use. ## What is the DW Playground, and why is it useful? DataWeave is MuleSoft’s scripting language that is mainly used to transform the data in your integration apps. To be able to use this, you need to download, install and open [Anypoint Studio](https://www.mulesoft.com/platform/studio) (Mule’s [IDE](https://www.prostdev.com/post/5-reasons-why-you-need-an-ide)). Then you need to create a new Mule Project, create a new flow, and finally add the Transform Message component. There are times when you already have a project, and you just want to test out a function to make sure that it works correctly. You can either create a new flow, or create a new project, and then add a new Transform Message component in order to test it out. After you test your code, you normally just delete whatever you created, right? Well, what if there was a way to use a browser, like Google Chrome, and already have your Transform Message component right there to test out the script you want directly? There *is* a way! It’s the DW Playground! Yes, you read that correctly. Let me show you a screenshot of the DW Playground for the 2.0 version. ![DataWeave Playground 2.0 open in Chrome at localhost:9999, outputting "Hello world!"](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-1.png) As you can see, I’m accessing it through Google Chrome, and it’s running on my local computer. There’s no need to open Anypoint Studio to create transformations to practice my DataWeave skills. Talk about convenience! Everything I create using the DW Playground is temporary, so there’s no need to delete any project or files after I finish using it. This Playground is just a Docker Image that can be downloaded to your computer. You can run it with various commands and have it ready to use whenever you want. ## How to install Docker Desktop I’m not going to explain the details of Docker, but I’ll leave some resources at the end of the post to learn more about it if you want to. Just know that previous knowledge in Docker isn’t needed in order to get the Playground running. If you already have Docker Desktop installed, you can skip this part. **1. Download and Install the Docker Desktop app** You can go to [this link](https://www.docker.com/products/docker-desktop) and select your operating system (Windows / Mac) in order to download the corresponding installer. Just follow the instructions to install as you would with any other regular application. **2. Run Docker** Double click on the installed application to open it and make sure that Docker is running in the background. To verify this, you can check your taskbar and look for the Docker icon. ![macOS menu bar with the Docker whale icon highlighted, indicating Docker is running](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-2.png) **3. Open the Docker Desktop application** Once you double click on it, you will see the main screen of the application. Notice that you can see whether Docker is running, and you can also sign in to your DockerHub account. ![Docker Desktop main screen showing "No containers running" with a Sign in button](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-3.png) ## How to run the DataWeave Playground Docker Image Now, let’s get started with the fun part: how to get the DW Playground up and running on our local computer. > [!NOTE] > DockerHub is where all the images are stored for you to download or pull to your local computer. You can create an account if you want to save your favorite images there, but it’s unnecessary. **1. Get the Docker Image from DockerHub** Go to [this link](https://hub.docker.com/r/machaval/dw-playground/tags) so you can see the different versions of the Playground and select the version you want. For this demo, I’ll choose the latest one, which is 2.3.1. On the right side of the Docker Image, you can see there’s a command ready to copy. Feel free to copy this by clicking on the copy button, but don’t run it just yet. ![DockerHub machaval/dw-playground Tags page with the docker pull command for tag 2.3.1 highlighted](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-4.png) **2. Docker Run command** This is what you get when you copy the previous “docker pull” command: ```bash docker pull machaval/dw-playground:2.3.1-SNAPSHOT ``` You just need the last part (the docker image and tag) to paste it into the next docker run command. In this case, machaval/dw-playground:2.3.1-SNAPSHOT. We’ll run the container with this command: ```bash docker run -d --name DWPlayground2 -p 9999:8080 machaval/dw-playground:2.3.1-SNAPSHOT ``` Here is a brief explanation of what you’re running with it: ![Table explaining each part of the docker run command: -d, --name, -p, and the docker image tag](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-5.png) Once the command runs, you will see something like this in your terminal or console output: ![Terminal output of the docker run command pulling the image layers and finishing with the container ID](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-6.png) **3. Verify the container is running in the Docker Desktop app** Once the previous command is done with its execution, you can go to your Docker Desktop application, and you will see the new Docker Container running. ![Docker Desktop showing the DWPlayground2 container running on port 9999](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-7.png) **4. Open the DW Playground** Now you’re ready to open the Playground from your browser! Simply click on the “Open in browser” button from the Docker Desktop app, and this will automatically open the Playground for you. ![Docker Desktop container row with the "Open in browser" action button highlighted](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-8.png) Alternatively, you can manually open your favorite browser and go to this address: ```bash http://localhost:9999/ ``` Note that the port should be replaced with the port you chose in case you changed the 9999 port from the original command. ![DataWeave Playground 2.0 open in Chrome at localhost:9999, outputting "Hello world!"](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-1.png) You're all set! To run the Playground for any other version, you can simply repeat the previous steps. But instead of using machaval/dw-playground:2.3.1-SNAPSHOT you would have to copy the docker image of the version of your preference. For example, to run the version 1.1.8, you could run the following command: ```bash docker run -d --name DWPlayground1 -p 9998:8080 machaval/dw-playground:1.1.8-SNAPSHOT ``` > [!NOTE] > if you intend to run both containers at the same time, you should change the ports to use different ones, so they don’t collide in your local computer. I personally run the DW Playground version 1 on port 9998 and version 2 on port 9999 even when I don’t have them both running at the same time. Here is my Docker Desktop app: ![Docker Desktop with two containers: DWPlayground2 on port 9999 and DWPlayground1 on port 9998](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-9.png) ## How to remove or stop the Docker Containers Maybe you don’t want to use the Playground anymore, or you just want to clear the containers from your app. Luckily for us, the Docker Desktop application provides a button to stop and remove the containers. Take a look at the screenshots below to be able to manipulate your containers. **Stop a running container** ![Docker Desktop with the Stop button highlighted on the running DWPlayground1 container](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-10.png) **Start a container** ![Docker Desktop with the Start button highlighted on the exited DWPlayground1 container](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-11.png) **Delete an existing container** ![Docker Desktop with the Delete button highlighted on the exited DWPlayground1 container](../../assets/blog/how-to-run-locally-the-dataweave-playground-docker-image-12.png) ## Recap In summary, this is what you need to do to run the DataWeave Playground Docker Image: - Download, install and open the Docker Desktop application to get Docker up and running. - Go to DockerHub to find the version of the Playground that you want to download: [https://hub.docker.com/r/machaval/dw-playground/tags](https://hub.docker.com/r/machaval/dw-playground/tags) - Run the docker container with the following command: ```bash docker run -d --name DWPlayground2 -p 9999:8080 machaval/dw-playground:2.3.1-SNAPSHOT ``` *Remember to replace the name (**DWPlayground2**), the port (**9999**), and the DW Playground version (**2.3.1-SNAPSHOT**) with your own settings.* - Open the local container by clicking on the “Open in browser” button from the Docker Desktop app or manually access it through the browser with ```bash http://localhost:9999/ ``` - Stop the running container right from the Docker Desktop app by clicking on the “Stop” button. You can start it again with the “Start” button; there’s no need to re-run the command. I hope this helped you to get started with the DW Playground. Let me know if you had any issues when running these commands or with any of the steps mentioned. *Prost!* -Alex ## More links - [Get Started with Docker](https://www.docker.com/get-started) - [Docker 101 Tutorial](https://www.docker.com/101-tutorial) - [Pluralsight training: Getting Started with Docker](https://www.pluralsight.com/courses/docker-getting-started) - [Docker Cheat Sheet](https://www.docker.com/sites/default/files/d8/2019-09/docker-cheat-sheet.pdf) --- ## Parallel For-Each Scope (Mule 4) Source: https://prostdev.com/post/parallel-for-each-scope-mule-4 | Published: Aug 6, 2020 | Category: Guides Processing a single element of a collection is a very common scenario and for that, we have the For-each scope and Batch processing in MuleSoft. In this article, we will talk about a new scope that is called the Parallel For-each scope. ## Parallel For-Each Scope Like the For-each scope, the Parallel For-each scope also splits the collection of messages into elements. But, unlike the For-each scope, it processes each element simultaneously in separate routes and the result is the collection of all messages aggregated in the same sequence they were before the split. ![Anypoint Studio flow with a Parallel For Each scope wrapping a Try with Is-number and Transform Message](../../assets/blog/parallel-for-each-scope-mule-4-1.png) Looking at the above picture you might be wondering, where are the separate routes? And how are these routes created? The number of routes is equal to the size of the collection and, unlike the scatter-gather, we cannot see these separate routes visually in the canvas. **For example:** The collection configured in the Parallel For-each scope is *[1,2,3,4].* The number of routes Mule runtime will create is 4. ## How does the Parallel For-each work? ![**Each arrow is a separate route inside the Parallel For-each scope and M is the Mule message.](../../assets/blog/parallel-for-each-scope-mule-4-2.png) As shown in the diagram, the collection gets split inside the Parallel For-each scope and each route is processed simultaneously. The output is the collection of Mule messages. > [!NOTE] > The processing of one element is invisible to other elements and creation or modification to an existing variable inside the scope is not visible outside the scope. ![JSON output M1 showing one Mule message with payload number 1 and empty attribute arrays](../../assets/blog/parallel-for-each-scope-mule-4-3.png) ## Configuration The configuration details are well documented in the MuleSoft documentation: [https://docs.mulesoft.com/mule-runtime/4.3/parallel-foreach-scope#configuration](https://docs.mulesoft.com/mule-runtime/4.3/parallel-foreach-scope#configuration) ## Error Handling All routes continue processing even if there is an exception in one of the routes and then runtime executes the code in the error handler. > [!NOTE] > The flow control will not go to the processor placed next to the Parallel For-each scope. ![Diagram comparing success and error scenarios, where an error routes to the Error Handler](../../assets/blog/parallel-for-each-scope-mule-4-4.png) Sometimes we may have a scenario in which we want the flow control to go to the next processor even in case of errors. This functionality can be achieved by using the Try scope and On-Error Continue processor. ![Parallel For Each scope using a Try with an On Error Continue handler so flow reaches the next processor](../../assets/blog/parallel-for-each-scope-mule-4-5.png) ## Conclusion If you want to execute elements of a collection simultaneously without using a batch processing, the Parallel For-each scope is another out of the box solution available in Mule 4. ## References [https://docs.mulesoft.com/mule-runtime/4.3/parallel-foreach-scope](https://docs.mulesoft.com/mule-runtime/4.3/parallel-foreach-scope) --- ## Reviewing Sorting Algorithms: Insertion Sort Source: https://prostdev.com/post/reviewing-sorting-algorithms-insertion-sort | Published: Aug 4, 2020 | Category: Guides Let’s sort it out! In a series of posts, we will be discussing some of the sorting algorithms listed in the below order: - [Bubble Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-bubble-sort) - [Selection Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-selection-sort) - **Insertion Sort** - [Merge Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-merge-sort) - Quick Sort - Heap Sort In this post, we will explore the next in a series of sorting algorithms, the Insertion Sort. If you are still wondering how we landed here with a bunch of sorting algorithms, please go through the previous posts on [Bubble Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-bubble-sort) and [Selection Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-selection-sort). We are yet to discover the fastest and most efficient algorithm to sort the long list of input elements. In this post, the question remains, is Insertion Sort "The One"? We don’t have to wait for the "Oracle" to confirm. Let’s find out. ## Why is it called Insertion Sort? Insertion Sort is one of the simplest and easiest sorting algorithms to understand. By the way, when’s the last time you sorted a hand of playing cards? If it’s today, congratulations! You have understood insertion sort even before we started. Brownie points to you!! If it’s been a while, you should try forming the thinking cloud of sorting the playing cards. You might have not realized it; you were implementing a version of the Insertion Sort algorithm to quickly sort the cards at hand. Well, if you are one of those, who never sorted playing cards, there isn’t a better time than “now” to expedite your understanding of the Insertion Sort. Remember, in our pilot post of [Bubble Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-bubble-sort), we discussed sorting as an act of arranging the elements systematically, based on a predetermined order or certain rules. We have observed from our previously discussed sorting techniques that a comparison of elements is the fundamental idea to arrive at a sorted list. In the case of the Insertion Sort, the basic principle is of inserting an unsorted element at a particular sorted position. The name “Insertion Sort” is derived based upon the concept, where an element has to find its rightful place and has to be inserted in there. If you are confused, let’s go through the below listed instructions to get some clarity. Breaking down the instructions: - For a given unsorted array [10,4,8,6,1], the iteration will be performed for every array element, from left to right. - Two subsets or sub-lists are maintained, the left side is a sorted subset and the right side is an unsorted subset. The first element in the array [10,4,8,6,1] will remain in place and is assumed to be sorted, as there are no further elements on the left side to compare. - Now we have two sections, one sorted section [10], and other unsorted sections [4,8,6,1]. Insertion Sort will iterate through each element of the unsorted list and compare it with the sorted subset, to shift the largest number to the right and insert the smallest element in its current rightful sorted position. - The first element in the unsorted subset (i.e., 4 in our example) is compared against each element from the sorted subset (i.e., 10 in this case) for its rightful sorted position. - If a number in a sorted list is found to be greater than the unsorted list number (i.e., 4), the number is shifted one position right. Meaning, the smaller number is inserted into the sorted subset. Result array would be [4,10,8,6,1]. - The sorting will iterate to the next number (i.e., 8) to compare against a new sorted subset [4,10]. The above steps will be repeated until a valid sorted position is obtained for the same number. - In the next iteration the sorted subset is [4,8,10], the process is repeated for the next number in an unsorted subset [6,1]. We observe that the unsorted subset shrinks with every iteration. Let’s play with our numbers: **Problem:** Sort the given array [10, 4, 8, 6, 1]. As discussed above, we will divide the array into two subsets of sorted and unsorted lists to demonstrate subsets. Iteration starting with the first index [0] as below. ![Insertion sort walkthrough of 10, 4, 8, 6, 1 showing iterations 0 to 2 with sorted and unsorted subsets](../../assets/blog/reviewing-sorting-algorithms-insertion-sort-1.png) ![Insertion sort iterations 3 and 4 shifting elements to reach the final sorted array 1, 4, 6, 8, 10](../../assets/blog/reviewing-sorting-algorithms-insertion-sort-2.png) For more insight into Step 4, below is a simple demonstration. **Observations based on Iterations:** - Well, the obvious observation here is, in each iteration, we compared the first unsorted item against the sorted item to its left to figure out if it’s in the right sorted position. - Iteration is performed on every element and it is from left to right, growing the sorted array. Although the first element is unsorted, it becomes the sorted element in the first iteration. - If the current unsorted array element is smaller than the sorted array element, the current element is shifted to one place higher than its current position. - The array size in our example is n = 5 and the total iterations performed will be (n-1). - The total number of comparisons required to sort: n = 5 elements is *(n-1) + (n-2) + (n-3) + (n-4).* You might recognize the similar pattern from Selection Sort. - In case of a completely sorted list, the Insertion Sort requires the same number of comparisons as that of an unsorted list. However, there will be no shifting or insertion of elements performed. - If there are duplicate elements in the sorting array, the Insertion Sort won’t move one element in front of the other, instead, the key value will be maintained in order. This characteristic makes the Insertion Sort a stable sorting algorithm. You may ask what is considered as stability in the sorting algorithm? ## Defining Stability ![Fig 1.1 Sorting Stability in an algorithm.](../../assets/blog/reviewing-sorting-algorithms-insertion-sort-3.png) ![Fig 1.2 Unstable sorting algorithm.](../../assets/blog/reviewing-sorting-algorithms-insertion-sort-4.png) ## Insertion Vs Selection Vs Bubble Sort ![Comparison table contrasting Insertion, Selection, and Bubble sort behavior](../../assets/blog/reviewing-sorting-algorithms-insertion-sort-5.png) **Time and space complexity** are listed in the below table: ![Table of time and space complexity for Insertion, Selection, and Bubble sort by best, average, and worst case](../../assets/blog/reviewing-sorting-algorithms-insertion-sort-6.png) ## Code Implementation: ```python # Python code for Insertion Sort # Additional Logs are used for demonstration purposes. def InsertionSort(arr): # Get the length of the array arrlen = len(arr) print(' Input Array : {} >> Length : {} '.format(arr,arrlen)) # iterate through the array elements starting from index[1], as we have to compare left index [0] for sorting for i in range(1,arrlen): print ('-----------------------------------------------------') print( 'Current Unsorted array : {} '.format(arr)) print ('At Index: {} >> unsortedValue value {} '.format(i,arr[i])) # Set up j value as a copy of i value j = i # set up the unsorted value to be compared to the left array of sorted values unsortedValue = arr[j] print ('Current unsortedValue array value : {}, value to the left : {}'.format(unsortedValue,arr[j-1])) # Loop from the j (copy of i) element to check for j-1 (left) index value to find the right sorting positon while j > 0 and unsortedValue < arr[j-1]: print ('As {} < {}, Move arr[j-1] value {} to the right at index {} '.format(unsortedValue,arr[j-1],arr[j-1],j)) # As unsortedValue is less then left index value, Move the left index value to right side arr[j] = arr[j-1] print ('At index {} - New value is {} '.format(j,arr[j])) # to check all the left index values, until the list is exhuasted. j = j-1 print (' >> Comparing Left of index: {}, Value {} with unsortedValue: {}'.format(j,arr[j-1],unsortedValue)) # ## ### ## ## # # ### ### # ### ## ## ## # ### # ## ### ## # ## ## # ### # ## ### ## # # The below If condition is not required >> It's added only for demonstration purpose #-------------------------------------------------------------------------------------- if unsortedValue < arr[j-1] and j > 0: print(' >> ({} < {}) => True >> Continue Checking left index values'.format(unsortedValue,arr[j-1])) else: print(' >> ({} < {}) => False and j : {} >> unsortedValue is at sorted position '.format(unsortedValue,arr[j-1],j)) # --------------------------------------------------------------------------------------- print ('At index {} >> Insert unsortedValue value {} '.format(j,unsortedValue)) # Insert unsortedValue to it's rightful position. arr[j]= unsortedValue print ('New array formation : {} '.format(arr)) return arr # Test Long unsorted list #arrlist = [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] # Test short unsorted list arrlist = [10,4,8,6,1] #Test sorted list #arrlist = [1,2,3,4,5] #Test n= 100 unsorted list #arrlist = [52,68,84,73,3,81,91,58,1,19,8,78,32,94,4,98,23,6,21,20,43,69,31,87,33,62,36,41,70,51,48,38,72,24,47,57,13,7,64,49,9,16,14,99,90,92,35,25,37,89,50,85,76,67,88,131,39,56,74,42,77,26,93,97,80,82,96,60,53,46,18,65,30,27,2,83,66,29,61,10,55,22,11,75,95,45,59,86,12,15,17,79,40,34,5,71,44,63,28,54] print(InsertionSort(arrlist)) ``` **Log Output:** Let’s check the log output to understand the case of a fully sorted array and unsorted array. ```text #------------------------------------------------------------------- # Test Case 1: Short unsorted list : arrlist = [10,4,8,6,1] #------------------------------------------------------------------- Input Array : [10, 4, 8, 6, 1] >> Length : 5 ----------------------------------------------------- Current Unsorted array : [10, 4, 8, 6, 1] At Index: 1 >> unsortedValue value 4 Current unsortedValue array value : 4, value to the left : 10 As 4 < 10, Move arr[j-1] value 10 to the right at index 1 At index 1 - New value is 10 >> Comparing Left of index: 0, Value 1 with unsortedValue: 4 >> (4 < 1) => False and j : 0 >> unsortedValue is at sorted position At index 0 >> Insert unsortedValue value 4 New array formation : [4, 10, 8, 6, 1] ----------------------------------------------------- Current Unsorted array : [4, 10, 8, 6, 1] At Index: 2 >> unsortedValue value 8 Current unsortedValue array value : 8, value to the left : 10 As 8 < 10, Move arr[j-1] value 10 to the right at index 2 At index 2 - New value is 10 >> Comparing Left of index: 1, Value 4 with unsortedValue: 8 >> (8 < 4) => False and j : 1 >> unsortedValue is at sorted position At index 1 >> Insert unsortedValue value 8 New array formation : [4, 8, 10, 6, 1] ----------------------------------------------------- Current Unsorted array : [4, 8, 10, 6, 1] At Index: 3 >> unsortedValue value 6 Current unsortedValue array value : 6, value to the left : 10 As 6 < 10, Move arr[j-1] value 10 to the right at index 3 At index 3 - New value is 10 >> Comparing Left of index: 2, Value 8 with unsortedValue: 6 >> (6 < 8) => True >> Continue Checking left index values At index 2 >> Insert unsortedValue value 6 As 6 < 8, Move arr[j-1] value 8 to the right at index 2 At index 2 - New value is 8 >> Comparing Left of index: 1, Value 4 with unsortedValue: 6 >> (6 < 4) => False and j : 1 >> unsortedValue is at sorted position At index 1 >> Insert unsortedValue value 6 New array formation : [4, 6, 8, 10, 1] ----------------------------------------------------- Current Unsorted array : [4, 6, 8, 10, 1] At Index: 4 >> unsortedValue value 1 Current unsortedValue array value : 1, value to the left : 10 As 1 < 10, Move arr[j-1] value 10 to the right at index 4 At index 4 - New value is 10 >> Comparing Left of index: 3, Value 8 with unsortedValue: 1 >> (1 < 8) => True >> Continue Checking left index values At index 3 >> Insert unsortedValue value 1 As 1 < 8, Move arr[j-1] value 8 to the right at index 3 At index 3 - New value is 8 >> Comparing Left of index: 2, Value 6 with unsortedValue: 1 >> (1 < 6) => True >> Continue Checking left index values At index 2 >> Insert unsortedValue value 1 As 1 < 6, Move arr[j-1] value 6 to the right at index 2 At index 2 - New value is 6 >> Comparing Left of index: 1, Value 4 with unsortedValue: 1 >> (1 < 4) => True >> Continue Checking left index values At index 1 >> Insert unsortedValue value 1 As 1 < 4, Move arr[j-1] value 4 to the right at index 1 At index 1 - New value is 4 >> Comparing Left of index: 0, Value 10 with unsortedValue: 1 >> (1 < 10) => False and j : 0 >> unsortedValue is at sorted position At index 0 >> Insert unsortedValue value 1 New array formation : [1, 4, 6, 8, 10] [1, 4, 6, 8, 10] #------------------------------------------------------------------------------------------------ # Test Case 2: Long unsorted list : arrlist = [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] #------------------------------------------------------------------------------------------------ Input Array : [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] >> Length : 13 ----------------------------------------------------- Current Unsorted array : [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] At Index: 1 >> unsortedValue value 56 Current unsortedValue array value : 56, value to the left : 3 New array formation : [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] At Index: 2 >> unsortedValue value 2 Current unsortedValue array value : 2, value to the left : 56 As 2 < 56, Move arr[j-1] value 56 to the right at index 2 At index 2 - New value is 56 >> Comparing Left of index: 1, Value 3 with unsortedValue: 2 >> (2 < 3) => True >> Continue Checking left index values At index 1 >> Insert unsortedValue value 2 As 2 < 3, Move arr[j-1] value 3 to the right at index 1 At index 1 - New value is 3 >> Comparing Left of index: 0, Value 45 with unsortedValue: 2 >> (2 < 45) => False and j : 0 >> unsortedValue is at sorted position At index 0 >> Insert unsortedValue value 2 New array formation : [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] At Index: 3 >> unsortedValue value 58 Current unsortedValue array value : 58, value to the left : 56 New array formation : [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] At Index: 4 >> unsortedValue value 79 Current unsortedValue array value : 79, value to the left : 58 New array formation : [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] At Index: 5 >> unsortedValue value 59 Current unsortedValue array value : 59, value to the left : 79 As 59 < 79, Move arr[j-1] value 79 to the right at index 5 At index 5 - New value is 79 >> Comparing Left of index: 4, Value 58 with unsortedValue: 59 >> (59 < 58) => False and j : 4 >> unsortedValue is at sorted postion At index 4 >> Insert unsortedValue value 59 New array formation : [2, 3, 56, 58, 59, 79, 34, 23, 4, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 56, 58, 59, 79, 34, 23, 4, 78, 8, 123, 45] At Index: 6 >> unsortedValue value 34 Current unsortedValue array value : 34, value to the left : 79 As 34 < 79, Move arr[j-1] value 79 to the right at index 6 At index 6 - New value is 79 >> Comparing Left of index: 5, Value 59 with unsortedValue: 34 >> (34 < 59) => True >> Continue Checking left index values At index 5 >> Insert unsortedValue value 34 As 34 < 59, Move arr[j-1] value 59 to the right at index 5 At index 5 - New value is 59 >> Comparing Left of index: 4, Value 58 with unsortedValue: 34 >> (34 < 58) => True >> Continue Checking left index values At index 4 >> Insert unsortedValue value 34 As 34 < 58, Move arr[j-1] value 58 to the right at index 4 At index 4 - New value is 58 >> Comparing Left of index: 3, Value 56 with unsortedValue: 34 >> (34 < 56) => True >> Continue Checking left index values At index 3 >> Insert unsortedValue value 34 As 34 < 56, Move arr[j-1] value 56 to the right at index 3 At index 3 - New value is 56 >> Comparing Left of index: 2, Value 3 with unsortedValue: 34 >> (34 < 3) => False and j : 2 >> unsortedValue is at sorted position At index 2 >> Insert unsortedValue value 34 New array formation : [2, 3, 34, 56, 58, 59, 79, 23, 4, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 34, 56, 58, 59, 79, 23, 4, 78, 8, 123, 45] At Index: 7 >> unsortedValue value 23 Current unsortedValue array value : 23, value to the left : 79 As 23 < 79, Move arr[j-1] value 79 to the right at index 7 At index 7 - New value is 79 >> Comparing Left of index: 6, Value 59 with unsortedValue: 23 >> (23 < 59) => True >> Continue Checking left index values At index 6 >> Insert unsortedValue value 23 As 23 < 59, Move arr[j-1] value 59 to the right at index 6 At index 6 - New value is 59 >> Comparing Left of index: 5, Value 58 with unsortedValue: 23 >> (23 < 58) => True >> Continue Checking left index values At index 5 >> Insert unsortedValue value 23 As 23 < 58, Move arr[j-1] value 58 to the right at index 5 At index 5 - New value is 58 >> Comparing Left of index: 4, Value 56 with unsortedValue: 23 >> (23 < 56) => True >> Continue Checking left index values At index 4 >> Insert unsortedValue value 23 As 23 < 56, Move arr[j-1] value 56 to the right at index 4 At index 4 - New value is 56 >> Comparing Left of index: 3, Value 34 with unsortedValue: 23 >> (23 < 34) => True >> Continue Checking left index values At index 3 >> Insert unsortedValue value 23 As 23 < 34, Move arr[j-1] value 34 to the right at index 3 At index 3 - New value is 34 >> Comparing Left of index: 2, Value 3 with unsortedValue: 23 >> (23 < 3) => False and j : 2 >> unsortedValue is at sorted position At index 2 >> Insert unsortedValue value 23 New array formation : [2, 3, 23, 34, 56, 58, 59, 79, 4, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 23, 34, 56, 58, 59, 79, 4, 78, 8, 123, 45] At Index: 8 >> unsortedValue value 4 Current unsortedValue array value : 4, value to the left : 79 As 4 < 79, Move arr[j-1] value 79 to the right at index 8 At index 8 - New value is 79 >> Comparing Left of index: 7, Value 59 with unsortedValue: 4 >> (4 < 59) => True >> Continue Checking left index values At index 7 >> Insert unsortedValue value 4 As 4 < 59, Move arr[j-1] value 59 to the right at index 7 At index 7 - New value is 59 >> Comparing Left of index: 6, Value 58 with unsortedValue: 4 >> (4 < 58) => True >> Continue Checking left index values At index 6 >> Insert unsortedValue value 4 As 4 < 58, Move arr[j-1] value 58 to the right at index 6 At index 6 - New value is 58 >> Comparing Left of index: 5, Value 56 with unsortedValue: 4 >> (4 < 56) => True >> Continue Checking left index values At index 5 >> Insert unsortedValue value 4 As 4 < 56, Move arr[j-1] value 56 to the right at index 5 At index 5 - New value is 56 >> Comparing Left of index: 4, Value 34 with unsortedValue: 4 >> (4 < 34) => True >> Continue Checking left index values At index 4 >> Insert unsortedValue value 4 As 4 < 34, Move arr[j-1] value 34 to the right at index 4 At index 4 - New value is 34 >> Comparing Left of index: 3, Value 23 with unsortedValue: 4 >> (4 < 23) => True >> Continue Checking left index values At index 3 >> Insert unsortedValue value 4 As 4 < 23, Move arr[j-1] value 23 to the right at index 3 At index 3 - New value is 23 >> Comparing Left of index: 2, Value 3 with unsortedValue: 4 >> (4 < 3) => False and j : 2 >> unsortedValue is at sorted position At index 2 >> Insert unsortedValue value 4 New array formation : [2, 3, 4, 23, 34, 56, 58, 59, 79, 78, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 4, 23, 34, 56, 58, 59, 79, 78, 8, 123, 45] At Index: 9 >> unsortedValue value 78 Current unsortedValue array value : 78, value to the left : 79 As 78 < 79, Move arr[j-1] value 79 to the right at index 9 At index 9 - New value is 79 >> Comparing Left of index: 8, Value 59 with unsortedValue: 78 >> (78 < 59) => False and j : 8 >> unsortedValue is at sorted position At index 8 >> Insert unsortedValue value 78 New array formation : [2, 3, 4, 23, 34, 56, 58, 59, 78, 79, 8, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 4, 23, 34, 56, 58, 59, 78, 79, 8, 123, 45] At Index: 10 >> unsortedValue value 8 Current unsortedValue array value : 8, value to the left : 79 As 8 < 79, Move arr[j-1] value 79 to the right at index 10 At index 10 - New value is 79 >> Comparing Left of index: 9, Value 78 with unsortedValue: 8 >> (8 < 78) => True >> Continue Checking left index values At index 9 >> Insert unsortedValue value 8 As 8 < 78, Move arr[j-1] value 78 to the right at index 9 At index 9 - New value is 78 >> Comparing Left of index: 8, Value 59 with unsortedValue: 8 >> (8 < 59) => True >> Continue Checking left index values At index 8 >> Insert unsortedValue value 8 As 8 < 59, Move arr[j-1] value 59 to the right at index 8 At index 8 - New value is 59 >> Comparing Left of index: 7, Value 58 with unsortedValue: 8 >> (8 < 58) => True >> Continue Checking left index values At index 7 >> Insert unsortedValue value 8 As 8 < 58, Move arr[j-1] value 58 to the right at index 7 At index 7 - New value is 58 >> Comparing Left of index: 6, Value 56 with unsortedValue: 8 >> (8 < 56) => True >> Continue Checking left index values At index 6 >> Insert unsortedValue value 8 As 8 < 56, Move arr[j-1] value 56 to the right at index 6 At index 6 - New value is 56 >> Comparing Left of index: 5, Value 34 with unsortedValue: 8 >> (8 < 34) => True >> Continue Checking left index values At index 5 >> Insert unsortedValue value 8 As 8 < 34, Move arr[j-1] value 34 to the right at index 5 At index 5 - New value is 34 >> Comparing Left of index: 4, Value 23 with unsortedValue: 8 >> (8 < 23) => True >> Continue Checking left index values At index 4 >> Insert unsortedValue value 8 As 8 < 23, Move arr[j-1] value 23 to the right at index 4 At index 4 - New value is 23 >> Comparing Left of index: 3, Value 4 with unsortedValue: 8 >> (8 < 4) => False and j : 3 >> unsortedValue is at sorted position At index 3 >> Insert unsortedValue value 8 New array formation : [2, 3, 4, 8, 23, 34, 56, 58, 59, 78, 79, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 4, 8, 23, 34, 56, 58, 59, 78, 79, 123, 45] At Index: 11 >> unsortedValue value 123 Current unsortedValue array value : 123, value to the left : 79 New array formation : [2, 3, 4, 8, 23, 34, 56, 58, 59, 78, 79, 123, 45] ----------------------------------------------------- Current Unsorted array : [2, 3, 4, 8, 23, 34, 56, 58, 59, 78, 79, 123, 45] At Index: 12 >> unsortedValue value 45 Current unsortedValue array value : 45, value to the left : 123 As 45 < 123, Move arr[j-1] value 123 to the right at index 12 At index 12 - New value is 123 >> Comparing Left of index: 11, Value 79 with unsortedValue: 45 >> (45 < 79) => True >> Continue Checking left index values At index 11 >> Insert unsortedValue value 45 As 45 < 79, Move arr[j-1] value 79 to the right at index 11 At index 11 - New value is 79 >> Comparing Left of index: 10, Value 78 with unsortedValue: 45 >> (45 < 78) => True >> Continue Checking left index values At index 10 >> Insert unsortedValue value 45 As 45 < 78, Move arr[j-1] value 78 to the right at index 10 At index 10 - New value is 78 >> Comparing Left of index: 9, Value 59 with unsortedValue: 45 >> (45 < 59) => True >> Continue Checking left index values At index 9 >> Insert unsortedValue value 45 As 45 < 59, Move arr[j-1] value 59 to the right at index 9 At index 9 - New value is 59 >> Comparing Left of index: 8, Value 58 with unsortedValue: 45 >> (45 < 58) => True >> Continue Checking left index values At index 8 >> Insert unsortedValue value 45 As 45 < 58, Move arr[j-1] value 58 to the right at index 8 At index 8 - New value is 58 >> Comparing Left of index: 7, Value 56 with unsortedValue: 45 >> (45 < 56) => True >> Continue Checking left index values At index 7 >> Insert unsortedValue value 45 As 45 < 56, Move arr[j-1] value 56 to the right at index 7 At index 7 - New value is 56 >> Comparing Left of index: 6, Value 34 with unsortedValue: 45 >> (45 < 34) => False and j : 6 >> unsortedValue is at sorted position At index 6 >> Insert unsortedValue value 45 New array formation : [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 79, 123] [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 79, 123] #------------------------------------------------------------------- # Test Case 3: sorted list : arrlist = [1,2,3,4,5] #------------------------------------------------------------------- Input Array : [1, 2, 3, 4, 5] >> Length : 5 ----------------------------------------------------- Current Unsorted array : [1, 2, 3, 4, 5] At Index: 1 >> unsortedValue value 2 Current unsortedValue array value : 2, value to the left : 1 New array formation : [1, 2, 3, 4, 5] ----------------------------------------------------- Current Unsorted array : [1, 2, 3, 4, 5] At Index: 2 >> unsortedValue value 3 Current unsortedValue array value : 3, value to the left : 2 New array formation : [1, 2, 3, 4, 5] ----------------------------------------------------- Current Unsorted array : [1, 2, 3, 4, 5] At Index: 3 >> unsortedValue value 4 Current unsortedValue array value : 4, value to the left : 3 New array formation : [1, 2, 3, 4, 5] ----------------------------------------------------- Current Unsorted array : [1, 2, 3, 4, 5] At Index: 4 >> unsortedValue value 5 Current unsortedValue array value : 5, value to the left : 4 New array formation : [1, 2, 3, 4, 5] [1, 2, 3, 4, 5] ``` **Unsorted array:** There are two loops performed, one is the outer loop for array iteration, and another is the inner loop for comparisons. At the start of the sorting, the first element (10) is considered as a sorted subset and the inner loop will compare the first unsorted element (4) against the sorted element. As the unsorted array shrinks on every iteration, the sorted array inner loop keeps growing to shift the elements. The inner loop will run for *(n-1)* elements wherein *‘n’* is the total number of array elements. For instance, for the last iteration in the test case, the array index [4] = 1 element is compared against (n-1) = 4 sorted array elements. If you observe closely, the inner loops iterate from right to left to compare against the sorted array list. The double looping is what makes the Insertion Sort worst-case time complexity О(*n^2*) because if the input doubles the running time will be quadratic. **Sorted array:** In test case 3, the log shows no inner loops, as the values are already sorted. However, the comparisons are performed to check the rightful sorting position. Insertion Sort requires ‘n’ steps to sort an already sorted array of ‘n’ elements, which makes its best-case time complexity a linear function of O(n). ## Final thoughts So next time you sort out cards manually or play bridge, remember it’s an Insertion Sort. It is yet another simple algorithm that is slow due to its internal looping. In all fairness, Insertion Sort has an edge over its peers Bubble and Selection Sort as it has a best-case running time for an already sorted array list. Moreover, it is a stable algorithm with in-place properties and the ability to sort a list as it is received. We have arrived at the most awaited conclusion, the Insertion Sort is not “The One” we were looking out for. Well, our quest to find “The One” continues and looks like we are getting really close. In the next post, we shall discuss “Merge Sort” to continue our pursuit of finding the most efficient sorting algorithm. Stay tuned!! I hope you enjoyed the post. Please subscribe to [ProstDev](https://www.prostdev.com/) for more exciting topics. Hasta luego, amigos! ## Study Resources Below are some resources which discuss Insertion Sort and its Time-Space complexity at length. - [The Insertion Sort - Runestone academy](https://runestone.academy/runestone/books/published/pythonds/SortSearch/TheInsertionSort.html) - [Insertion Sort - Wikipedia](https://en.wikipedia.org/wiki/Insertion_sort) - [Video] [Insertion Sort - CS50 Harvard](https://www.youtube.com/watch?v=O0VbBkUvriI) - [Analysis of Insertion Sort - Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/insertion-sort/a/analysis-of-insertion-sort) - [Video] [Insertion Sort - CS310 San Diego University](https://www.youtube.com/watch?v=eTvQIbB-AuE) - [Video] [Insert Sort with Romanian Folk Dance](https://www.youtube.com/watch?v=ROalU379l3U) - [Asymptotic-Notation- Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/asymptotic-notation) - [Big-O cheat sheet](https://www.bigocheatsheet.com/) - [Insertion Sort Animation](https://www.toptal.com/developers/sorting-algorithms/insertion-sort) - [Stable Sorting Algorithms](https://www.baeldung.com/cs/stable-sorting-algorithms) - [Sorting Algorithms Stability](https://en.wikipedia.org/wiki/Sorting_algorithm#Stability) - [A Comparative Study of Selection Sort and Insertion Sort Algorithms](https://www.irjet.net/archives/V3/i12/IRJET-V3I12115.pdf) Fahriye Gemci FuratResearch Assistant, Department of Computer Engineering, Iskenderun Technical University, Hatay, Turkey --- ## Dell Boomi Integrations: Using the Atom Queue Connector Source: https://prostdev.com/post/dell-boomi-integrations-using-the-atom-queue-connector | Published: Jul 28, 2020 | Category: Tutorials The Atom Queue connector is used to send and receive messages to and from native Boomi Integration Atom message queues. The connector supports both Point-to-Point and Publish/Subscribe messaging. For better understanding of how Atom Queue works, I did a proof of concept. Below are the screenshots of the flow which will give you a better idea of how this component works. ## 1. Creating a message and sending it to an Atom Queue To send a message to another process, first you need to create the process where we’ll send the queue message from (in this case, we create the Send_Queue process). ![Boomi Send_Queue process: Start (No Data), Message, Notify, Atom Queue, then Stop](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-1.png) The “No Data” option should be selected in the “Start” Shape connector, if the process will not receive or retrieve data from any source. In the Message connector, I have entered a “Hello” message (as shown below). This is the message that will be sent to the second process. ![Boomi Message Shape config holding a JSON "Hello" demo message](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-2.png) The message sent from the Message Shape is sent to the Atom Queue connector, which will be sending the message to native Boomi Integration Atom Message Queues. ![Boomi Connector Shape set to Atom Queue connector with the Send action](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-3.png) To configure a connector to send a message to an Atom Message Queue, I need to set up two components: - Atom Queue Connection - Atom Queue Operation The Atom Queue Connection specifies the message queue for an Atom Queue Operation. As shown below, I have configured the Atom Queue Connection with the Queue Name “AtomQueue_Test”. ![New Atom Queue Connection config with Queue set to AtomQueue_Test](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-4.png) The Atom Queue Operation below is used to send (this is the Connection Action) messages to a Message Queue and Batch Size is set to 5 (default value), which means it sets the number of documents to batch in each outgoing message. ![Atom Queue Operation set to the Send action with Batch Size of 5](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-5.png) The Stop Shape indicates the “End and continue” of the flow. ## 2. Reading the message from the Atom Queue After we send the message to the queue, the next step would be to read that message from the second flow. I created the below process (Read_Queue) to read the file that was previously sent to the queue. ![Boomi Read_Queue process: Start (Atom Queue), Notify, then Stop](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-6.png) In the Start Shape connector, “Atom Queue” must be selected as the connector, and the Action must be selected as “Listen”. ![Boomi Start Shape with Atom Queue connector and the Listen action selected](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-7.png) The connection will be the same as the Atom Queue Connection we used in the Send Process, and the operation will be set to Listen (Connector Action). “Listen” is set in event-driven retrieval of messages from a Message Queue. To receive first-in, first-out (FIFO) processing of messages across nodes, set the Maximum Concurrent Executions value to 1. ![Atom Queue Operation set to Listen with Maximum Concurrent Executions of 1](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-8.png) I used a Notify Shape to send a customized message. Here I am just using the Current Data option, which will print the message that was received from the Atom Queue. ![Boomi Notify Shape config set to print the Current Data message](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-9.png) After these two processes are ready, the next step is to create a packaged component for the “Read_Queue” (Listen) process and deploy it. After deploying, the process can be seen in the Atom Management page -> Listeners, which means the process is in Listen mode. ![Atom Management Listeners page showing Read_Queue in listen mode](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-10.png) Next step is to test the process in Test Mode. ![Boomi Test dialog with a Test Atom Cloud selected and a Run Test button](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-11.png) Since the “Read_Queue” is already deployed, I will run the first process (in this case, “Send_Queue”). After the execution is completed without any errors, the message will be sent to the queue, which can be seen from the “Process Reporting” tab, as shown below. ![Boomi Executions report showing a completed Read_Queue process run](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-12.png) Click on the “View Process Logs” icon. The message can be seen from the Notify Shape, which prints the data that is passed from the first process (Send_Queue). ![Boomi process log showing the Notify shape printing the received "Hello" message](../../assets/blog/dell-boomi-integrations-using-the-atom-queue-connector-13.png) ## Errors Common errors when using Atom Queues are: - If message delivery fails, the Dead Letter Queue will be created corresponding to the Destination Queue and the failed messages will be placed in the Dead Letter Queue after 6 retry attempts by the shared queue server of the same Atom Queue component. The messages in the destination Dead Letter Queue can be manually retried from the “Queue Management” tab, via the Platform UI. - If the destination is other than an Atom Queue Server and there is a need to catch the failed messages in case of any failures, the best way to handle this scenario would be to re-queue the failed messages in the catch branch of a try/catch. They can either be placed back on the original queue or they could be placed on some sort of retry queue. --- ## Understanding the "illegal base64 character" error (Java, Groovy and Mule 4 - DW 2.0) Source: https://prostdev.com/post/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0 | Published: Jul 21, 2020 | Category: Tutorials Are you familiar with the fromBase64 or the toBase64 functions from DataWeave 2.0? What about the getUrlEncoder or the getEncoder functions from Java? Do you know the differences between the “basic” Base 64 encoding and the “URL and Filename safe” Base 64 encoding? Well, you may have guessed it by now, but you’re about to find out the answers to these questions! Or maybe you’re here because you keep getting the “Illegal base64 character” error in DataWeave. Even if you get this error using any other programming language, this post can help you understand why it is happening. ## The Problem I was recently presented with this problem when Maria Isabel Vargas asked a question about base64 decoding in a Slack channel. She was using the [fromBase64](https://docs.mulesoft.com/mule-runtime/4.3/dw-binaries-functions-frombase64) function that’s available in Mule 4 - DataWeave 2.0 inside the [dw::core::Binaries](https://docs.mulesoft.com/mule-runtime/4.3/dw-binaries) module to transform a basic Base 64 string into a binary value. The problem was that the server was returning a Base 64 **URL** Encoded String (e.g. “cHJvc3RkZXY**_**YmxvZw==”) opposed to the **basic** Base 64 string from which the fromBase64 function attempts to transform. As a result, the Transform Message component was returning this error: “Illegal base64 character 5f". This error happens when the string that you are trying to transform contains a character not recognized by the **basic** Base 64 Alphabet (in this case it was an underscore character). Below you can see which characters are accepted. ![ietf.org - RFC 4648](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-2.png) Since the server was using the Base 64 **URL** encoding, the string that we were trying to decode in DataWeave contained different characters from the ones above because the Base 64 URL has a different alphabet. You can see this alphabet below. ![ietf.org - RFC 4648](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-3.png) Notice that the characters 62 and 63 differ from the **basic** Base 64 Alphabet and the Base 64 **URL** Alphabet. The first one contains the characters plus (+) and slash (/), while the second one uses the characters minus (-) and underline (_). In other words, if you have an encoded string like "cHJvc3RkZXY_YmxvZw==", you wouldn’t be able to transform it using a **basic** Base64 decoder because it contains characters that are not recognized by its alphabet (the underline character). You can get these two errors when you try to transform a Base 64 **URL** string using the fromBase64 DW function (at least when this post was created): - java.lang.IllegalArgumentException: Illegal base64 character 5f (when containing an underscore) - java.lang.IllegalArgumentException: Illegal base64 character 2d (when containing a minus) ![DataWeave Playground running fromBase64 on a URL-encoded string and throwing an Illegal base64 character error.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-4.png) ## The Solution Java contains a function called [getUrlDecoder](https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html#getUrlEncoder--), which is used to decode a string that was encoded using the Base 64 **URL** alphabet. This is exactly the problem that we were trying to solve. But before you start panicking and coding your solution using Java and creating a whole Java class, it's way easier to just use the [Mule 4 Scripting Module](https://docs.mulesoft.com/scripting-module/1.1/). I’ll show you how. This module should already be installed in your Studio. However, if you can’t see it, you can download it from Exchange. Alternatively, you can use the “Add Modules” button from Anypoint Studio, which is located on your Mule Palette. ![Anypoint Studio Mule Palette with the Add Modules and Search in Exchange options.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-5.png) After that, you can simply take the Execute component and drag-and-drop it into your flow. For this demonstration, I will be using an HTTP Listener as a trigger, and I will send the encoded string in the body of the request from Postman. Once you have the Execute component, you can select “Groovy” as your Engine and paste the following line of code into your script: ```groovy Base64.getUrlDecoder().decode(payload) ``` You should end up with this: ![Flow with an HTTP Listener and an Execute component whose Groovy code calls Base64.getUrlDecoder().decode(payload).](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-6.png) As you can see in this Groovy script, we are using the getUrlDecoder function to decode whatever is in our payload. The result will be returned to our Postman application in the body of the response. You need to make sure that the payload that is sent to the Groovy script is a Java String. So, let’s add a Transform Message right after the HTTP Listener and transform the payload into a Java string. Like this: ![Flow with a Transform Message added between the Listener and Execute, casting the payload as String for Java.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-7.png) Let’s start this Mule app and test! We send the same string as before (cHJvc3RkZXY_YmxvZw==) from Postman, and we expect to get the decoded message as a response. ![Postman POST to localhost:8081/flow1 returning 200 OK with the decoded "prostdev?blog" response.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-8.png) And… it worked! ## Try it yourself Do you want to experiment a bit more with all of this? For sure! Let me give you a quick guide on how to do this. ### 1. Encode the bytes/string into a Basic 64 URL encoded string You will need Java (or you can also do it in Mule 4 using a Groovy script) to get the encoded string first. You don’t need to download and install Java onto your computer to do this; you can search for a free online Java compiler like this one: [TutorialsPoint Java Compiler](https://www.tutorialspoint.com/compile_java_online.php) There, just copy and paste this Java code: ```java import java.util.Base64; public class Base64UrlEncoding { public static void main(String []args) { byte[] bytes = "prostdev?blog".getBytes(); String encodedString = new String(Base64.getUrlEncoder().encode(bytes)); System.out.println(encodedString); } } ``` Next, click on “Execute” in the top left corner. You will see the encoded string on the right side of the website (the last line in the white background). Copy this string and save it somewhere so that you can use it in the next steps. ![Online Java compiler running the Base64UrlEncoding class, printing the encoded string cHJvc3RkZXY_YmxvZw==.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-9.png) ### 2. Create your Mule App in Anypoint Studio Follow the steps explained previously in “The Solution”, or paste this code into the XML configuration of your file: ```xml ``` ### 3. Create a Postman request If you don’t have Postman, you can download it for free from here: [Postman Download](https://www.postman.com/downloads/). When you open it, you just have to click on the plus (+) button as you can see here: ![Postman with no collections yet and an arrow pointing at the plus button to create a new request.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-10.png) Then, add this in the Request URL section: ``` localhost:8081/flow1 ``` And add the previously copied string inside the Request Body. Don’t forget to select “Raw” and “Text” from the two dropdowns that are over the request body text area. ![Postman request set to localhost:8081/flow1 with the encoded string in a raw Text request body.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-11.png) ### 4. Run your Mule App Once it is deployed successfully, you can click on the big blue “Send” button from Postman and see the results which should match the string you encoded in Step 1 (using Java). ![Anypoint Studio console showing the testbase64 app starting and reaching DEPLOYED status.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-12.png) ![Postman POST to localhost:8081/flow1 returning 200 OK with the decoded "prostdev?blog" response.](../../assets/blog/understanding-the-illegal-base64-character-error-java-groovy-and-mule-4-dw-2-0-8.png) ## Recap Let’s do a quick recap on what we just learned from this post: - The **basic** Base 64 alphabet can recognize the characters plus (+) and slash (/). - The Base 64 **URL** alphabet can recognize the characters minus (-) and underline (_). - Depending on how a string was encoded (using basic Base 64 or Base 64 URL encoding), it can include any of the characters from the chosen alphabet. - An encoded string with one alphabet may not be decoded using the other alphabet’s decoding system. If the decoder attempts to read an unrecognized character, it’ll send an Illegal base64 character error message. - You **can’t** decode a Base 64 **URL** string with the fromBase64 DataWeave 2.0 function. - You **can** decode a Base 64 **URL** string by using an Execute component along with a Groovy engine and the getUrlDecoder Java function. Had you seen this error before? How did you solve it? *Prost!* -Alex ## References - [Mule docs - fromBase64 function](https://docs.mulesoft.com/mule-runtime/4.3/dw-binaries-functions-frombase64) - [Mule docs - dw::core::Binaries](https://docs.mulesoft.com/mule-runtime/4.3/dw-binaries) - [Mule docs - Scripting Module](https://docs.mulesoft.com/scripting-module/1.1/) - [IETF - RFC4648](https://www.ietf.org/rfc/rfc4648.txt) - [Oracle docs - getUrlEncoder](https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html#getUrlEncoder--) - [TutorialsPoint - Online Java compiler](https://www.tutorialspoint.com/compile_java_online.php) - [Download Postman](https://www.postman.com/downloads/) ### GitHub repository [ProstDev GitHub - testbase64](https://github.com/ProstDev/testbase64) --- ## Custom Modules in Mule 4 - DataWeave 2.0 Source: https://prostdev.com/post/custom-modules-in-mule-4-dataweave-2-0 | Published: Jul 16, 2020 | Category: Tutorials *GitHub repository with the Mule Project can be found at the end of the post.* Some days ago, I started using this great functionality in DW 2.0, and I faced some challenges while creating the code, so I figured I’d write a blog post about my experience. [MuleSoft’s documentation](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-create-module) offers a great explanation of how to create these custom modules. One problem that I couldn’t find how to solve in the documentation was how to correctly load a property from the properties file into the custom module. If you’re not familiar with custom modules, don’t worry, I got your back ;) For this post, I’ll explain what a custom module is, why you would want to use it, some examples on how you can use it, and finally, how to load properties into this module. ## What are Custom Modules in DataWeave 2.0? First of all, let me tell you what a regular Module is in DW 2.0: from [MuleSoft’s documentation](https://docs.mulesoft.com/mule-runtime/4.3/dw-functions), “*DataWeave 2.0 functions are packaged in modules. Functions in the Core (**dw::Core**) module are imported automatically into your DataWeave scripts.*”. Yes, all those functions that you use in your DW scripts, come from the Core module. Functions like [++](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-plusplus), [flatten](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-flatten), [map](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-map), [joinBy](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-joinby), that you may use on your day-to-day, come from Modules, you just don’t need to explicitly import them into your script. Other developers may call them [libraries](https://en.wikipedia.org/wiki/Library_(computing)#:~:text=In%20computer%20science%2C%20a%20library,classes%2C%20values%20or%20type%20specifications.) however in DW 2.0 they are called modules. Mule offers a wide choice of modules that you can use to transform and manipulate your data, like the [Arrays](https://docs.mulesoft.com/mule-runtime/4.3/dw-arrays) module, the [Strings](https://docs.mulesoft.com/mule-runtime/4.3/dw-strings) module, the [Objects](https://docs.mulesoft.com/mule-runtime/4.3/dw-objects) module, even [Binaries](https://docs.mulesoft.com/mule-runtime/4.3/dw-binaries) or [Tree](https://docs.mulesoft.com/mule-runtime/4.3/dw-tree). You don’t need to know how all of these work, just understand the basic meaning of a module, which is: functions created by MuleSoft that can be reused into your DW scripts. Custom Modules are pieces of code that can be stored into an external file, in order to be reused in other parts of the Mule application. These are not created by MuleSoft, but by yourself, or your company. ## Why are they useful? Here is an example to illustrate why Custom Modules are useful: you just created a new function that will join an array of words into a complete sentence. Like this: ![Transform Message joining a words array with joinBy into the sentence "Hello ProstDev :)"](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-1.png) As you can see, we are using the joinBy function in the code to select which character we want our words to be separated by, in our complete sentence. If we were to change our code from joinBy “ “ to joinBy “-”, our final output would be "Hello-ProstDev-:)" instead. If this piece of code, to create a complete sentence from an array of words, was used in several parts of your application, it would be a good idea to create a function inside a Custom Module to store this code. Then just reference that function from anywhere else in the rest of the app; instead of having to copy and paste this code into each part where you need to use it. Let’s see how we can create this module for our Mule Application: - Create a “modules” folder inside your `src/main/resources` folder, and then create a new “.dwl” file inside the new modules folder, you can name it something like `Custom.dwl` (you can name the folder and the file however you prefer). ![Project tree showing a new Custom.dwl file inside a modules folder under src/main/resources](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-2.png) - Inside this file, create a new function and paste the code that we had just created to generate the output sentence from our words array (note that you don’t need to add the “output” data type, or the 3 dashes (-) before the code). ![Custom.dwl defining a getSentenceFromWords function that joins words by a space](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-3.png) - Save everything, and then go back to the Transform Message where we have our DW script. Import the new function from the “Custom” module by writing import getSentenceFromWords from modules::Custom under the output statement. Now you can simply call this function in your code to use the Custom Module and we’ll get the same results as before. ![Transform Message importing getSentenceFromWords from modules::Custom and calling it](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-4.png) Of course, this is a very simple example, it’s only one line of code. You may think that we generated even more code by doing this than by copying and pasting the code in several scripts. Fair enough. But, imagine what would happen if you suddenly decide to change the functionality? You’d have to go to each of the DW scripts and manually change every single line of code. If you have a custom module setup, you’d only have to change the function from the `Custom.dwl` file, and all the scripts referencing it will automatically have the new functionality. Copy and Pasting means you have to go through and change each one but with this method it's done automatically - for graphic designers out there it's like the embed feature in photoshop. ## Loading properties into a Custom Module Remember that character we used after the joinBy function? What if we wanted that character to be in a properties file, instead of hardcoded (added directly) in our script? Can we do that when using Custom Modules? Yes, we can! I already have my `default-properties.yaml` file created under `src/main/resources`, with one property that I decided to name “joinByChar” ![default-properties.yaml file defining a joinByChar property set to a space](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-5.png) We can go back to our `Custom.dwl` file and change the script to reference this property instead of hardcoding the character there. You may already know that the properties can be used in your Mule Application by using either the ${ } or the p(‘ ‘) syntax. However, if you try to use any of these from a Custom Module, DataSense will send you back an error (at least for Mule Runtime version 4.3.0 and Anypoint Studio version 7.5.0). I was losing my mind trying to search in the [Mule Docs](https://docs.mulesoft.com/general/) or in the [Mule Forums](https://help.mulesoft.com/s/forum) how I could import a property from my custom module. In the end, one of the [Mule Ambassadors](https://developer.mulesoft.com/dev/ambassadors), Manish Yadav, was able to point me in the right direction. Remember at the beginning of this post, when I explained that all the DataWeave functions come from a module that is created by MuleSoft. Well, when you use this function: p(‘my-property’) to load a property into the script, do you know which module it comes from? It comes from the [Mule module](https://docs.mulesoft.com/mule-runtime/4.3/dw-mule)! We can rewrite our function like this: ![Custom.dwl rewritten to read the separator via Mule::p('joinByChar') instead of hardcoding it](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-6.png) Aaaaand… Magic! We have our same output, but now we’re using an external property to generate the final sentence. ![Mule flow and Transform Message still output "Hello ProstDev :)" using the space property](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-7.png) This way we can change our property joinByChar right from the `default-properties.yaml` file and we do not need to touch the code to select a new character. ![Changing joinByChar to a dash makes the output become "Hello-ProstDev-:)" with no code edits](../../assets/blog/custom-modules-in-mule-4-dataweave-2-0-8.png) Feel free to download the Mule project from our github repository and give it a try! Maybe you get it to work with another syntax :) I’d love to keep learning more DataWeave functions. ## Recap Let’s do a quick recap on what we learned with this post: - DataWeave 2.0 functions are packaged in modules. - Modules are created by MuleSoft and you can use their predefined functions in your DW scripts by importing the module, using the import keyword. - Custom Modules are just like regular modules, but they are created by you or your company, as opposed to MuleSoft. - Modules are useful because they let you reuse the same piece of code in different DW scripts. - Custom Modules are stored into an external file and can be imported into your DW script the same way that a regular module is imported: using the import keyword. Special thanks to Manish for this tip and for being so quick to respond to my messages! Stay tuned for more DataWeave tips and tricks. *Prost!* -Alex ## References - [Mule docs - MuleSoft Documentation](https://docs.mulesoft.com/general/) - [Mule docs - Create Custom Modules and Mappings](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-create-module) - [Mule docs - DataWeave Reference](https://docs.mulesoft.com/mule-runtime/4.3/dw-functions) - [Mule docs - joinBy function](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-joinby) - [Mule docs - Mule Module](https://docs.mulesoft.com/mule-runtime/4.3/dw-mule) - [Mule Forum](https://help.mulesoft.com/s/forum) - [Mule Ambassadors](https://developer.mulesoft.com/dev/ambassadors) - [Wikipedia - Library](https://en.wikipedia.org/wiki/Library_(computing)#:~:text=In%20computer%20science%2C%20a%20library,classes%2C%20values%20or%20type%20specifications) ## GitHub repository [ProstDev GitHub - Custom Modules DW 2.0](https://github.com/ProstDev/custom-modules-dw2) --- ## Reviewing Sorting Algorithms: Selection Sort Source: https://prostdev.com/post/reviewing-sorting-algorithms-selection-sort | Published: Jul 14, 2020 | Category: Guides Let's sort it out! In a series of posts, we will be discussing some of the sorting algorithms listed in the below order: - [Bubble Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-bubble-sort) - **Selection Sort** - [Insertion Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-insertion-sort) - [Merge Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-merge-sort) - Quick Sort - Heap Sort In this post, we will explore the next, in a series of sorting algorithms, the selection sort. If you feel nostalgic about sorting algorithms, I don’t blame you and it’s totally fine if you want to skim through the first post of the series on [Bubble Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-bubble-sort). Good read eh! Now you know that bubble sort’s time and space complexity is kind of meh! We crave for a faster and more efficient algorithm to sort the long list of input elements. Is “Selection Sort” the answer? Maybe. ## Why is it called Selection Sort? The sorting algorithm gets its name from the iterations it performs through input arrays or elements. The algorithm selects the current smallest element and swaps it into its sorted place and continues doing it until the list is sorted. It creates two sections of the arrays; one section is unsorted, and another is sorted. In short, we will be selecting one item at a time by its size and moving it to its sorted position. Breaking down the instructions: - Given an unsorted array, set the first index [0] element as the smallest number. - Iterate through the array to find the smallest number. - Now swap the smallest number as Index [0] and the original index [0] value to index of the smallest number. - Continue the above process for the rest of the unsorted array until the last element is reached. I know, it’s like “yeah I get it, but wait I’m lost” and that’s why we have visuals! Yay! We will pick up the same numbers we introduced during the bubble sort as an example. Yes! I’m trying to make you compare the techniques already. Let’s play with the selections: **Problem:** Sort the numbers [10, 4, 8, 6, 1]. Let’s start with the first *iteration* of the unsorted list based on our understanding of the above instruction. ![Step-by-step selection sort iterations sorting the array [10, 4, 8, 6, 1]](../../assets/blog/reviewing-sorting-algorithms-selection-sort-1.png) **Observations based on Iterations:** - In our example, the array size is *n=5* and 4 iterations were performed. Similarly, for *‘n’* numbers *‘n-1’* iterations will be performed. - Only one element gets sorted on each pass. - The number of comparisons needed for first iterations was *(n-1),* as we compared 4 elements to find the smallest number. Likewise, comparisons needed for the second iteration is (n-2), third is (n-3), and so on. - The total number of comparisons required to sort n=5 elements are *(n-1) +(n-2) +(n-3) +(n-4)*. From our example problem length of array is *n= 5,* no. of comparisons is 4+3+2+1 = 10. - Summation of the above stated turns out to be *‘n(n-1)/2’*. So, guess what would be the total # of comparisons if n=100. - In the case of a sorted list, selection sort requires the same number of steps as running through an unsorted list. The default implementation of selection sort is not stable; however, it can be made stable if required. To explore more about stable selection sort, refer to this [link](https://www.geeksforgeeks.org/stable-selection-sort/). ## Selection Vs Bubble Sort - In bubble sort, the adjacent element is compared and swapped, whereas in selection sort the smallest element is selected to swap against the largest element. - The selection sort has improved efficiency and is faster when compared to bubble sort. - The number of swaps is O(n) in comparison to О(*n^*2) of bubble sort. In our example , we have very few swaps when compared to swaps in bubble sort. - The worst-case complexity in both algorithms is О(*n^*2). In the case of an unsorted list, selection sort has to iterate over each of the ‘n’ elements to find the smallest number. One has to repeat ‘n’ times to sort out the array. - In the best case of an already sorted list, selection sort will need to iterate over all elements to check if it's sorted. Due to this reason, the time complexity remains the same as in the worst case О(*n^*2). - The best case of bubble sort is O(n) because in the first iteration, if the swaps are zero the array is determined to be completely sorted. - Bubble sort consumes additional space for storing variables and needs more swaps, whereas selection sort is an in-place algorithm, so it doesn’t need to allocate any memory. - Selection sort stores one minimum value and its index to compare results. The space required doesn’t increase with the input size, hence space complexity remains O(1). **Time and space complexity** are listed in the below table: ![Table comparing time and space complexity of selection sort versus bubble sort](../../assets/blog/reviewing-sorting-algorithms-selection-sort-2.png) ## Code Implementation: > [!NOTE] > The code has a couple of counters and log messages only for demonstration purposes. ```python #Function to sort array using Selection Sort def selectionSort(arr): # Get the length of the array arrlen = len(arr) print('Length of the Input Array : {}'.format(arrlen)) # Counter to count # of iterations cnt = 0 cnt_comps = 0 # count the comparisions for i in range(0, arrlen-1): # Initial Loop to iterate through Arrays # set the indexposition for example i = 0,1,2,3,4 indexpos = i # set the smallestValue as the Current index position array Value smallValue = arr[indexpos] # increment the iteration counter cnt = cnt+1 print('- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -') print('Iteration : {} starting array is {} '.format(cnt,arr)) # Loop to compare the index position value with rest of the values in array for j in range(i+1, arrlen): print('comparing {} and {} '.format( smallValue , arr[j])) cnt_comps = cnt_comps + 1 # Compare smallestValue with the rest of the values # This is to check for any values which is smaller than the current position set value. if(smallValue > arr[j]): # swap the indexposition to the index of newly found small value indexpos= j # Imp: Swap smallvalue with current array indexposition value. # This means the comparision value is changed. This works for large arrays with a complete unsorted list smallValue= arr[j] # if indexposition is not equal to original index position Swap the values if (indexpos!= i): currentvalue = arr[i] print('Swapping the number {} >> for the number {}'.format(arr[indexpos],arr[i])) arr[i], arr[indexpos] = arr[indexpos], currentvalue print('Current Array after swap : {} '.format(arr)) else: # else just print No swap is performed print('>>> Looks like value is sorted. No Swap is performed >>>') print('Total iterations :{} Total Comparisions : {} Output Array is : {}'.format(cnt,cnt_comps,arr)) return arr # Test Long unsorted list #arrlist = [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] # Test short unsorted list arrlist = [10,4,8,6,1] # Test sorted list arrlist = [1,2,3,4,5] print(selectionSort(arrlist)) ``` **Log Output:** Let’s decipher the log output to align with our understanding of selection sort. ```text # Log for Test short unsorted list arrlist = [10,4,8,6,1] Length of the Input Array : 5 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 1 starting array is [10, 4, 8, 6, 1] comparing 10 and 4 comparing 4 and 8 comparing 4 and 6 comparing 4 and 1 Swapping the number 1 >> for the number 10 Current Array after swap : [1, 4, 8, 6, 10] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 2 starting array is [1, 4, 8, 6, 10] comparing 4 and 8 comparing 4 and 6 comparing 4 and 10 >>> Looks like value is sorted. No Swap is performed >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 3 starting array is [1, 4, 8, 6, 10] comparing 8 and 6 comparing 6 and 10 Swapping the number 6 >> for the number 8 Current Array after swap : [1, 4, 6, 8, 10] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 4 starting array is [1, 4, 6, 8, 10] comparing 8 and 10 >>> Looks like value is sorted. No Swap is performed >>> Total iterations :4 Total Comparisons : 10 Output Array is : [1, 4, 6, 8, 10] [1, 4, 6, 8, 10] ---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---* # Log for Test long unsorted list arrlist = [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] Length of the Input Array : 13 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 1 starting array is [3, 56, 2, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] comparing 3 and 56 comparing 3 and 2 comparing 2 and 58 comparing 2 and 79 comparing 2 and 59 comparing 2 and 34 comparing 2 and 23 comparing 2 and 4 comparing 2 and 78 comparing 2 and 8 comparing 2 and 123 comparing 2 and 45 Swapping the number 2 >> for the number 3 Current Array after swap : [2, 56, 3, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 2 starting array is [2, 56, 3, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] comparing 56 and 3 comparing 3 and 58 comparing 3 and 79 comparing 3 and 59 comparing 3 and 34 comparing 3 and 23 comparing 3 and 4 comparing 3 and 78 comparing 3 and 8 comparing 3 and 123 comparing 3 and 45 Swapping the number 3 >> for the number 56 Current Array after swap : [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 3 starting array is [2, 3, 56, 58, 79, 59, 34, 23, 4, 78, 8, 123, 45] comparing 56 and 58 comparing 56 and 79 comparing 56 and 59 comparing 56 and 34 comparing 34 and 23 comparing 23 and 4 comparing 4 and 78 comparing 4 and 8 comparing 4 and 123 comparing 4 and 45 Swapping the number 4 >> for the number 56 Current Array after swap : [2, 3, 4, 58, 79, 59, 34, 23, 56, 78, 8, 123, 45] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 4 starting array is [2, 3, 4, 58, 79, 59, 34, 23, 56, 78, 8, 123, 45] comparing 58 and 79 comparing 58 and 59 comparing 58 and 34 comparing 34 and 23 comparing 23 and 56 comparing 23 and 78 comparing 23 and 8 comparing 8 and 123 comparing 8 and 45 Swapping the number 8 >> for the number 58 Current Array after swap : [2, 3, 4, 8, 79, 59, 34, 23, 56, 78, 58, 123, 45] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 5 starting array is [2, 3, 4, 8, 79, 59, 34, 23, 56, 78, 58, 123, 45] comparing 79 and 59 comparing 59 and 34 comparing 34 and 23 comparing 23 and 56 comparing 23 and 78 comparing 23 and 58 comparing 23 and 123 comparing 23 and 45 Swapping the number 23 >> for the number 79 Current Array after swap : [2, 3, 4, 8, 23, 59, 34, 79, 56, 78, 58, 123, 45] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 6 starting array is [2, 3, 4, 8, 23, 59, 34, 79, 56, 78, 58, 123, 45] comparing 59 and 34 comparing 34 and 79 comparing 34 and 56 comparing 34 and 78 comparing 34 and 58 comparing 34 and 123 comparing 34 and 45 Swapping the number 34 >> for the number 59 Current Array after swap : [2, 3, 4, 8, 23, 34, 59, 79, 56, 78, 58, 123, 45] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 7 starting array is [2, 3, 4, 8, 23, 34, 59, 79, 56, 78, 58, 123, 45] comparing 59 and 79 comparing 59 and 56 comparing 56 and 78 comparing 56 and 58 comparing 56 and 123 comparing 56 and 45 Swapping the number 45 >> for the number 59 Current Array after swap : [2, 3, 4, 8, 23, 34, 45, 79, 56, 78, 58, 123, 59] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 8 starting array is [2, 3, 4, 8, 23, 34, 45, 79, 56, 78, 58, 123, 59] comparing 79 and 56 comparing 56 and 78 comparing 56 and 58 comparing 56 and 123 comparing 56 and 59 Swapping the number 56 >> for the number 79 Current Array after swap : [2, 3, 4, 8, 23, 34, 45, 56, 79, 78, 58, 123, 59] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 9 starting array is [2, 3, 4, 8, 23, 34, 45, 56, 79, 78, 58, 123, 59] comparing 79 and 78 comparing 78 and 58 comparing 58 and 123 comparing 58 and 59 Swapping the number 58 >> for the number 79 Current Array after swap : [2, 3, 4, 8, 23, 34, 45, 56, 58, 78, 79, 123, 59] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 10 starting array is [2, 3, 4, 8, 23, 34, 45, 56, 58, 78, 79, 123, 59] comparing 78 and 79 comparing 78 and 123 comparing 78 and 59 Swapping the number 59 >> for the number 78 Current Array after swap : [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 79, 123, 78] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 11 starting array is [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 79, 123, 78] comparing 79 and 123 comparing 79 and 78 Swapping the number 78 >> for the number 79 Current Array after swap : [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 123, 79] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 12 starting array is [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 123, 79] comparing 123 and 79 Swapping the number 79 >> for the number 123 Current Array after swap : [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 79, 123] Total iterations :12 Total Comparisons : 78 Output Array is : [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 79, 123] [2, 3, 4, 8, 23, 34, 45, 56, 58, 59, 78, 79, 123] ---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---*---* # Log for Test Sorted list arrlist = [1,2,3,4,5,6] Length of the Input Array : 5 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 1 starting array is [1, 2, 3, 4, 5] comparing 1 and 2 comparing 1 and 3 comparing 1 and 4 comparing 1 and 5 >>> Looks like value is sorted. No Swap is performed >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 2 starting array is [1, 2, 3, 4, 5] comparing 2 and 3 comparing 2 and 4 comparing 2 and 5 >>> Looks like value is sorted. No Swap is performed >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 3 starting array is [1, 2, 3, 4, 5] comparing 3 and 4 comparing 3 and 5 >>> Looks like value is sorted. No Swap is performed >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Iteration : 4 starting array is [1, 2, 3, 4, 5] comparing 4 and 5 >>> Looks like value is sorted. No Swap is performed >>> Total iterations :4 Total Comparisons : 10 Output Array is : [1, 2, 3, 4, 5] [1, 2, 3, 4, 5] ``` If you analyze the short unsorted list [10,4,8,6,1] log, length of the array is 5, total iterations performed are 4 and total Comparisons 10. The total comparison value is based on our earlier formulae n(n-1) /2. Another test performed with a long unsorted list has log output of array length as n=13, total iterations performed (n-1) = 12, total comparisons as 78 based on the above formulae. If you recollect, we had to guess what would be the total comparisons for n=100, it will be 4950. We also tested a completely sorted array [1,2,3,4,5], although no swapping was required, the total number of comparisons remains 10 and iterations as 4. This clearly justifies the selection sort time complexity of О(*n^*2) in best and worst cases. ## Final thoughts Selection sort is yet another simple algorithm that is slow and not fast in comparison to bubble sort. However, it works well with a small list to be sorted and in cases where space is limited. This feature is attributed mainly due to the minimal swaps O(n) requiring very less space. To conclude, selection sort doesn’t seem to be the answer for the fast and efficient sorting algorithm we were looking for at the beginning of the post. However, this isn’t the end of it, in the continuum we have a couple more sorting algorithms to be discussed, to arrive at the most efficient sorting techniques. Stay tuned!! I hope you enjoyed the post. Please subscribe to [ProstDev](https://www.prostdev.com/) for more exciting topics. Hasta luego, amigos! ## Study Resources Below are some resources which discuss Selection sort and its Time-Space complexity at length. - [The Selection Sort - Runestone academy](https://runestone.academy/runestone/books/published/pythonds/SortSearch/TheSelectionSort.html) - [Selection Sort - Wikipedia](https://en.wikipedia.org/wiki/Selection_sort) - [Video] [Selection Sort - CS50 Harvard](https://www.youtube.com/watch?v=f8hXR_Hvybo&feature=emb_title) - [Analysis of Selection Sort - Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/sorting-algorithms/a/analysis-of-selection-sort) - [Video] [Select Sort - CS310 San Diego University](https://www.youtube.com/watch?v=fgYlVyrt1vE) - [Video] [Selection Sort with Gypsy Folk Dance](https://www.youtube.com/watch?v=Ns4TPTC8whw) - [Asymptotic-Notation- Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/asymptotic-notation) - [Big-O cheat sheet](https://www.bigocheatsheet.com/) --- ## Software Quality is not cheap, but would you risk your company's future for the lack of it? Source: https://prostdev.com/post/software-quality-is-not-cheap-but-would-you-risk-your-company-s-future-for-the-lack-of-it | Published: Jul 9, 2020 | Category: Guides You have less than 6 minutes before your customer ditches your application because they perceived a bad customer experience. If you knew this, would you risk your company’s future on the lack of software quality? Mark is the CEO of a start-up company. He had 3 teams of developers and only one QA (Quality Assurance resource) to perform the vast amount of work before pushing the web application to go live (to make it available for customers). He had a healthy number of customers that were eager to try the new CRM (Customer Relationship Management) system. Little did everyone know; things went south very quickly. Customers weren’t happy because the web application wasn’t working as expected. There was a gap between what the application was supposed to do and what it actually did. So, what really went wrong? Short answer: Lack of quality and testing. Today we are going to talk about the importance of testing. We will also talk about some testing types and objectives but, first things first. ## What is Quality? There are 3 points of view to look at the software product. These views pay attention to different attributes of the software. **The first point of view concentrates on the functionality** of the software to conform to requirements. A few examples of this: - It uses a specification guide in the form of: - User Stories – In the agile world. - Specifications document. - The QA (Quality Assurance resource) will focus all their efforts into making sure the software solution provides the minimum specifications stated in the document they got assigned. **2. Best practices when coding.** A few examples of this: - Write reusable code (reutilization of existing code for new functions in the software) - Write unit tests for your unit of behavior (to make sure a piece of code behaves as it’s supposed to). - Make your code readable and self-documenting through good naming conventions and programming standards. - Not relying on code comments because they become outdated often. . (Software developers use comments to explain what their code performs) - Refactor (process of restructuring existing computer code) whenever you can or need. - Avoid technical debt (additional rework caused by choosing a limited solution now instead of using a better approach that would take longer). - Try to make your code fail a test at least once, that way you know that it works as expected. **3. The end user or customer.** A few examples of this: - Quality is a customer determination. - Exceed customer’s expectations. - Provide customer satisfaction. **In my opinion, good software quality could be defined as:** A software product that is free from defects, that is efficient, secure, reliable, easy to maintain, that has the ability to escalate with little effort and time and that exceeds customer’s expectations by providing a solution that is, not only useful, but adds more value to the customer. ## What is the importance of testing? I’ll use an analogy to try and explain the importance of it. It’s like throwing yourself from an airplane and you didn’t check if your backpack had a parachute. In this case you are the software product and the empty backpack is your product quality. From a business point of view, **software quality is your safety net and it saves you time and money.** **It brings you economic benefits** in the medium-long run. Every time a customer is not able to interact successfully with your software product, it becomes a customer that will never come back. When you lose that customer, you lose money and your software product might get a bad review. **Did you know that it takes around 40 positive reviews to undo only one negative review?** And most happy customers don’t leave a positive review. From a security point of view, software quality protects vulnerabilities of your product. That means that your product might be targeted to exploit those vulnerabilities in order to steal customers personal information, money, benefits-free products or anything else you might keep in order to process their software solution. **From a customer satisfaction point of view, software quality provides precisely that: Customer satisfaction.** The software product that you create will bring a polished solution and the best user experience possible (at that precise time). It will be efficient, secure, reliable and will work like magic. So easy to interact with that no one will ever know the complexity in the code. And you might be hesitant into investing in testing because it seems to be so invisible. No one cares when your software solution is working fine, right? But what happens when it doesn’t perform well? Software quality is not cheap when perceived as invisible but if you start adding $ into each defect found you soon will learn that it is not only necessary, it is intrinsic to your business success. **A software with good software quality, reputation and positive user experience will bring you more customers.** ## Software Quality Types and Objectives Software quality is divided into 2 main categories. Functional and non-functional. **Functional testing** is the **“what”** in your software. What will your application do? What should your TV do when you push the power button? - Turn on the screen - This is a functional test. How long should your TV take when you push the power button to turn on the screen? - 2 seconds - This is a non-functional test. **Non-functional testing** is the **“how”** in your software. There are more than 100 types of testing but some of the most common types are listed below. ![Diagram listing common functional and non-functional software testing types](../../assets/blog/software-quality-is-not-cheap-but-would-you-risk-your-company-s-future-for-the-lack-of-it-1.png) In software testing you have several opportunities to catch the defects in your product before launching it to your customers. **Unit Testing** is the first level or opportunity for software testing that is performed by software developers. It is the first piece of the puzzle. It is performed by using the White Box Testing method, which means that the person who is testing the functionality also knows the internal structure, design and implementation. The main objective is to isolate your unit and test its behavioral performance. Make sure it performs as designed. The second level or opportunity for testing is **Integration Testing**. Integration Testing combines individual units, like the one we talked about above, and tests them as a group. The idea behind this type of testing is to expose defects in the interaction between the units. An example of integration testing could be something similar to a lightbulb connected to a battery and a switch. Each one of the items listed is a unit, and I expect that when I interact with the switch, the lightbulb will turn on and off. The third level or opportunity to catch a defect comes in the form of **System Testing**. This means that we will combine all the product components and modules and perform an end-to-end testing on it. In this type of testing we use a persona and try to accomplish a task in the software product. There’s a beginning and an end to the workflow that has to be followed to get the task done. The objective is to replicate and validate real user scenarios. The fourth level is named **Acceptance Testing**. The software product is tested against the business requirements. There has to be a perfect compliance with the specifications on the solution we are presenting to the end user. Product quality is a joint effort. To have a mesmerizing quality you need everyone in your company to put on their Quality Assurance hat and evaluate their own designs, implementations and reasons behind each solution. A software tester (or QA) is your software solution protector. It is the last gate to open before launching your product to your customer. The difference between making it or breaking it. ## Recap Quality can be explained from 3 different perspectives: - The functionality of the software to conform to requirements. - Using best practices when coding the software product. - Exceed customer’s expectations. Testing your software product will result in more customer satisfaction, which can give you a great reputation as a business and will bring you more customers, resulting in more money. The types of Software Testing can be divided into Functional and Non-Functional. - Functional testing is the “what” in your software. - Non-Functional testing is the “how”. --- ## Scatter-Gather Integration Pattern (Mule 4) Part 2 Source: https://prostdev.com/post/scatter-gather-integration-pattern-mule-4-part-2 | Published: Jul 7, 2020 | Category: Guides *GitHub repository with the Mule Project can be found at the end of the post.* In the previous post, [Intro to Scatter-Gather Integration Pattern](https://www.prostdev.com/post/intro-to-scatter-gather-integration-pattern), we have discussed the basics of the Scatter-Gather message routing Enterprise Integration Pattern and also demonstrated a simple case of the pattern using MuleSoft Connector. Moving forward, from a happy path scenario, we will see the demonstration on how Mule 4 Scatter-Gather connector behaves in case of processing errors with or without Mule Error handlers. Additionally, we will explore the example of processing multiple services using the Mule 4 Scatter-Gather connector. **Gathering** **thoughts from the previous blog:** - Scatter-Gather processes messages in-parallel and the result is an aggregated response of the routes. - To live up to its merits, Scatter-Gather needs more than one message processing route. - When the process responses are independent of each other, it is efficient to use Scatter-Gather. - Use Scatter-Gather for multicasting scenarios. - Scatter-Gather is concurrent processing and there is no sequence of execution. - Scatter-Gather will not stop processing messages to other successful routes during failures. ## How does the input message get processed in Scatter-Gather? The input payload/message is distributed or copied across all the routes of the Scatter-Gather. Our [previous example](https://www.prostdev.com/post/intro-to-scatter-gather-integration-pattern) of the MuleSoft Scatter-Gather is modified, by adding a new Set Payload component to check the status of the payload during the Scatter-Gather processing. As shown below from the log results, the payload is copied or distributed across all the routes of the Scatter-Gather. In the listed example, the payload value “result0: 42” is not honored by the three flows, as they have their payload values. In short, the Set Payload value before the Scatter-Gather component is overwritten by the flow's payload value. ![Fig 2.0: Input payload is copied to all the route processors.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-1.png) However, the Set Payload information can be very well used to add business logic to the multiple routes accordingly. For demonstration purposes, Flow One payload is modified to retrieve the “result0” information. ![Fig 2.1: Initial Set Payload result is displayed to include in the flow-one response.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-2.png) ## Error Handling in Scatter-Gather In our [previous post](https://www.prostdev.com/post/intro-to-scatter-gather-integration-pattern), we stated that the Scatter-Gather avoids a complete meltdown, unlike sequential processing. But, without an effective error handling in place, you can question the merits of the latter statement. One can use a MuleSoft Try scope in the routers of a Scatter-Gather component for error handling purposes. If a particular route encounters errors, the Try scope will take care of the error using the [On-Error-continue](https://docs.mulesoft.com/mule-runtime/4.3/on-error-scope-concept) error handler, which would make the route successful by handling the error logic. The Scatter-Gather route failures are handled well to process the error unless it is required to end the process abruptly. Let’s try to prove the same using some twists in our current Scatter-Gather example. ### Happy Path ☺ Everything is green, life’s pretty smooth and happiness sways around. The Scatter-Gather route flows one, two, and three payloads have been modified to implement a simple addition, subtraction, and division of numbers (ex: 10+2, 16-2, 30/2) to be aggregated as an output result of each flow. In the screenshot below, results are displayed as expected, all looks good! Yay!! ![Fig 2.2: Result of the happy path Scatter-Gather flow without any errors.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-3.png) ### Not so Happy Path Well, life isn’t all rainbows and unicorns as the above result made you believe for a while. If the flow-three fails due to a *division by zero* error, the Scatter-Gather will fail as there isn’t any mechanism to avert the catastrophe. The process will halt, and the error will be displayed as given below. ![Fig 2.3: Mule debugger showing the Flow-Three failure due to error. No error handling is implemented.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-4.png) ![Fig 2.4: Postman test of the Scatter-Gather Flow-Three failure due to error.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-5.png) Although not so happy path the above case is, one would still want to see some result as an output instead of a long semi-cryptic error message. Here comes the error handling Try scope for the Scatter-Gather routes! All the three flows are wrapped with [Try Scope](https://docs.mulesoft.com/mule-runtime/4.3/try-scope-concept)to capture the errors in any of the routes if they failed while processing the message. The output is guaranteed to be some aggregated result instead of a long error description. ![Fig 2.5: Try-Scope error handling implemented in all three processes of the Scatter-Gather.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-6.png) From the above fig. 2.5, an error handling scope (Mule 4) of On-Error Continue is implemented which will make sure a result is derived from the Scatter-Gather. More information about Mule 4 error handling can be found at [MuleSoft documentation](https://docs.mulesoft.com/mule-runtime/4.3/intro-error-handlers). The transform message component, inside the On-Error Continue scope, is used to provide more information on the type of error as output. By adding this feature, we can now identify the failure reason for flow three. After all, our goal here is to achieve a more composed state of the output. There will be valid business scenarios in real-time, where the output of the route in error is not required to be displayed. In such cases, the output will be an aggregate response of successful flows, as shown in Fig 2.5.1. In the On-Error continue component, the transform message payload is given as [ ] (empty array) to bypass error information. ![Fig 2.5.1: Postman test of the Scatter-Gather flow with error handling implemented. Only success responses from the processes are displayed.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-7.png) Fig.2.6 is showing results of the flow stepping into the error handling scope through Anypoint studio mule debugger. The error is bypassed, and the Scatter-Gather has aggregated the messages from successful processes. ![Fig 2.6: Display of the Flow-Three error handling and log showing error information.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-8.png) When the flow-three result is required irrespective of failure or success, a descriptive user-friendly error message can be added. This would help the user to identify the reason for Scatter-Gather route failures (if any). If a business logic needs to be invoked on flow-three failure, it can be incorporated in the On-Error continue error handler scope. The business logic can be anything specific to invoking another process upon failure or notifying the stakeholders through email. ![Fig 2.7: Postman test of the Scatter-Gather flow with error handling implemented. All the route responses are displayed, including error.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-9.png) Error message to be displayed as the output of the failed process flow-three. ![Fig 2.8: Flow three error handling response returned.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-10.png) Yay!! We searched for happiness, in the not so happy path of Scatter-Gather and we were able to strike the right balance. A thought to ponder and reflect upon, which brings us to our reflection questions, and I would encourage you to try out the code snippets in mule 4 to try out the below cases and get the answers. ### Reflection Questions: - What happens to the initial payload, when one of the processes in the Scatter-Gather fails? - What happens if the variable in one process is modified and not in the other? - What would be the result of Scatter-Gather if “On-Error propagates” is used instead of “On-Error continue” when one of the routes fails? ## Multiple Services Call Using Mule 4 Scatter-Gather Okay! Moving ahead, remember we are yet to see a case of multiple services being invoked through the Scatter-Gather Mule 4 component. The example demonstrated below is retrieved from the free [Mule 4 developer](https://training.mulesoft.com/course/development-fundamentals-mule4) course. It’s a very simple example to showcase the power of the Scatter-Gather component. The scenario is to search the flights of United Airlines and Delta airlines for a given airport code. The United Airlines flight search is retrieved from REST API calls. Whereas Delta airlines information is retrieved through SOAP WSDL. The Set-Variable component is used to store the airport code value to retrieve the available flight information. Variable ‘code’ value is set as #[message.attributes.queryParams.code]. ![Fig 2.9: Scatter-Gather example showing multiple services (HTTP REST, SOAP WS) invoked.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-11.png) The following result is derived from the Scatter-Gather processing of the two different service calls. ![Fig 2.10: Postman test displaying the results of United and Delta flights.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-12.png) You have made it to this part of the post, now the last stop in the journey is to add the missing part in the above demonstration, error handling. If you check *Fig 2.9,* Try scope hasn’t been implemented yet. Applying the learnings from our previous section, let’s add the Try scope to each of the Scatter-Gather routes. We can add a flavor to the scenario by displaying only available flights. Meaning, we are looking to retrieve any flight information irrespective of route failure. You can most definitely hash out a test case to check the output if both the flights fail. To demonstrate the error handling, the web services URL is tweaked as [http://mu.learn.mulesoft.com/delta12?wsdl](http://mu.learn.mulesoft.com/delta12?wsdl) ![Fig 2.11: Implemented Try-scope to all the processes.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-13.png) Upon checking the flow through mule-debugger, we can see the getDeltaFlights process failed due to error type `WSC: CONNECTIVITY`. ![Fig 2.12: getDeltaFlights process failure due to connectivity.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-14.png) However, the failure was handled successfully by the MuleSoft Try scope and the On-Error continue to Transform message has an empty array returned as a response. Scatter-Gather has now aggregated the response from both the flows as output shown below: ![Fig 2.13: Postman test of the Scatter-Gather flow with the final response from Scatter-Gather with only United flights.](../../assets/blog/scatter-gather-integration-pattern-mule-4-part-2-15.png) ## Penultimate Thoughts Finally, we have reached a point where we understand the ball bearings of Scatter-Gather through the Mule 4 connector example. To reiterate: - Scatter-Gather is best suited for Multicast scenarios. - The Scatter-Gather messages are processed in-parallel with aggregate responses. - More than one process is required for Scatter-Gather to function according to its discussed merits. - The MuleSoft Scatter-Gather has many features to control the execution flow in case of route failures. - In case of errors, a new business logic such as notifications can be invoked as required. Now that you’re familiar with Scatter-Gather, it’s the time to check out the real code in action. ### Configuration File Code: ```xml { airlineName: flight.airlineName, availableSeats: flight.emptySeats, departureDate: flight.departureDate, destination: flight.destination, flightCode: flight.code, origination: flight.origin, planeType: flight.planeType, price: flight.price } as Object { class : "com.mulesoft.training.Flight" }]]> { airlineName: return.airlineName, availableSeats: return.emptySeats, departureDate: return.departureDate, destination: return.destination, flightCode: return.code, origination: return.origin, planeType: return.planeType, price: return.price } as Object { class : "com.mulesoft.training.Flight" }]]> ``` ## Next Steps It is highly recommended that you check the simple example code snippets and test out the concepts discussed in these pair of blogs. Download [MuleSoft Anypoint Studio](https://www.mulesoft.com/lp/dl/studio) to test out the Scatter-Gather flows by goofing around in your unique ways. Test the above reflection questions to grasp an in-depth understanding of the Mule 4 Scatter-Gather component. Try out some [DataWeave](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-cookbook) functions to make the error response interesting for the end-users. For a better understanding of the Mule 4 error handling refer to the below amazing blogs: - [Mule 4 error handling demystified](https://blogs.mulesoft.com/dev/training-dev/mule4-error-handling/) - [Mule 4 error handling deep dive](https://blogs.mulesoft.com/dev/training-dev/mule4-error-handling-deep-dive/) - [Use case-specific error handling in Mule 4](https://blogs.mulesoft.com/dev/training-dev/mule4-error-handling-use-cases/) I hope you enjoyed the post. Please subscribe to ProstDev for more exciting topics. Hasta luego, amigos! ## GitHub repository [ProstDev GitHub - Scatter-Gather Error Handling Example](https://github.com/ProstDev/apdev-scatter-gather-error-handling) --- ## Reviewing Sorting Algorithms: Bubble Sort Source: https://prostdev.com/post/reviewing-sorting-algorithms-bubble-sort | Published: Jun 30, 2020 | Category: Guides Let’s sort it out! In this series of posts, we will be discussing basic computer science algorithm sorting techniques. Yes, you are right we are here again trying to shake up our memory boxes for these algorithms long forgotten. Recently, I attended an interview for a Silicon Valley company, and you guessed it right, KO (Knock Out) coding rounds were the appetizers and I figured pretty much on my day 1 of coding interview preparation, that I'm not serving KO’s back on coding challenges. You may most definitely choose to agree with me that we always remember the most popular and unpopular of everything. I was sure the most unpopular sorting technique was “Bubble Sort” and the popular one was “Quick Sort”. Basically, my statement is incorrect on many counts about the popularity of the sorting techniques, the most unpopular kid in your high school is pretty much popular for being not so. ## Why is sorting a big deal? We know sorting things almost in every instance reduces the complexity of the problem at hand. Sorting is a well-solved problem in computer science, It puts any list coming out of a computer into a certain required order efficiently to reduce the searching complexity of the problem. In a series of posts, we will be discussing some of the sorting algorithms listed in the below order: - **Bubble Sort** - [Selection Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-selection-sort) - [Insertion Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-insertion-sort) - [Merge Sort](https://www.prostdev.com/post/reviewing-sorting-algorithms-merge-sort) - Quick Sort - Heap Sort Let’s start with the most popularly known Sorting technique. You are getting better at the game of *“who’s more popular!”* ## Bubble Sort Bubble sort is a very simple comparison-based algorithm used to sort the elements which are out of order. It compares the two elements at a time to check for the correct order, if the elements are out of order they are swapped. The swapping of elements continues, until an entire sorted order is achieved. Although a naïve algorithm, knowing bubble sort helps in understanding the technicalities of sorting in comparison with other techniques. Let’s play with the bubbles. **Problem:** Sort the numbers 10, 4, 8, 6, 1. As mentioned earlier, bubble sort will compare the two pairs at a time. Let’s start with the *first iteration* of the numbers: ![First bubble sort pass over [10, 4, 8, 6, 1], swapping pairs until 10 reaches the end](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-1.png) Rinse and repeat! Until the required sorting order is achieved. This begs the question, what if the problem has a very long list of elements to be sorted like for example 100 or 1000? Well, the number of iterations to check every pair of elements repetitively might exhaust you (n-1) times. Which means, it will take (n-1) iterations in order to sort a list of n elements. *Mind you, the above example was just the very first iteration and for* *n=5**, it takes a total of 4 iterations to complete the sorting.* *2nd Iteration:* ![Second bubble sort pass comparing pairs, leaving 8 and 10 sorted at the end](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-2.png) *3rd iteration:* ![Third bubble sort pass, swapping 6 and 1 so 6, 8, 10 are now in order](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-3.png) *4th iteration:* ![Fourth bubble sort pass swaps 4 and 1, producing the fully sorted [1, 4, 6, 8, 10]](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-4.png) **Observations based on Iterations:** - *‘n’* number of elements will undergo (n-1) iterations to complete the sort. - If you have noticed, the max number in the list 10 is placed in its ordered position after the first iteration. - Likewise, the next max number 8 will be placed in its sorted position after the 2nd iteration, 6 from the 3rd iteration and number 4 for the 4th iteration. - Based on this pattern, we can see that in each iteration the largest element tends to move up in the correct order similarly to bubbles rising on the surface or the smaller elements bubbles up to top the list. Hence, the name suffices the feature as “Bubble Sort”. - We also observed that it's totally not required to check the last pair after the 2nd iteration as they are already sorted in their place. This pattern is observed in the 3rd iteration too, where the last 3 elements are already sorted. Which means that for every ‘m’ iterations, checking last ‘m’ elements will be a repetitive activity and avoiding the check can result in an optimal bubble sorting solution. - Hold on to these observations for now, as we still have to check why this kid on the sorting block is notoriously ‘unpopular’. ## Coding Demonstration of Bubble Sort using Python: ```python # Function to demonstrate bubble sort technique def bubbleSort(alist): length = len(alist) # Counter is used only for displaying iteration count cnt = 0 for i in range(length): # Check the element pairs for correct Order. 'n' elements will yield n-1 iterations to achieve sorted list cnt = cnt+1 isSwapped = False print('- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -') print(" Iteration : {} starting array is {} ".format(cnt,alist)) for j in range(length-i-1): # now, compare the elements to probe the order. Example : First element pair is compared print ("Now Comparing >> {} with {} : ".format(alist[j],alist[j+1])) if alist[j] > alist[j+1]: # swap the elements. For clarity purposes the swapping is shown in seperate steps # Use a Temp variabe to store the first element number in a element pair ex (1,2), temp will store value 1 # Temp variable is not required for swapping, is used only for understanding the swap logic temp = alist[j] # swap first element in pair with second element alist[j] = alist[j+1] # swap second element in pair with first element alist[j+1]= temp # As the element pair is sorted set it to true isSwapped = True if isSwapped : print(" Array is modified as {} ".format(alist)) else: print(" Array is not modified {} ".format(alist)) # Increase the counter to process into next element pair print( 'Iteration {} ending array {} : '.format(cnt,alist)) print( ' is array Swapped at least Once in the Iteration >> {} '.format(isSwapped)) if isSwapped == False: break return alist alist = [10,4,8,6,1] print(bubbleSort(alist)) ``` **Log Output:** ![Python console log of bubble sort iterations 1 and 2 with each comparison and swap](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-5.png) ![Python console log of bubble sort iterations 3, 4 and start of 5](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-6.png) ![Final log line: iteration 5 makes no swaps, confirming [1, 4, 6, 8, 10] is sorted](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-7.png) From the above code snippet, we have demonstrated our earlier example on how bubble sort bubbles up the largest number to the end of the list in each iteration. What if the Incoming list is already sorted? In this case, if there are no swaps performed in the first iteration, it's positive that the list is pretty much already sorted out. ![Python bubble sort code beside its log, where an already-sorted list ends after one pass](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-8.png) ## Time and Space complexity AKA why is this kid so Unpopular? To achieve the sorted list, we had to iterate ‘n’ times through an array of ‘n’ elements, in order to check all the ‘n’ elements. Meaning, the bubble sort performs *‘n*’ iteration *‘n’* times making it n*n = n^2. *But what is Time and space complexity? Why does it matter in this case?* - In simple terms, time and space complexity analysis is required to analyze the resources required by an algorithm to complete the function of the input (ex: in our case sorting). - Time complexity of an algorithm is the amount of time taken by the algorithm to complete processing as a function for given input length *n*. It is commonly expressed using asymptotic notations as given below table. - Space complexity is the amount of runtime memory used by the algorithm to achieve the result. Total space includes auxiliary space and the space used by the input. Auxiliary space is the temporary space allocated by the given algorithm so achieve the result for a given input size. Overall, memory usage includes the memory used to save the compiled version of instructions, space used by variables and constants and in cases where algorithms function internally another algorithm function. ![Table of bubble sort worst, best, average time complexity and space complexity](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-9.png) ![Big-O complexity chart plotting operations vs elements from excellent to horrible](../../assets/blog/reviewing-sorting-algorithms-bubble-sort-10.png) *Fig: Reference -* [bigocheatsheet.com](http://bigocheatsheet.com) Best case time complexity: If the array is an already sorted list of elements, at most ‘n’ comparisons are made and no swaps are required. For example, input: [2,3,6,7,9,12] Worst case time complexity: Array is fully unsorted or in descending order. Example: [10,4,8,6,1] Average case time complexity: A mix of sorted and unsorted list of elements in the array. Example: [19,15, 21,7,9,10] Space complexity: Amount of memory required to run the algorithm grows no more than O(n). In our example, total space required would be for variables assigned and array size *n*. To understand why bubble sort is the least preferred sorting technique, consider the input list is grown in size to a more significant number (n=1000), the amount of comparisons and swaps will increase significantly making it more inefficient. I believe now it’s clear why bubble sort is so unpopular, as there are other sorting techniques which have a better time complexity and can run much faster for an increasing input size. To that point, bubble sort is mainly used for educational purposes. Stay tuned!! I hope you enjoyed the post. Please subscribe to [ProstDev](https://www.prostdev.com/) for more exciting topics. Hasta luego, amigos! ## Study Resources As bubble sort is a well known algorithm, lots of resources can be found with varying examples on the internet. This blog post was written to provide a basic understanding of the bubble sort, for a short read to refresh your memory. Below are some resources which discuss bubble sort and Time-Space complexity. - [The Bubble Sort - Runestone Academy](https://runestone.academy/runestone/books/published/pythonds/SortSearch/TheBubbleSort.html) - [Bubble Sort Wiki](https://en.wikipedia.org/wiki/Bubble_sort) - [Bubble Sort - CS50 Harvard](https://www.youtube.com/watch?v=8Kp-8OGwphY) - [Asymptotic-Notation- Khan Academy](https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/asymptotic-notation) - [Big-O cheat sheet](https://www.bigocheatsheet.com/) --- ## 5 reasons why you need an IDE (and how it can save you so much time) Source: https://prostdev.com/post/5-reasons-why-you-need-an-ide | Published: Jun 25, 2020 | Category: Guides If you’re an experienced developer, you probably already know why using an IDE is a great idea, but if you’re completely new to the Software Development world, you may not even know what an IDE is. Well, not to worry! For this post, I will explain what is an IDE and five reasons why you need to start using one. I will also give you examples of my favorite IDEs for different programming languages and technologies. ## What is an IDE? IDE is an acronym for “Integrated Development Environment”, and it is nothing more than an application that can be used to develop any kind of software programs. You can usually download an IDE from the IDE’s creators’ website and install it like any other application; for example: for Windows, using a .exe file, or for MacOS, using a .dmg file. IDEs are designed to aid the programmers in their development process, and help them to be the most productive as possible by providing a Graphical User Interface (GUI) that can be easily used to perform basic day-to-day functions. A metaphor that comes to my mind is a calculator - if you’re a mathematician, you probably have bigger problems to solve than a basic average or division calculation; you can be more productive and solve more complex problems, if you don’t have to worry about the smallest operations that a tool can solve for you. This is basically what an IDE does for a Developer. If your idea of an IDE is not completely clear yet, with the next five reasons you’ll get a better understanding on what an IDE can do for you as a Software Developer. ### 1. Syntax highlighting If you speak english, you may say “cheers” after a toast at a wedding; but if you speak spanish, you’ll say “salud”; or you’ll say “prost” in german. Just like a regular language, every programming language has their own syntax, or their own way of “writing” or “communicating” the same thing. Here are some examples of how you would create two variables (x and y) and add them to create a new variable (z), in different programming languages: ![Same x + y addition written in plain Java, JavaScript, Python and DataWeave](../../assets/blog/5-reasons-why-you-need-an-ide-1.png) As you can see, they have some things in common, like the equal (=) or plus (+) symbols, but they also have some differences, like the end of a line (;) or the variable declarations (*int, var*). When you’re using an IDE that knows the programming language’s syntax that you’re using, the IDE is able to recognize what a symbol or a keyword means, and then it shows your code with certain colors or formats (like **bold** or *italic*) in order to make it a bit more readable. Here are the same previous examples, but using a different IDE for each of them: ![The same snippets with syntax highlighting in IntelliJ, Sublime Text, PyCharm and Anypoint Studio](../../assets/blog/5-reasons-why-you-need-an-ide-2.png) Note that the one for JavaScript (top-right corner) does not specify an IDE, but an Editor. Some text editors also offer syntax highlighting, just like an IDE would. However, an editor won’t give you the rest of the benefits (listed below) and if they do, it’s not going to be as efficient as an IDE. ## 2. Text autocompletion You know that thing Google does when you start writing some text for a new search and it wants to *complete* or give suggestions as to what you might be searching for? ![Google search box autocompleting the query technology blog with suggestions](../../assets/blog/5-reasons-why-you-need-an-ide-3.png) The same thing happens with an IDE. Since it already knows the syntax of the language you're programming in, it can give you suggestions of what you want to write next. This helps you to be faster because you don’t have to write everything yourself, you can just start writing something and the autocompletion will give you a list of possible choices! ![Animation of IntelliJ autocompleting a Java String keyword as the developer types](../../assets/blog/5-reasons-why-you-need-an-ide-4.gif) For example, say you forget a specific keyword that you need to use, but you do remember part of the instruction that is needed; instead of going to Google or to check on a programming book, you can just start writing what you remember, and the IDE will try to guess the command, then you just need to choose between your options and press Enter to select it. ![Animation of an IDE suggesting Java method calls like setDefaultCloseOperation as you type](../../assets/blog/5-reasons-why-you-need-an-ide-5.gif) Pretty cool, right?! ## 3. Refactoring options “Woah, wait! What does *refactoring* mean?” It means that you can restructure your existing code or project’s resources without affecting any behavior. For example: renaming a file, changing files to different folders, or even renaming a variable. “Ok, why would I want to do that if it already works as it is?” Sometimes the code can get too complex to understand, even by the developer who wrote it, and there are some changes that can be done to improve the code’s readability. For example, you create a function that will add two numbers and return the result, but you don’t want to think in descriptive names for the function or for the two variables, so you just named them “x”, “a” and “b”. This code will look like this: ```python def x(a, b): return a + b x(1, 2) ``` Not too readable, right? I have to go into the definition of the function “x” to know that the function is adding “a” plus “b”. Maybe if this function was named something like “add_numbers”, I wouldn’t have to waste my time going into the code to figure out that it’s adding two numbers. Instead of manually renaming the “x” in the definition of the function, and then once more in the function call (the last line), I can just select the “x”, right-click on it, and select “Rename”. ![PyCharm right-click context menu with Refactor and Rename highlighted on the function x](../../assets/blog/5-reasons-why-you-need-an-ide-6.png) After writing the new name for my function, I will see that all the references were correctly updated. This can also be done for the “a” and “b” variables, that can have a more descriptive name, like “first_number” and “second_number”. ![The function refactored to add_numbers(first_number, second_number) with all references updated](../../assets/blog/5-reasons-why-you-need-an-ide-7.png) Way better! Now I can clearly know what the “add_numbers” function will be doing. Without even going to the function’s definition, I can know in advance that the result of add_numbers(1, 2) will be 3. Now, this was a small example, but imagine that you have more than 20 files that are referencing the same function that you want to rename. Instead of manually going to each of the files, you can just use the IDE’s refactoring options to change the name. The same applies when moving files from one folder to another, or when renaming files. ## 4. Importing libraries When you’re working on a Software Development project, there will be some tough requirements that you need to figure out how to code. Luckily for us, you don’t have to create code from scratch, another developer on the internet already has and you can use their code as a new library. Again with my metaphors: imagine that you want to paint a house, but you don’t have any paint. Will you go to a forest to get some berries and insects to make your own paint? Or will you go to Home Depot and buy whatever paint you need? (I hope you chose the second one!). The same applies with code. There are some functions that someone created that are available for you to re-use in your project; there’s absolutely no need for you to reinvent the wheel in your code if you can just import a library and take advantage of it! There are times when you remember a library’s function, but you don’t remember the name of the library. So, same as before, instead of taking time off your day to google the library, you can just write the function that you wanted to use and let the IDE handle the import for you! ![Animation of an IDE auto-importing the library for Swing classes like JFrame and JButton](../../assets/blog/5-reasons-why-you-need-an-ide-8.gif) Are you convinced now? Yes? No? Maybe? Ok, let me give you the last reason why you need and want an IDE. ## 5. Build, compile, or run Once you finish coding your project or feature, you’ll want to do a quick test to verify that it’s working as you expected. How are you going to run your code in order to check that? Well, if you generated a bunch of Java code, you have to first *compile* your .java files into .class files (these are binary files that are interpreted by the computer), only after you do that, you can run your code. To do this, you have to do something in the Console (or Terminal) that looks like this (you don’t have to understand what I’m doing here, just notice there are some statements to run): ![Terminal manually compiling with javac then running java Main, printing Hello World!](../../assets/blog/5-reasons-why-you-need-an-ide-9.png) What if I told you that you can just click a button if you use an IDE? ![IntelliJ green-arrow menu with Run 'Main' option for the Hello World class](../../assets/blog/5-reasons-why-you-need-an-ide-10.png) Yes! Instead of doing all those commands manually, you can just use the “Run” button. You get exactly the same result! ![IntelliJ project tree and editor with the Run console printing Hello World! at the bottom](../../assets/blog/5-reasons-why-you-need-an-ide-11.png) If you need to build, compile, or run your code; you can do all these things just by clicking on a button from the IDE. No need to compile into a different format! (this can be time consuming). ### Recap To recap what we learned today, an IDE saves you so much time (and headaches) and we went through 5 ways that do this. - **Syntax highlighting** that helps you be able to read the code and not get lost in between hundreds of lines of code. The IDE uses different colors or formatting for the code, to make it more readable. - **Text autocompletion**, just like the Google search bar does, but for your code. If you forget a keyword, the IDE will give you the best suggestion of what you were looking for. - **Refactoring options**, like renaming a file, a function, or a variable; or moving files from one folder to another. This way you don’t have to manually update every single reference in the whole project. - **Importing libraries** in case you remember the code, but not the name of the library. Most likely the IDE already knows what function you’re referring to, it just has to import that library to your project and you’re all set. - **Build, compile, or run** your project just by clicking one button. There’s no need to memorize all those commands and run them one by one from the Console or Terminal. There are not only 5 benefits that an IDE can give you; there are a lot more that I love! Debugging, testing, generating pre-defined code, managing your dependencies, managing environments, using version control systems are all options that are more advanced. If you’re a new developer, or just new to the technology world, these 5 ways to save time are the most important ones that you need to know. What is your favorite IDE? Do you prefer editors over IDEs? Log-in now to leave me a comment about it! I’d love to hear your opinion :) *Prost!* -Alex ## More Links - [Download IntelliJ (Community Edition)](https://www.jetbrains.com/idea/download/) - [Download PyCharm (Community Edition)](https://www.jetbrains.com/pycharm/download/) - [Download Sublime Text](https://www.sublimetext.com/) - [Download Anypoint Studio](https://www.mulesoft.com/lp/dl/studio) --- ## Intro to Scatter-Gather Integration Pattern Source: https://prostdev.com/post/intro-to-scatter-gather-integration-pattern | Published: Jun 23, 2020 | Category: Guides *GitHub repository with the Mule Project can be found at the end of the post.* In this blog post we will be *gathering* the *scattered* thoughts by routing them into a singular direction. You would have probably guessed it by now, we will be discussing an Enterprise Integration Pattern for Message Routing called the Scatter-Gather. We will first try to understand the concept of Scatter-Gather and then demonstrate its ability with the help of MuleSoft ESB connector for the same. The goal of this post is to follow a very basic and simplistic approach, before diving further into the concept. If you are someone who doesn’t like a lot of text, the depicted pictures will whisper the secret, in parallel. ## What is Scatter-Gather? As described in [Enterprise Integration Patterns,](https://www.enterpriseintegrationpatterns.com/patterns/messaging/BroadcastAggregate.html) Scatter-Gather is a Message routing pattern which broadcasts messages to multiple recipients and aggregates the response back in a single message. In simple words, the messages are executed in parallel (scatter) and response from each execution is bundled (gathered) as one single message. Below picture depicts the same regarding Scatter-Gather: ![Fig 1.1: Request is routed to three different processes in scatter-gather, Output response will be the aggregated result of all 3 processes.](../../assets/blog/intro-to-scatter-gather-integration-pattern-1.png) > [!NOTE] > Response of individual processes are independent of each other. ## When to use Scatter-Gather? Depends on what you are trying to achieve, if it’s a [Fork-Joined model](https://en.wikipedia.org/wiki/Fork%E2%80%93join_model) (imagine a fork) or Multicast scenario, where the messages are required to be communicated simultaneously, Scatter-Gather is a better choice, in case of “[Fire-Forget”](https://www.enterpriseintegrationpatterns.com/patterns/conversation/FireAndForget.html) model use asynchronous processing blocks, as there is no standard requirement to gather all the response in given time. If performance is a parameter (most certainly), Scatter- Gather pattern helps to immensely cut down the processing duration and improves the performance. ## How does Scatter-Gather work? As already explained in the above section, Scatter-Gather will execute the messages in Parallel a.k.a. concurrent processing, meaning sending the messages to desired routes at the same time through a parallel thread pool. While a particular message is being processed, the remaining messages should wait for the completion of the message. In other words, the Scatter-Gather processing will be completed only when all the messages are fully processed. ![Fig 1.2: Scatter-Gather process 1 is complete (Green), Other two processes are work in-progress (Yellow). The response is not yet captured (Red).](../../assets/blog/intro-to-scatter-gather-integration-pattern-2.png) ![Fig 1.3: Scatter-Gather process 1 and 2 are complete (Green), Whereas process 3 is work in-progress (Yellow). The response is not yet captured (Red).](../../assets/blog/intro-to-scatter-gather-integration-pattern-3.png) ![Fig 1.4: Scatter-Gather process 1, 2, and 3 are complete (All Green. Yay!). The response is captured as a bundle of all three responses (Green).](../../assets/blog/intro-to-scatter-gather-integration-pattern-4.png) ### Quiz 1: What is the Total time taken for the completion of the Scatter-Gather process below? ![Scatter-gather diagram with three processes taking 20, 14, and 10 seconds, asking the total time.](../../assets/blog/intro-to-scatter-gather-integration-pattern-5.png) ## Why use the Scatter-Gather Integration Pattern? It may turn out to be a simple decision on why to use the Scatter-Gather pattern when compared to sequential processing, although the responses retrieved can be the same in both cases. Sequential processing should be considered when there is an interdependency of the processes up on their responses, meaning response from process 1 is required for process 2 to complete. In cases where the process responses are independent of each other, parallel processing can be efficiently executed. If using sequential processing, in case of one message/process failure the complete processing will come to halt. This may be a valid business scenario to end the process, whereas in case of scatter-gather the responses will include or exclude the error cases (scenario based) therefore avoiding a complete meltdown. ### Quiz 2: Scatter-Gather halts the processing of the messages if any one of the routing processes fail. *Answer: True or False.* We started with scattered thoughts about the pattern and now we have reached a state of composed thoughts, which were pretty *scattered* to begin with. Next, we will attempt to demonstrate our understanding through a MuleSoft (Mule 4) usage of Scatter-Gather Component. I won’t go deep into the specifics of Mule 4 Scatter-Gather Router as it has been explained very well in [MuleSoft documentation](https://docs.mulesoft.com/mule-runtime/4.3/scatter-gather-concept). ## Mule 4 Scatter-Gather Connector Example In the below example, a mule Scatter-Gather router invokes the three flows concurrently and aggregates the response as a new payload. *A point to note is Scatter-Gather block should have at least two routes to process.* The combined response is captured through Mule Transform Message component, where-in we flatten the payload to an array object. The output resultant payload of Scatter-Gather is an Object of Object. ![Mule flow with a Scatter-Gather routing to three Flow References, then a Transform Message.](../../assets/blog/intro-to-scatter-gather-integration-pattern-6.png) ![Transform Message showing the Scatter-Gather Object payload flattened with flatten(payload..payload).](../../assets/blog/intro-to-scatter-gather-integration-pattern-7.png) **Scatter-Gather Output Payload:** ![Scatter-Gather output index "0" holding the FlowOneResult payload plus its HTTP attributes.](../../assets/blog/intro-to-scatter-gather-integration-pattern-8.png) ![Scatter-Gather output index "1" beginning, showing its empty attachment and property name arrays.](../../assets/blog/intro-to-scatter-gather-integration-pattern-9.png) ![Scatter-Gather output index "1" holding the FlowTwoResult payload plus its HTTP attributes.](../../assets/blog/intro-to-scatter-gather-integration-pattern-10.png) ![Scatter-Gather output index "2" holding the FlowThreeResult payload plus its HTTP attributes.](../../assets/blog/intro-to-scatter-gather-integration-pattern-11.png) ![Continuation of the index "2" entry showing its GET method and TestScatterGather request paths.](../../assets/blog/intro-to-scatter-gather-integration-pattern-12.png) The payload size is 3 after the Scatter-Gather processing is complete. ![Mule Debugger showing the Scatter-Gather payload as a LinkedHashMap of size 3.](../../assets/blog/intro-to-scatter-gather-integration-pattern-13.png) ### Result of Scatter-Gather: The flow has been tested using POSTMAN. ![Postman GET to localhost:8081/TestScatterGather returning the aggregated result1, result2, result3 array.](../../assets/blog/intro-to-scatter-gather-integration-pattern-14.png) The debug process shows that the Scatter-Gather output payload has been aggregated (remember, we have flattened the payload to make it an array object). ![Mule Debugger showing the flattened payload as an array of the three flow results after Transform Message.](../../assets/blog/intro-to-scatter-gather-integration-pattern-15.png) The example shown above is a very simple case of implementing and understanding Scatter-Gather Pattern. In the [next post](https://www.prostdev.com/post/scatter-gather-integration-pattern-mule-4-part-2), we shall add more twists with different service call routings and discuss the topic of error handling in case Mule 4 Scatter-Gather. Until then, try to find your reflections of the *scattered* thoughts. Oh wait! The quiz? Yes! The answer to the first quiz is **20 seconds** and the second is **False**; Scatter-Gather doesn’t halt the processes abruptly with an effective error handling in place, indeed it will gather all the route information including the route which has errored and aggregates the results as output. In order to retrieve only successful route payload and ignore errors from Scatter-Gather processing, this can be efficiently handled using Mule 4 error handling process. We shall discuss the Scatter-Gather error handling cases in our next post. I hope you enjoyed the post. Please subscribe to ProstDev for more exciting topics. Hasta luego, amigos! ## References - [Enterprise Integration Patterns - Fire-and-Forget](https://www.enterpriseintegrationpatterns.com/patterns/conversation/FireAndForget.html) - [Enterprise Integration Patterns - Scatter-Gather](https://www.enterpriseintegrationpatterns.com/patterns/messaging/BroadcastAggregate.html) - [Enterprise Integration Patterns Using Mule](https://docs.mulesoft.com/mule-runtime/4.3/understanding-enterprise-integration-patterns-using-mule) - [Mule Docs - Scatter-Gather Router](https://docs.mulesoft.com/mule-runtime/4.3/scatter-gather-concept) ## GitHub repository [ProstDev GitHub - Scatter-Gather Example](https://github.com/ProstDev/apdev-scatter-gather-example) --- ## JSON Module Component in Mule 4 Source: https://prostdev.com/post/json-module-component-in-mule-4 | Published: Jun 16, 2020 | Category: Tutorials *GitHub repository with the Mule Project can be found at the end of the post.* The JSON Module Connector is used to validate a JSON Payload against the predefined JSON Schema of your choice. Since schema validation falls outside the scope of DataWeave Functionality, the JSON Module can be used. To use the JSON module, add it manually to your Mule app in Anypoint Studio or Design Center; or, if you want to use Maven, you can add the following dependency into your `pom.xml` file: ```xml org.mule.modules mule-json-module RELEASE mule-plugin ``` For better understanding of how the JSON Module works, I did a proof of Concept. Below is the screenshot of the flow which includes the JSON Module component. ![Anypoint Studio flow: Scheduler, Read, Logger, Validate schema, plus an On Error Propagate handler](../../assets/blog/json-module-component-in-mule-4-1.png) To validate the JSON payload, I need an example JSON file (in my case, `document.json`) and I would need the JSON schema file (`mySchema.json`) to validate my example JSON file against it. Since I chose to read the `document.json` file, I placed it below `src/main/resources/examples`: ![Project tree under src/main/resources showing document.json in examples and mySchema.json in schemas](../../assets/blog/json-module-component-in-mule-4-2.png) The `document.json` file looks like this: ![document.json array of employee records with some empty employeeID and employeeName values](../../assets/blog/json-module-component-in-mule-4-3.png) The schema against which the `document.json` file would be validated is defined as below: ![mySchema.json defining an array of objects requiring employeeID and employeeName with length limits](../../assets/blog/json-module-component-in-mule-4-4.png) A fixed frequency scheduler is set to run the flow every few seconds/minutes. The `document.json` file is read using the File Read operation as shown below: ![File Read operation configuration with File Path set to examples/document.json](../../assets/blog/json-module-component-in-mule-4-5.png) Once the file is read, the payload is saved in the Target Variable *jsonDoc* as shown below: ![Read operation Advanced tab with Target Variable set to jsonDoc and Target Value to payload](../../assets/blog/json-module-component-in-mule-4-6.png) After the file is saved in the *jsonDoc* variable, it is validated against the schema using the JSON Module as shown below: ![Validate schema component configured with the schema path and content set to the jsonDoc variable](../../assets/blog/json-module-component-in-mule-4-7.png) As per our example, `document.json` does not match `mySchema.json`, so, it throws a `JSON:SCHEMA_NOT_HONOURED` exception. This exception is caught in the error handler and the `error.errorMessage.payload` is written to a file as shown below: ![Write operation in the error handler saving the error payload to errorMessage-pay.csod](../../assets/blog/json-module-component-in-mule-4-8.png) The file will contain the following error message payload: ![JSON validation error output listing schema violations like values too short and missing required properties](../../assets/blog/json-module-component-in-mule-4-9.png) ![Continued JSON validation error report detailing more per-record schema violations](../../assets/blog/json-module-component-in-mule-4-10.png) The JSON Module can throw any of these exceptions: - `JSON:INVALID_INPUT_JSON` - `JSON:INVALID_SCHEMA` - `JSON:SCHEMA_NOT_FOUND` - `JSON:SCHEMA_NOT_HONOURED` Some JSON schemas might reference other schemas through a public URI. However, you might not want to fetch those schemas from the internet, mainly for performance and security reasons. The JSON Module has an option to redirect the schema, which lets you replace an external reference with a local one, without the need to modify the schema itself. The JSON Module also has an option called "Allow Duplicate Keys", which when set true, it will allow the duplication of keys; otherwise, it will fail. This is why the JSON Module is a very useful component for schema validation. Enjoy the benefits of using this component :) Thank you for reading. -Pravallika Nagaraja ## GitHub repository [ProstDev GitHub - JSON Module Component Mule 4](https://github.com/ProstDev/JSONModuleComponentMule4) --- ## Intro to Regression Testing with Postman Source: https://prostdev.com/post/intro-to-regression-testing-with-postman | Published: Jun 8, 2020 | Category: Tutorials *Gist file with the tests script and GitHub repository with the Postman collection can be found at the end of the post.* There are different types of software tests that a team can build to make sure that the final product will work as expected. However, in this post, we’ll focus on one particular type: Regression Testing. ## What is Regression Testing? Regression testing is a type of software testing, in which tests are created to ensure that a new change to the code did not break existing functionality. Let’s illustrate this definition with a metaphor: Imagine that you have a car that works perfectly fine; and you know this, because you checked that everything was working: - Do the lights turn on? Yes. - Does the car start? Yes. - Does the air conditioner work? Yes. - Do the windows go down and up? Yes. - Does the radio work? Yes. - Are my tires pumped up? Yes. Now, let’s say that you installed some neon lights to your car and then when you tried to turn the car on, but guess what? It didn’t work! The lights must’ve done something to the car, because it was working before installing them. Remember the 6 tests that you performed before, to make sure that your entire car was working? Well, now that you installed the neon lights to the car, let’s do these tests once more… - Do the lights turn on? **No**. - Does the car start? **No**. - Does the air conditioner work? **No**. - Do the windows go down and up? **No**. - Does the radio work? **No**. - Are my tires pumped up? Yes. So, a lot of them are failing now, and the ones that are failing are relying on the car’s battery to be able to work properly. It’s possible that when you installed the neon lights, something went wrong and it messed with the battery. You got to this conclusion by paying attention to the failed tests; and you were able to isolate this, because you did one additional test that didn’t have anything to do with the battery, and it did not fail. This set of 6 tests, are actually regression tests. Why? Because you are performing different tests to make sure that the whole car is working properly, after you added new functionality to it. If you add more detailed tests, you will ensure even more that the car doesn’t break with a future change. ## Creating basic tests with Postman You can download Postman for free from their website: [https://www.postman.com/downloads/](https://www.postman.com/downloads/) Once you have it, open it, and it will look something like this: ![Postman opened on the Collections tab with no collections yet created](../../assets/blog/intro-to-regression-testing-with-postman-1.png) On the left side, you have the menu to create a new collection. Collections are basically the folders where you can keep your requests. Let’s create one by clicking on “Create a collection”, and name it however you want. ![A new empty collection named MyNewCollection in the Postman Collections sidebar](../../assets/blog/intro-to-regression-testing-with-postman-2.png) Now, if you click “Add requests”, you can start creating specific calls to your API. I’ll be creating some tests for the Slack API. Here I’m sending a request to `slack.com/api/team.info`, and I’m correctly getting a response back. ![GET to slack.com/api/team.info returning 200 OK with ok:true and team name ProstDev](../../assets/blog/intro-to-regression-testing-with-postman-3.png) I will create some tests for this request, to make sure that the response I get from this call is always successful. To make sure that this is the case, I want to check the following: - That the Content-Type header is present in the response. - That the Content-Type header in the response is of “application/json”. - That there is a response body. - That the response body is in a JSON format. - That the response body contains “ok”: true. - That the team.name in the response is “ProstDev”. - That the response status is 200 OK. I go to the request’s “Tests” tab and I create the previous 7 tests in code (Note that Postman scripts are based on JavaScript), then I simply run the request again and see if the tests passed or failed from the “Test Results” tab: ![Postman Tests tab with seven test scripts and all 7 passing in the Test Results panel](../../assets/blog/intro-to-regression-testing-with-postman-4.png) As you can see, all of them passed as expected. But this is only one request that can be done to the Slack API, if I were to test more, or most of the functionality from it, I would have to add **at least** one request per method, each with their own set of tests. The number of tests and the number of requests depend on the amount of detail that you want to achieve with the testing, or how many different results you can get from each request. The previous request was checking that the response was a "successful" response - this means, no error whatsoever. But now I'm going to create a *negative scenario* - this means that the response that I'm expecting to get is supposed to be an error, because I'm testing that an incorrect request **does** return an error. So, for this incorrect request, I'll add tests that check that I am indeed getting an error back, and that I am actually getting the error that I was expecting to receive. Here’s the response I get when trying to call `slack.com/api/unexistent`: ![Slack response with 200 OK but ok:false and error unknown_method for an invalid method](../../assets/blog/intro-to-regression-testing-with-postman-5.png) This time we received “ok”: false in the response, and an error that specifies “unknown_method”. You will notice, however, that the status code remained as 200 and we didn’t receive any of the other possible error codes (3xx, 4xx, 5xx). This is Slack’s specific design for their API. We will create some tests that check for all these as well. ![Negative-scenario test scripts checking ok:false and error unknown_method, all 7 passing](../../assets/blog/intro-to-regression-testing-with-postman-6.png) We copy/pasted the same tests we had before, but made some minor tweaks to match our response for the negative scenario. We changed test [#5](https://www.prostdev.com/blog/hashtags/5) to search for “ok”: false instead of “ok”: true, and test [#6](https://www.prostdev.com/blog/hashtags/6) to search for “error”: “unknown_method”. Note that we didn’t have to modify test [#7](https://www.prostdev.com/blog/hashtags/7), because the status code is still 200, even though the request returned an error. Now, in real life, you’d have created a lot more requests and tests to make sure that the API would not break with new functionality. Once all your tests are created, you don’t have to run all of them one at a time; you can click on the “Runner” button, located on the top-left side of the app. This will open a new window that looks like this: ![Postman Collection Runner with the Slack-API collection selected before starting a run](../../assets/blog/intro-to-regression-testing-with-postman-7.png) Here you can simply select your collection, and click on the blue “Run” button at the bottom. Postman’s Runner will run all of your requests and tests, and will give you a visual report: ![Collection Runner results: 14 passed, 0 failed across the team.info and negative-test requests](../../assets/blog/intro-to-regression-testing-with-postman-8.png) In this case, all our tests were successful. How would this help me in the future? Let’s imagine that you, as a developer, introduce a new functionality to the Slack API. You’re sure that everything is working, but before continuing with the process of pushing your code to the remote repository, you run the set of Regression Tests, using Postman’s Runner. If even one of the tests fails, you’ll know that you broke some functionality that you weren’t aware of; so, you have to go back to the code and fix that error. After you fix it, you have to run all the tests again to make sure that everything works; and so on, and so on, until all of the tests are passing and your new functionality is working as expected. Let’s do a quick recap of what we learned today: - Regression Testing is one type of software testing, in which tests are created to ensure that a new change to the code did not break existing functionality. - Postman is a great tool to create regression tests for your API, and you can use Postman’s Runner every time that you need to add new features, to make sure that you did not break any existing functionality. Stay tuned for more posts to use Postman as your testing tool! I really recommend it, and you’ll see why ;) *Prost*! -Alex ### GitHub repository [ProstDev GitHub - Postman Collections Examples](https://github.com/ProstDev/postman-collections-examples) ### More links - [HTTP Status Codes](https://www.restapitutorial.com/httpstatuscodes.html) - [Postman](https://www.postman.com/) - [Download Postman](https://www.postman.com/downloads/) - [Slack API](https://api.slack.com/) - [Slack API - team.info method](https://api.slack.com/methods/team.info) Gist file with the first tests script (successful request): ```js var jsonData = pm.response.json(); pm.test("Content-Type is present", function () { pm.response.to.have.header("Content-Type"); }); pm.test("Content-Type is application/json", function () { pm.expect(postman.getResponseHeader("Content-Type")).to.include("application/json"); }); pm.test("Body is present", function () { pm.response.to.have.body(); }); pm.test("Body is a valid json", function () { pm.response.to.be.json; }); pm.test("ok: true", function () { // #5 pm.expect(jsonData.ok).to.eql(true); }); pm.test("team.name: ProstDev", function () { // #6 pm.expect(jsonData.team.name).to.eql("ProstDev"); }); pm.test("Status code is 200", function () { // #7 pm.response.to.have.status(200); }); ``` --- ## Combining Objects: Concatenation in DW 2.0 Source: https://prostdev.com/post/combining-objects-concatenation-in-dw-2-0 | Published: May 28, 2020 | Category: Guides In this post, I will explain what object concatenation is and how to utilize it. You may be familiar with other language’s concatenation functions, especially when using two strings. For example: ``` “Hello”.concat(“ World!”); “Hello” + “ World!”; print(“Hello”, “ World!”) Concatenate ‘string “Hello” “ World!” ``` This can also be achieved in DataWeave with the ++ function. ```dataweave “Hello” ++ “ World” ``` But, can this also be used for objects? Keep reading to find out! ## What is object concatenation? When I say “object concatenation”, I mean the action of combining 2 different objects, and creating one single object containing all the fields (or keys) and values of these objects. For example, we have these 2 JSON objects: ![Two separate JSON objects side by side, one with Key1/Key2 and one with Key4/Key5](../../assets/blog/combining-objects-concatenation-in-dw-2-0-1.png) And we want to achieve this: ![A single merged JSON object containing all six keys from both source objects](../../assets/blog/combining-objects-concatenation-in-dw-2-0-2.png) In the end, we have a single object, that contains everything that the other 2 objects had. This is object concatenation. ## Object concatenation using ++ The most commonly used way to achieve object concatenation in DataWeave, is the ++ function. The previous operation, using DW, would look something like this: ![DataWeave script using object1 ++ object2 and its merged JSON output](../../assets/blog/combining-objects-concatenation-in-dw-2-0-3.png) You can see why this is one of the favourites. It’s easy to read and understand. It is very intuitive as well (object1 plus object2). There is another syntax to do the same thing: ![DataWeave script using prefix ++(object1, object2) syntax producing the same merged output](../../assets/blog/combining-objects-concatenation-in-dw-2-0-4.png) In this syntax, you are writing the function first, and then positioning the first and second arguments inside parentheses, separated by a comma. Either way is correct; it just depends on what you are used to or what is the standard practice in your project. Both parentheses and ++ do the same thing. ## Object concatenation using {( )} This syntax can be cleaner to see in code, but it can also be a bit confusing for new developers if they’re not familiar with the use of {( )}. ![DataWeave script using the parenthesized object-destructor syntax and its merged output](../../assets/blog/combining-objects-concatenation-in-dw-2-0-5.png) I’m sure you’re wondering how this works, if you haven’t seen this syntax before. When there are parentheses ( ) inside curly brackets { }, the parentheses become object destructors. The object in question is separated into pairs of keys and values, and these are then added to the new object that’s being created by the outer curly brackets { }. This transformation would go like this: Step 1: destroy object1 and object2 into their key/value pairs ![Both objects destructured into separate key/value pairs with a blank line between them](../../assets/blog/combining-objects-concatenation-in-dw-2-0-6.png) Step 2: create a new object that contains all these key/value pairs. ![The destructured key/value pairs combined into one new object with all six keys](../../assets/blog/combining-objects-concatenation-in-dw-2-0-7.png) I bet now that you understand how it works, you want to change all your old transformations from using ++ to using {( )}. At least that was my first reaction ;) ## Object concatenation using {( [ ] )} In case you’re not familiar with DataWeave syntax, you can create an array of any type of data, using square brackets [ ]. For example, to create an array of numbers, you’d write it like this: [1, 2, 3, 4, 5]. “Wait, what? Why would I use arrays if I can just use parentheses for my object destructors?” Just hear me out… isn’t it a bit annoying to have to add parentheses for each of the objects? Sometimes I forget to add them and I end up with syntax errors, because you can’t just do this: ![DataWeave script listing two objects without the inner parentheses, raising an invalid input error](../../assets/blog/combining-objects-concatenation-in-dw-2-0-8.png) Turns out that the parentheses destructors not only destroy an object into key/value pairs, they also destroy arrays containing objects! Whaaat? You can do something like this: ![DataWeave script using the {([ object1, object2 ])} array destructor syntax with successful merged output](../../assets/blog/combining-objects-concatenation-in-dw-2-0-9.png) See? Now you can list your objects inside an array (using square brackets [ ]) and then use the object destructor {( )} on that array. Yes, this is the most complex syntax out of the 3 that we saw today. It can be quite confusing for new developers, and you may have to write this explanation out in your internal project’s documentation (if you don’t want to keep repeating the same thing every time you get a new resource). Lucky for you, you can send them the link to this post! In summary, **there are three options** to concatenate objects in DataWeave. 1. You can use the most popular choice by Muleys ```dataweave object1 ++ object2 ++(object1, object2) ``` 2. You can opt for a cleaner, but less known option, using the object destructor: ```dataweave { (object1), (object2) } ``` 3. Or, the most complex one, but without having to surround each object in parentheses: ```dataweave {([ object1, object2 ])} ``` Do you know of any more ways to achieve the same output? Leave me a comment and let me know! Stay safe, get a beer, and *Prost*! -Alex ## More links - [DW Core Functions - PlusPlus](https://docs.mulesoft.com/mule-runtime/4.3/dw-core-functions-plusplus) - [Gist from Alex Martinez - DW 2.0 Cheatsheet](https://gist.github.com/alexandramartinez/f7037eaf970fb02a04017657cd18b8ad) ## Videos - [Getting to the point (under 30 seconds)](/video/3-ways-concatenate-objects-dataweave) - [Complete explanation](https://youtu.be/4OGSdpmtR-s) --- ## MuleSoft Training and Certification Source: https://prostdev.com/post/mulesoft-training-and-certification | Published: May 25, 2017 | Category: Guides Do you sometimes feel like you want to get a certification, but you’re not sure about which one? Or maybe you do, but you have to spend some money just to start the learning process and then pay some more to get the exam. Maybe you start paying for the classes, but then you decide it’s not really what you wanted, so you feel obligated to finish because you already paid for it, but not because it’s what you’d like to do… Some other scenarios come to my mind, but you got the idea. If you ever had some of these problems, or you just want to get a new certification in a new technology, then, you should read this post! ## -MuleSoft? -Yes! MuleSoft! ![Network of connected app and service icons linked by MuleSoft nodes](../../assets/blog/mulesoft-training-and-certification-1.png) Yeah, it sounds really nice, but, what **is** this *MuleSoft?* MuleSoft is the name of the company that created these technologies that help to integrate services. I’ll talk more about what they do and some examples later in this post. About the name, you got it right… Their representative animal is a Mule! But why? > The “mule” in our name comes from the drudgery, or “donkey work,” of data integration that our platform was created to escape. Also, like a mule, we deliver the strength of a donkey to haul the heavy workload, and the speed of a racehorse to get it done quickly. MuleSoft fits the bill. Its name is Max, by the way. Max the Mule. ![Photo grid of Max the Mule figurine posed at travel destinations worldwide](../../assets/blog/mulesoft-training-and-certification-2.png) ## Awesome! But… what is it that they do? They help their clients to achieve a customizable integration with their APIs. It is convenient, because the development can be done without actually touching a line of code and just dragging-and-dropping the components into the MuleSoft Application (using [Anypoint Studio](https://www.mulesoft.com/platform/studio): an IDE based on Eclipse). You can design an API before starting the actual development with [API Designer](https://www.mulesoft.com/platform/api/anypoint-designer), and get the chance to test it before implementing it, using the API Console and the Mocking Service that is integrated with the designer. You can even create tests the very same way you develop the application (with drag-and-drop), using [MUnit](https://www.mulesoft.com/platform/munit-integration-testing). These are just some examples of all the products that MuleSoft offers, you can find the full list in their official website: [www.mulesoft.com](https://www.mulesoft.com/). ## And… what about the trainings? Finally, we get to the coolest part! ![MuleSoft Training and Certification page with Developers, Architects, Operations options](../../assets/blog/mulesoft-training-and-certification-3.png) There is a catalog for different trainings and some of them can get you a certification. Some of them are not free, or have to be taken with an instructor. **BUT!!!** We started this post talking about the costs and the regrets. Hopes and dreams, and stuff… There **is** a 5-days online fundamentals course for you to take whenever it suits you better. It is called [Anypoint Platform Development: Fundamentals](https://training.mulesoft.com/course/development-fundamentals-mule4). The only thing you need to do is to watch the videos and practice with the instructor. You will also receive pdfs with the content. After you finish the training, you will get a voucher in order to take the certification exam for free. Yes. You only have to complete the training, study and practice on your own, and get certified! You have 2 attempts to pass the exam. My advice, practice and read a LOT before taking it. If you don’t make it in the first attempt, don’t worry, you still have another, but take that as a sign and investigate a little bit more about what you didn’t know how to answer. I really hope I could get your attention on a new technology. Even if you don’t get a certification or register to a training, I’m glad you stayed ’til the end of the post and now you have an idea of what MuleSoft is, what it does, and how can you learn. I have to get this out of my chest… I actually completed the training I mentioned before, it was not that easy because of my schedule, but it was really fun, each step of the way. I highly recommend it, even if you don’t want to follow this career path. It opens your mind to a new world (if you’re not familiar with integration technologies), and gives you a newer perspective. I had so much fun using the API Designer, it was the best of the training, in my opinion. Thank you for reading, guys! Get a beer for me, I just got some teeth removals, so I can’t. Don’t forget to say Prost! -Alex --- ## Less time, less cost, less resources, more apps! Source: https://prostdev.com/post/less-time-less-cost-less-resources-more-apps | Published: Aug 23, 2016 | Category: Opinion We’re not supposed to judge a book by its cover, but what happens when you try to get the online menu of your favorite restaurant and you can’t even find their webpage? Or what if you want to get a tattoo, but you can’t find the tattooer’s Facebook page? You may think they’re not professional enough, or they’re not “official”. In today’s world, mobile apps are as important as having an official site for your company, and creating them gets easier and easier. Today we can create a website in minutes and we don’t even have to write a single code line at all! What if you could create a mobile app in some weeks, or months? ## This is Ionic! ![Ionic homepage hero "Create incredible apps" with an iPhone showing an app](../../assets/blog/less-time-less-cost-less-resources-more-apps-1.png) If you’re thinking of creating a new app and you’re worried about choosing Android or iOS, or if you’re a designer and you don’t want to learn how to program an app for two or more different operating systems, then you should check out this framework. You just have to worry about basic HTML, CSS, and Javascript (that, if you want a more personalized app). Besides, you will be learning [Angular](https://angularjs.org/)as you write your new app. > Ionic builds on top of Angular to create a powerful SDK well-suited for building rich and robust mobile apps for the app store and the mobile web. Ionic not only looks nice, but its core architecture is built for serious app development. Once you have a progress on your app and want to try it, you can either run it as a Web App, or use *Cordova* or *Phonegap* for deployment. You can develop one app and get it running on Android or iOS without having to work twice! > Ionic is modeled off of standard native mobile development SDKs, bringing the UI standards of native apps together with the full power and flexibility of the open web. Ionic runs inside [Cordova](http://cordova.apache.org/) or [Phonegap](http://phonegap.com/) to deploy natively, or as a Progressive Web App. Develop once, deploy *everywhere*. But that’s not it. Let me introduce you to [Creator](http://ionic.io/products/creator). ![Ionic Creator landing page "Build amazing mobile apps, faster" with a desk and iMac](../../assets/blog/less-time-less-cost-less-resources-more-apps-2.png) If you’re not into programming, you can always choose to use this tool that Ionic created just for you. You can start a mobile app just by clicking some buttons, and using this friendly *drag and drop* interface. Just click a button for a new project, ![Green "+ New Project" button in the Ionic Creator toolbar](../../assets/blog/less-time-less-cost-less-resources-more-apps-3.png) write the name of your project, choose its type, and one more click to create it. ![New Project dialog naming the app and choosing a type like Side Menu + Tabs](../../assets/blog/less-time-less-cost-less-resources-more-apps-4.png) And now, you have your new app ready to be personalized! ![Ionic Creator drag-and-drop editor with the new app previewed on an iPhone](../../assets/blog/less-time-less-cost-less-resources-more-apps-5.png) You can choose your device, you can see a preview, you can export your app, and more. > Your time is precious. **Creator** has your back. Spend more time on building a great app experience, and less time managing separate code bases and devices. Export cross-platform mobile prototypes that look, feel, and perform amazing across all modern phones and devices. Wether you like code or design, I think this framework is ideal for mobile apps. There is a lot of documentation, if you prefer reading. There are YouTube tutorials, if you learn better that way, or you can just start by yourself! This is just an example of how easy it is to get along with new technologies. If you’re a fan of learning new things, then you will never be bored in this world. New programming languages, new architectures, new tools, new frameworks are being developed right now. One thing is for sure: if you work in this area and you like to learn once and stick with it forever, you will never succeed. Something new comes out, gets popular for some seconds, and then it’s replaced by something newer. I encourage you to continue learning every day, even if it’s just a fun fact you didn’t know before. I hope this post was of your interest, and if you already knew all this, you can contact me and tell me more about it, maybe I don’t know something that you do. Also, if I wrote something wrong, please let me know! I’m human too 🙂 Thank you so much for reading! Now go get a beer and say Prost! -Alex. --- # Video Transcripts ## Claude Code vs CurieTech AI: Solving Advent of Code 2025 Days 1 & 2 in DataWeave (MuleSoft) Watch: https://www.youtube.com/watch?v=C0QB4Z1LwxM | Page: https://prostdev.com/video/claude-code-vs-curietech-advent-of-code-dataweave | Series: AI Showdown: MuleSoft Edition (2026) ### Intro Hey everyone, my name is Alex Martinez and in today's video we're going to do something really exciting: we're going to compare AIs using DataWeave. For this comparison I'm using Advent of Code 2025 — the most recent one. The AIs (not me!) solved Days 1 and 2, Parts 1 and 2 — so four different challenges. I used Claude Code on MAX effort and CurieTech AI. For Day 1 I used CurieTech's MCP server inside Claude Code. For Day 2 I used CurieTech directly through its chat. The first couple of days of Advent of Code aren't very complex, so the generated code — and the execution time — doesn't differ much between Claude and CurieTech. But there are other things worth keeping in mind. Let's see how it went. ### Day 1 — Claude Code We're in Advent of Code 2025, Day 1. I copied the whole challenge, went back to VS Code with my Claude Code extension, and said: "I'm trying to solve a programming challenge using DataWeave. I'll attach a scenario, and you have to create the DataWeave code to run it." Then I pasted everything I copied. My effort setting is MAX. After a few seconds it generated DataWeave code plus a brief explanation of how the script works. I took the code, pasted it into the Playground, and got the expected result for the example input. Then I grabbed the real puzzle input from Advent of Code, pasted it into the payload, ran it again, and got **997** — the correct answer. Yay Claude. For Part 2 I didn't even read it — I just copied everything and told Claude this was a DataWeave challenge that had a Part 1, and here's Part 2. It correctly referenced the Part 1 code, modified it for Part 2, and generated new code. I ran it, entered the answer, and it was correct. ### Day 1 — CurieTech AI (MCP server) Now to CurieTech. I'd already installed the CurieTech DataWeave MCP server — you can find the install command in their docs, and you'll need to generate an API key in CurieTech, which I did. I did the same thing as before: copied the whole Part 1 challenge, but this time told Claude to solve it using the CurieTech AI MCP server. It asks for permission before running, then generates a task you can see in the browser version of CurieTech. It works for about ten seconds and checks whether the task finished. It gave me a brief explanation and the full code. I copied it into the Playground, set the input, ran it, and got **997** again — the same correct answer. For Part 2, I again asked it to solve using the CurieTech MCP server. After a few seconds it generated code; I ran it and got **61**, which was wrong. So I asked it to try something different. I set my effort to low to make sure it was using the CurieTech MCP server and not Claude, and gave it more context. CurieTech noticed something useful on its own: the payload had line numbers prefixed, but the script expected raw instructions. It sent the corrected version, generated a task, and this time it returned a **verified solution** — it even updated my `mapping.dwl` directly. Note that Claude didn't run any of these validations, so if you want accurate validation, CurieTech is the better option. I ran it and got the correct answer: **5978**. Back in CurieTech you can actually see the chats and tasks it generated — all four, with the DataWeave scripts it created. Some of those we never saw because it was working behind the scenes, but in the end it gave the answer I needed. ### Day 2 For Day 2 I copied everything again. I hadn't solved this one yet, so I created a new chat directly instead of creating tasks, and asked CurieTech to help me solve the DataWeave puzzle. It gave me DataWeave code, made sure it matched the expected output, and even left a **performance note**. Then I gave it the actual input and asked it to generate the result directly — so I didn't even have to copy the code into the Playground myself. Meanwhile I ran the same thing with Claude, and it was still thinking — this puzzle was a little more challenging. CurieTech produced the answer (no new tasks, just the chat); I copied it and it was right. Back to Claude: when I ran its code with the real input, it threw an **error**. So I selected the whole error and told Claude to make sure the code actually works. While it fixed that, I copied Part 2 into CurieTech. Claude figured out why the error happened, I ran it again, and got the answer — which matched. For Part 2, CurieTech had already taken the real input; I just asked it to generate the result without commas so it was easier to read. Both parts were correct with CurieTech, and after Claude's fix, Claude got them too. ### Reviewing the code So which AI did better? Let's look at the code side by side. **Day 1, Part 1:** Using the Timer in the DataWeave Playground, the execution times were extremely close — milliseconds apart, even running in the browser-based Playground (which can choke on large payloads or heavy iteration). The fact that all four ran cleanly says a lot. The structures were similar too: both used nested `do`/`var` blocks, `splitBy`, `reduce`, `map`, `filter`, and `mod`. Honestly impressive — both did a really good job, and the answers matched. **Day 1, Part 2:** Here the approaches start to diverge a bit. Claude leaned on `splitBy` then `reduce` with a couple of `mod`s and `if/else`. CurieTech used more functions — `splitBy`, `map`, `filter`, type coercions (`as Number`) — yet still matched Claude on timing. **Day 2, Part 1:** Both got complex: `pow`, `max`, `min`, `floor`, `flatten`, `map`, `replace`, `splitBy`, `trim`, `filter`, `sum`, `sizeOf`. CurieTech stood out for using clean conventions and typed signatures, and remember it had also given a performance note. **Day 2, Part 2:** Both were complex because the problem is complex. The standout difference: CurieTech used proper **types**, while Claude tended to use `Any`. The types make CurieTech's code a little more readable — genuinely helpful for someone reading the code without having read the challenge (like me). In the Playground, hovering over a typed expression actually tells you it's a Number, whereas the untyped version doesn't. ### CurieTech's performance notes One thing I really liked about CurieTech was the performance notes. On Day 2 Part 1 it explained that the ranges here are tiny, so brute forcing works — but if you ever face huge ranges, you'd want to generate candidates, loop over half-lengths, and concatenate instead of scanning every number. It even gave me a formula for double-digit numbers of half-length K. And it flagged a subtle gotcha: some IDs match multiple factorizations and could be triple-counted, so Part 2 needs deduping. Claude gave some insight too — it explained the error and the new challenge — but CurieTech's performance notes were the highlight for me. ### My two cents In summary, both did well, and both have pros and cons. - If you want to run code from the CLI / terminal / VS Code, you can use the CurieTech MCP server to run it right there — which is neat. - You can also run it in the cloud via CurieTech's chat, and it generates tasks you can always go back and review — unlike a Claude Code session, which is local to your terminal and harder to recover once you close VS Code. - For someone brand new to both, CurieTech is easier — even just using the chat. Claude can be a little confusing for newbies, but once you know it, it makes a big difference, especially if your code is already open in VS Code. I liked that CurieTech generated typed functions (better readability, fewer mistakes), that it actually **runs** the DataWeave code and returns a working result (so it rarely fails), and that it adds those performance notes. Claude only generates code — it doesn't really run it — so it can ship runtime errors, as we saw on Day 2. So this round was a little inconclusive — both are strong. I'm putting everything in my Advent of Code 2025 repo; you can open each file or jump straight into the Playground links, which include the code and some notes on how I generated it. I'm going to keep making videos comparing different AI tools — let me know what you'd like to see next. (Probably not Codex: I don't have paid access, and honestly I've lost some interest in ChatGPT in general.) Remember to subscribe so you never miss an update, and I'll see you in the next showdown. Bye! --- ## 3 AIs, 1 MuleSoft Prompt — Who Codes Like an Architect? Watch: https://www.youtube.com/watch?v=qB6x64yt5m4 | Page: https://prostdev.com/video/ai-showdown-api-led-claude-curietech-vibes | Series: AI Showdown: MuleSoft Edition (2026) ### Intro Claude Code, CurieTech AI, MuleSoft Vibes. I gave these three different AIs the exact same prompt: build the MuleSoft API-led connectivity network. Three layers, 11 operations, customers and orders. Same brief, same rules, no hints. One of them wrote a documented, tested, restart-proof build. One went full microservices with 41 MUnit tests. And one of them looks great but doesn't actually save your data. Let's find out who did what. ### The prompt Here are the full, actual instructions that I gave every single one of them. Create an API-led connectivity network example with the three layers: experience, process, system — I even put them there. Create a customers plus orders architecture for this. These are the operations you can do: - List all the customers. - List one customer's details. - Get each customer's orders. - See one customer's order details. - Create a new customer. - Edit an existing customer's details. - Delete a customer only if no orders are attached. This one is very, very important. - Create a new order attached to a customer. - Edit an existing order, still attached to a customer. - Cancel an existing order. This does not delete it. Note, this is also very important. - Delete an existing order. Then the other tricky thing that I put here: we are not going to be connecting to an actual DB just yet. Mock and seed the test data in the system layer as if we were connected to a DB. Create as many APIs as you think is needed for this architecture to be successful and to really follow API-led connectivity, without having too much redundancy where code is repeated — but also don't just create one big API for everything. There must be at least three APIs since there are three layers, but there can be more than three APIs if you choose that. Choose the best approach. Think like a MuleSoft architect. Use the latest Mule, Maven, and DataWeave version for the Mule project. So it is a little bit abstract in the sense that there are a lot of things that I'm just letting it decide what to do on its own. But on some other things I'm very specific, like the latest Mule, Maven, and DataWeave versions. That's very specific. Now, to figure out which ones those are was up to them. But also note this: think like a MuleSoft architect. That was just to see what they would do. But being honest, anyone can make 11 endpoints and return a 200 for all of them. The question is who actually built something that you would actually put out in production. ### How many APIs each AI created Let's take a quick look first at how many APIs each one of them created. First we have Claude Code. I used Claude Code with Opus 4.8 in the MAX setting. It created four different APIs: an experience customer-orders API, a process customer-orders API, system customers, and system orders. These two are kind of expected, hopefully. But the important thing is that it decided to create just one process API. The other cool thing is that it created one parent POM to rule all of them, even though each one of them has its own POM. This was something that the other two didn't do. And it also created a RAML, so you could actually see what it generated. Next up we have CurieTech AI, which created five APIs: one experience, two process (so one for each), and two systems. Before we continue with the explanation, I just wanted to show you really quickly here how the five APIs look different than the four that the other two created. Remember that I did say in the instructions to only delete a customer if there are no orders attached to it. So this is how CurieTech decided to manage that: it comes here to the process customer and then checks the orders in the system orders API directly. Whereas the other two will do all of the orchestrating inside the process APIs and will just decide to either call customers or orders depending on the case. So there we have it — one experience, two process, two systems, and no parent POM at the beginning. And finally we have MuleSoft Vibes, which created four different APIs: the experience API, one process API, and two system APIs. Something worth noting though is that when I was running MuleSoft Vibes, I did notice that it was actually using Claude Sonnet in the background. That was very interesting, because then it's pretty much just comparing Claude — since I was using Opus, it's pretty much just comparing Opus versus Sonnet. But also, MuleSoft Vibes in theory contains all of the skills, all of the docs, and the best practices and stuff. So that would be one of the differentiators between just using plain Claude Code or using MuleSoft Vibes. That might also explain why Claude Code and MuleSoft Vibes created four APIs instead of five. ### Do the operations persist data? Now, from my list of operations, are all of them there? Here is the result. I did tell it to mock but that we were not actually going to use a database, so I understand why MuleSoft Vibes is really not persisting anything. But Claude and CurieTech found a way to mock the data and also make it last, which was really cool. Claude is using a file connector to create files in my local where it would persist the data, mocking the database but without actually using a database. CurieTech decided to use an Object Store connector so it would mock my data. And Vibes just created the mock of the sample data being returned, but it actually doesn't persist the data. The other cool thing is that, because CurieTech created five different APIs, it created five different API specs. And on this one — delete a customer only if no orders — this API will actually go and query the order system API. Another difference is that CurieTech is using a PATCH to change the status as canceled, which is again a better practice. ### The scorecard Now let me show you this scorecard. We already went through the number of apps, the three layers, the parent POM, and the RAML (the API specification). As we can see, MuleSoft Vibes didn't create any specs. It just went ahead and created the HTTP listeners, which could also be a good approach since I didn't actually specify to use API specs — and that would have used more tokens for me. So maybe that's what I wanted. It doesn't know. It's just one approach versus the other. The other interesting thing is that none of them actually added any security. Not even in the API specs or in the actual implementation. There was no security at all, which again was expected because I didn't mention any security. But then this one also becomes really interesting: the error handling, both in the spec and in the code. Both Claude and CurieTech added all of that, and MuleSoft Vibes didn't add any. Again, it doesn't have any spec, but it also didn't add any global handler. I did not ask for any error handling, but it was a nice-to-have in case I didn't know what I was doing. Then the thing that I mentioned about the mock and the seed: Claude created a file-backed JSON, so it will persist because it's using a file connector. CurieTech decided to use an Object Store connector, so it's going to persist in memory. I personally like the file connector more than the Object Store, because then I have to do way more setting up for the Object Store. And finally, MuleSoft Vibes did a read-only JSON, which again was expected since I did not really say anything about the approach. But take a look at the MUnit tests. I did not ask for any MUnit tests. MuleSoft Vibes didn't create any, expected. But Claude created 18 cases and CurieTech created 41 cases. That is really interesting. And Claude — I don't think it can run them. I'm almost sure that it cannot run the MUnit cases, so I'm not sure if they run. But CurieTech for sure is running these cases. It will not give you broken MUnit tests. ### The versions The other thing that I did specify in my prompt was to use the latest Mule, Maven, and DataWeave version for the Mule projects. So who actually did that? First of all, the latest Mule version right now is 4.12. Claude went really, really bad with 4.9. But CurieTech and Vibes went to 4.11, which is not bad — it's pretty good. But then CurieTech made a mistake in four apps: it did add the 4.11, but in one app it added the 4.9. So it's close enough. I'm really glad that the thing that I did specify, MuleSoft Vibes was able to do. The other interesting thing about CurieTech is that with Claude and with MuleSoft Vibes — they're both using Claude underneath, but whatever — they are being consistent on the versions they're using throughout the whole project network, whereas CurieTech actually used different versions in different apps, like from the connectors and stuff. So that was also very interesting. If you scroll down a little bit in the comparison doc, you'll also be able to see what versions each one of them actually used. ### TL;DR — Claude Code Claude was the most engineering-disciplined. It created contract-first — a RAML and APIkit with a documented design rationale. It created a parent POM. All 11 operations work and data persists across restarts using the file connector. There is global error handling in every app, and there are 18 MUnit tests. The cons are that it was the oldest runtime of all three, and there was no security layer — which we already know none of them did. ### TL;DR — CurieTech AI CurieTech was the most complete and granular. It created five apps per entity split (the process and system layers). All 11 operations worked. It created the most MUnit tests that are actually running — 41 MUnit tests — and it had the richest error handling, plus a dedicated status-change endpoint for the cancel order. Cons: the version drift across its five apps, there's no central POM, no security layer, and the data is lost on restart because the Object Store is in memory. ### TL;DR — MuleSoft Vibes And finally, MuleSoft Vibes. It has a runnable demo. I don't blame it — I was very abstract with what I was asking for. I did not specify a lot of things, and it was running Claude Sonnet underneath, which I don't really like. So, pros: it runs and uses the freshest HTTP connector, there's a consistent runtime version across all apps, and all 11 operations are coded with correct business rules. The cons: it completely skipped the API rigor — no specs, no APIkit, no tests, no handler, no docs. I did not ask for any of those, I know, but it was nice to have from the other two. The data doesn't persist. I also didn't specify that, but the other two tried to do it. And there is no security layer, which none of them had. But it's also worth noting that out of all three, MuleSoft Vibes was the fastest. It took maybe some minutes, not an hour for sure. Claude Code again, with Opus and MAX, it took a little bit, but again not up to an hour. Whereas CurieTech, it took a long time. I'm not really sure how long it took because I left my task at that point. But it also makes sense, because CurieTech wants to give you something that is actually runnable and that actually works. It goes and deploys everything internally, it checks that it works locally, and it even runs all the MUnit tests. If some of them don't work, it won't give it to you like that — it will make sure that all of the MUnits are running and that there are no errors when you try to deploy. So it completely makes sense that it's the one that took the longest. ### So who won? Depends on what you wanted, right? I would want Claude's discipline with CurieTech's completeness, but at MuleSoft Vibes' speed. At the end of the day, if I just want a running demo, and I want it fast, and I want the bare minimum of what I asked, and I don't want it to overthink on everything that I'm telling it — MuleSoft Vibes is the way. If you want the most complete, granular approach with a lot of tests, a lot of safeguards, error handling, and so on, then CurieTech is the way to go. I wonder — what makes MuleSoft Vibes MuleSoft Vibes except ACB? Just taking out ACB, what really is there? Because it's running Sonnet, right? It's already running Claude, so what really is the difference there? It has skills, it has MCPs, it has a lot of things that plain Claude would not have. So I wonder: if I use Claude Code with Opus with MAX and I bring in all of these skills and all these MCP servers, would that be the best approach? I don't know. Maybe something for a future video. ### Wrap-up If you want to take a look at the whole code, the whole comparison, and the whole shebang, just go here to my GitHub repo, `ai-showdown-api-led`. You can find the instructions given, the comparison, the RAML, and the three different folders with all of the APIs that they created inside. Of course, this link is also in the description of the video. Go ahead and check it out. But the real question here is: which one would you have chosen for this? Tell me in the comments below which one was your favorite from everything that you just saw here. And what next video do you want me to do? Remember to subscribe so you can continue watching all my AI showdowns, plus some other MuleSoft tutorials as well. All right, that is all for this video. I will see you in the next video. Bye! --- ## Can AI Build MuleSoft Apps? Cursor AI + Anypoint Code Builder Test Drive Watch: https://www.youtube.com/watch?v=9cOjP7x2DLk | Page: https://prostdev.com/video/cursor-ai-acb-build-mulesoft-apps-test-drive | Series: Adventures in MuleSoft + AI (2025) ### Intro Hi everyone, Alex here. I come to you with AI content now I guess. I wanted to talk about this app called Cursor. Cursor is actually based on VS Code, same as Anypoint Code Builder (ACB), so they are actually kind of compatible. If you download Cursor right now I have the free trial, so you can do everything that I am doing. I think I have a two-week pro trial — anyway, I'm on a trial of some sort. I'm not paying anything. So I kind of want to check it out. ### Setting up the project First of all, I'm going to open some projects here, but I'm actually going to create a new one. I already installed Anypoint Code Builder in Cursor because it does take a while. So I wonder if I can just create one from scratch here. I'm going to do new folder in Downloads — let's do "best-practices" — and open it. This looks pretty much like Visual Studio Code. It's just like the activity bar — I cannot move it. Actually I tried moving it. In appearance you see activity bar position. This is default. In Visual Studio Code you can actually move it to the left or to the top. Right now this one is on the top. I could not move it to the left. Maybe there is a way. I just don't know how. But anyway. So we have the new chat here. I'm going to minimize that for now. If I click here, "develop an integration in Anypoint Code Builder," and I'm here. So let's do project name — actually I already have the thing. I don't know if it's going to create it inside the best-practices folder. Let me just select Downloads then. So let's do "mule-best-practice-template." Mule project, Mule runtime. Let's do 4.9 because that's the latest. Java version 17. And let's say create project. Because I have Anypoint Code Builder, it is going to show me Agentforce and Einstein and all of that fun stuff. But for now I just want to try it with Cursor. Right now you can build it with Agentforce and do a lot of different stuff. You can use a prompt with Agentforce, but for now I'm not going to do that. I'm also not logged in to Anypoint Platform right now. So that's okay. ### Building the flow with Cursor AI Let's go ahead and start from scratch for now. So let me minimize this. To hide the sidebar, use the icon in the top right — oh, this one. Okay. Right now I'm going to leave this as a flow. I wonder if this thing — the agent — can actually create the base thing. Let's try. So I say: "create an HTTP listener and add a transform message to output a JSON payload saying hello world." Let's see if this works. Okay, it created mule-best-practice-template.xml. There's a linter error. I don't know what that means, but it fixed it. It's still doing a bunch of stuff. I'm going to close it for now. Let's see what it did. So it did a flow name "hello-world-flow" with an HTTP listener, HTTP listener config "hello," and the set payload with the message hello world. Okay, I feel like this is good enough. And then the HTTP listener config. Okay. ### Moving config to global.xml So now I want to move the HTTP listener from this file to a new global.xml file. Okay, so it created the global.xml. It added the HTTP listener config in the thing and it removed it from here. Okay, cool. Awesome. So now we are kind of following the best practices there because we have this thing here and now we want to extract the properties. ### Creating YAML properties files So: "create a new properties file. Make it a YAML file and add the host and port only properties only." Okay. So it's going to create an application.yaml with HTTP listener host and port and then it's going to update the global.xml. So now it has the HTTP listener host, HTTP listener port. Yeah, let's accept that. And did it create a file? Let me see. Resources. Yeah, application.yaml. It was smart enough to know that the host is a string, so it put the quotes here and the port is 8081. It's even telling me how to do it in properties. Okay. But it didn't create — so: "add the application.yaml properties file in the global configurations." Okay. So it knows it's on global.xml and it is creating the configuration properties name, configuration properties. It keeps changing stuff. Configuration properties file application.yaml. Awesome. Let's accept it. But then what would happen if we want to have two different environments? So let's say instead of having just one application file: "can you actually create two separate files? One for dev and one for local." Okay, so local.yaml and dev.yaml and then it's going to change the global.xml. Okay, so it knows to do the env property, but it didn't create the other property. Accept all. And let me just tell it: "can you rename dev properties and local properties?" So it's going to delete — I just need to press accept. Deleted local and dev. And now it's creating the local-properties.yaml and the dev-properties.yaml. Okay, so it created both. Now it's going to update the global.xml. Okay, awesome. Env properties. Yes, the functionality remains the same, but now the file names are more descriptive. Awesome. You can still run the application using env equals local for local, env equals dev for development. I don't know if it can add this for me to run, but let's just leave it for now. ### Cleaning up the XML "Prettify the XML files, but don't add comments." Okay. So it changed this. Let's just accept and accept. Oh, but it kept the comments. That's fine. Let's just remove it. Accept and accept. I don't know if this makes it better to read or — what if I use format document directly? Yeah, let's just do that. I like this better. That's just me. Here too. Let's just do format document and then remove this space. Awesome. So save. Save. Let me remove this. Save. Okay. So I have the configuration file, the HTTP listener. This is going to host import. Oh wait, I'm not using application.yaml anymore, right? Yeah. I'm just going to delete it myself. Delete, move to trash. Awesome. So now we have the properties, local-properties. We can add prod and everything, but let's just leave it like that for now. "Create a global property env equals local." And it knows what I mean. I think yeah — global property name env, value local. By default the application will use local-properties.yaml file. You can still override this setting by setting the env when running the application. Cool. So there's that. Oh, I didn't even accept these changes. Accept. Accept. ### Setting up Git So let me see here. If I initialize a repo, what will I see? I see the .gitignore. That's okay. Let's add that. I see the mule-artifact.json with my specifications. I see the pom.xml. Okay. That's the basic stuff. I don't want to have — "Add code to the .gitignore." Accept. Okay, so it added that. If I check here it's added. I can accept the change. .gitignore. My global. I think that's good. My flow. I may come back to that one, but for now it's good. Oh, I just noticed it changed the dev. This should be 8081 in both. 8081. Okay. Add. Add. And the log4j that I never changed. Cool. I don't know why it's doing this thing. It's like we're in a party. But anyway. All right. So now: "Can you help me run it? Can you run this application locally?" Okay. I don't think you know how to run this. Reject. Stop. "You don't know how to run this." Okay, that is fair. That is fair. You helped me do everything else. You didn't change anything else. It is only fair that I run this. ### Running and debugging the app So let's go to run and debug. Man, I really hope that I can change this thing to this side because I don't really like having it over here, but that's okay. So let's start debugging. Let's run it to see if this works. And if not, let's tell this thing how to fix it. Failed to execute. I see. Error loading global.xml. Can't resolve this. Okay, so it did make an error. Let me put this error. "I see the issue. We need to add the HTTP module dependency." Oh. That is right because it added everything. It modified the global, but that was not the issue because it just removed the spaces and whatnot. Stop. Because you added — okay, you added one that wasn't the one that we wanted. "You forgot to add the HTTP listener dependency in the pom.xml. Please add it and make sure the version is the latest." All right, let's see if it knows how to do that. I added the HTTP listener dependency. So it's going to be on the pom. Okay. It was able to add it. It says it's the latest stable version of the HTTP connector. Okay, let's accept it. And yeah, it's going to tell me this is not cool. So I'm just going to go to project properties and fix this. It added an even further version. Okay. So it made sure that it was the latest version, I guess, but that's not the version that I needed anyway. It was not good enough to know what it was. And this is freaking out again. So let's try to run it now. Wait. I don't know why this keeps getting modified. I think it's because I'm opening the thing. Anyway, we modified the dependency and that's it. Cool. You don't have to keep adding stuff. It's just that I want to see what are the changes to my files that are happening. So let's run this now. Okay, failed. And I think I saw the YAML thing. YAML configuration properties only support string values. Yep. To be honest I thought this one was going to pass but apparently it didn't. I'm just going to add it directly. Save. Stop. Save and save. All right. So it should work now. Let's try again to run and let's add the changes here. So we do have to make sure that this is a string otherwise it's not going to run. Okay, I think it was deployed now. ### Testing the endpoint Now can I ask this thing to send a request? "Send an HTTP request to localhost 8081." No. Okay. You don't know. You don't know how to do it. Okay. Fine. Let's do localhost 8081. Oh, it was slash hello, right? Oh, I think I'm — wasn't it a message? Hello world? Oh, so it did not output up. Okay. "Modify the set payload to output an application JSON." What did you do? Value, doc name, mime type. Nope, that is not what I wanted. That's all right. I will just do it myself. So we want an application JSON DataWeave and then the message and it's not liking it. "Fix in chat." Oh, I forgot the output. Oh, well, thank you. I appreciate it. I don't need the mime type. You know what? Yeah, I'm just going to remove it because I don't want it to cause more issues. Stop in the name of love before you break my heart. All right, save. Let's run this again. Oh, what? Oh, I didn't add the embedded. "Add the source test resources embedded folders to the .gitignore, please." Resources. Awesome. Appreciate it. Let's do accept and let's see. So we have — close all. This screen looks really cramped because I have a zoom because I would normally develop something like this because I can still read but it will be really hard for you to read it from your screen. So like as you can see from here it looks really cool if I'm zooming out but you cannot read it. So that's why this looks really cramped because I am zooming in. All right. So that was a change there and we added the source resources embedded in the .gitignore. So we can try running one more time and see if this finally runs. All right, it was deployed. And let's try this again. Yay. So we received the hello world now. And the last thing that I had in mind — let me close this — is can I debug this? I think I can but let me just try. And yes I can debug it. So because Cursor is based on Visual Studio Code you can actually continue using Visual Studio Code. You can install extensions and do any of those things that you do in VS Code. It's only that the activity bar is on the top instead of being on the left like in the regular VS Code. But oh well. ### Closing thoughts Apart from that, it's pretty much doing everything. And maybe it took me a little bit more to do this with Cursor with the agent. But I guess once I know what it can and cannot do, it will be easier. And now that I know that I have this template on my own — like best practices — maybe I can put this out in GitHub and I can tell it to base on that GitHub repo to create my stuff in a new project. So I'm going to try that in the next video. But for now, I think this is really cool to kind of see. Once we can figure out how to use it better, I think this is going to be a really good tool because it did do a lot of things for me like moving the properties and renaming and stuff. You just have to review that what it did was what you wanted. But it did save me the time of doing copy and paste. And yes, you may argue that maybe I spent more time doing the other things, but it's going to be a tool in a while. It's just getting there for now. But I can see the added value into it. So I'm into it. I like it. So I'll see how to make my own life better. All right. That is all for this video then. I will see you in the next video. Bye. --- ## Can Cursor AI Recreate a MuleSoft Project Using Best Practices? (Spoiler: Not Really) Watch: https://www.youtube.com/watch?v=pH_rZN8MCEw | Page: https://prostdev.com/video/cursor-ai-recreate-mulesoft-project-best-practices | Series: Adventures in MuleSoft + AI (2025) ### Intro Hi everyone, my name is Alex and in the last video about Cursor AI and Anypoint Code Builder, we created this Mule best practice template — a regular Mule application that uses some of the basic best practices. I even generated the README with Cursor, which was really cool. This is the project structure: we created the `global.xml` that has all of the global configurations, the `mule-best-practice-template.xml` with the main application flow (it just outputs a hello world in JSON format), and the resources folder with `local-properties.yaml` and `dev-properties.yaml`. So we're creating two different properties files, and they're YAML — I'm not using `.properties` files anymore because I think YAML is a best practice now. The README lists all the prerequisites: how to install Cursor, how to install Anypoint Code Builder (ACB) inside Cursor, the configuration we created, and how to test the app. You run the application and use curl to call it; you also have to set the env property because it's specified as local or dev (if you don't specify, it defaults to local). PS: this README was generated using Cursor too. Really cool. ### Testing if Cursor can infer best practices So we have this project. Now I'm going to test if Cursor can be smart enough to know that that's what I want. I'm going to open my downloads folder, create another project, and see if Cursor can understand the best practices from the existing one. Let me create a new one using ACB. All right. I wonder if it can create a new project using the best practices. Let me ask it to create a Mule project called "testing-cursor" and use the best practices from the other Mule project. Let's see if it knows what to do. Okay, it's going to create the whole thing. For now I'm not going to do that — let me create the base project myself and then I'll ask it to do the other things. It created the "testing-cursor" folder but didn't open it. That's weird. Okay, let me just open it manually. So I was in downloads, testing-cursor… there you go. Open. Okay, so it created the project. Doesn't have any resources, any tests, anything. It doesn't have any best practices because, yeah, that's normal. So let's see. ### Creating the flow Let me ask it to create a flow with an HTTP listener and add a Transform Message component. You can test this flow by making a GET request there. All right. So HTTP listener, Transform, and the HTTP listener config. Let's accept these. Now let me take a look at the code. We have the flow, main flow, HTTP listener, response body — I don't think this is needed — Transform Message, output application JSON, message "hello from Mule." This should work in theory. We can try it out. Wait, it's not going to work because it didn't create the dependency in the pom.xml, right? Yeah, it didn't add the dependency in the pom. That is one of the annoying things with Cursor — it's not really adding the HTTP listener in the pom. So you kind of have to remember that. If you're using this, add the HTTP dependency in `pom.xml` and make sure it's compatible with the Mule runtime and Java versions. It's going to try to add the Mule runtime version in the pom, but yeah. Let's reject this and let me try again: "Add the HTTP dependency in pom." Okay, looks like it added it. Let me just make sure… dependencies. Okay. I don't know if this is going to be compatible with what I have, but we'll see. Accept. And it's not going to be compatible, but that's all right. Let me just fix that manually. And it's annoying that I have to do this manually, but hopefully that is the problem we're trying to solve. Okay, 1.0.3. All right, perfect. Did it apply it? Yeah, 1.0.3. Cool. So now we can try to run it and see if it runs. Deployed. I didn't even see the configuration. Okay, localhost 8081. Perfect. Let me create a new terminal and do it with curl: `curl localhost:8081/api`. "Hello from Mule." Okay, so this works. Awesome. ### Trying to apply best practices Now let's see if we can add some best practices from the other project. Let me modify my project to have best practices — you can base it on this project here. Hmm. I feel like it didn't really understand what to do. So what did you do in testing-cursor? Oh, you created some things. I just wanted the base things — I didn't want anything really fancy. It took out the path and renamed the config. I don't know why it did this. I don't need this. Let's reject this. An error handler? No. I just wanted you to move this. Reject all, accept all. Oh boy. So it did that there. I don't like that. Then in the pom it keeps adding the Mule runtime version, and it keeps adding the wrong one. So okay, this is not working. Configuration module, but it changed my HTTP connector version. It knew that it had to add the local properties, but it didn't actually do it right. Only this would be useful — this is not useful. The properties, maybe like the logging level maybe, but it's not really changing it anywhere I think. The environment property? That's not really useful. Configuration property, the local pro… no. You didn't know. Nope, no. You clearly don't know what you did. Okay, reject all. You clearly don't know what you did. ### Answer: it can't infer best practices from a repo So to answer the question: no, it's not really creating the best practices. Coming from just providing a GitHub repo, it doesn't actually know what to do, because if we take a look at everything it just did… First, let's create the property directory structure and configuration files. Mule resources, Java — but it should already be there. Why are you modifying that? Then the environment-specific YAML files, sure, and then the global configuration file. Update the main flow to use the new configurations, right. Let's update the `pom.xml` to include the necessary dependencies — like, this is where you're losing me. Environment-specific configurations, global configuration, main flow improvements, dependency management, error handling. Yeah. ### Next idea: instructions file I wonder if I just create a doc or a blog post where I can reference it and say, "See what's being listed here and apply that as best practice." Maybe that would work, and that way you can generate just a document or something. Let me see what we have here in context. Do we have files and folders, code, docs, Git, paths, chats, Cursor rules, terminals… I don't know what's "linter errors" and "web." So what if I select "web"? Oh, that's just like taking the thing from there anyway. I can just paste, like I just did — I pasted the GitHub repo. Maybe I can just paste a blog post from somewhere that will have this documentation, and I can put that there. So maybe it can be like, even from the docs — like the official MuleSoft docs — if they create one site with all of the best practices to feed this LLM, and then you can add other context like your company-specific ones. Maybe this will work. ### Closing thoughts All right, so in the next video I will try something like that. I will create a blog post and reference it to see if that improves, but it's not smart enough to just take the GitHub project that I just did and be like, "Okay, this is what you need." So maybe a set of instructions will work. All right, see you in the next video with that experiment. Bye. --- ## Teaching Cursor AI Best Practices for MuleSoft with Rules (Big Improvement!) Watch: https://www.youtube.com/watch?v=8qEXJRJX27E | Page: https://prostdev.com/video/teaching-cursor-ai-mulesoft-best-practices-rules | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone, Alex here again, and welcome to the third part of my AI Cursor series. On the first video, I went ahead and created a project using best practices with Cursor — I had to tell Cursor exactly what the best practices were because it wasn't smart enough. Then on the second one, I used that GitHub repo I generated in the first video to try to tell Cursor to just generate another project with best practices based on that repo. It didn't work. So on this third try, I generated some MuleSoft best practices — just instructions on what to tell Cursor to do. We're going to use this Cursor rules file, put it in our Mule project, and see if that works, if that makes it better. ### The Cursor rules file I'm in my GitHub and I created this gist file, which is an MDC file. These are the rules that I generated so far — just the basic stuff, I guess. I also generated some of these with Cursor. Like, I was asking Cursor to do things and it was either doing them or not. Through trial and error it was finally able to do it for me, so then I asked it to keep adding rules to this file every time that it learned something about what I was telling it to do. We generated this together and I'm going to use this now. ### Setting up the new project Let's go to Cursor. Here I'll go to Anypoint Code Builder. By the way, you can pin Anypoint Code Builder if you just click on this pin right here. Anyway, we are here. Let's develop an integration. I don't need you yet because I'm going to start from scratch. Let's do "testing cursor rules" and let's put this in Downloads. Select empty project, 4.9 latest runtime, Java 17. Create project. From what we've seen, it's not going to open this for me. So I just have to go to File → Open, go to Downloads, select the "testing cursor rules" folder that I just created, and open. And now I will be able to be inside my project. It generated my project and now I'm going to start. Let me close this. Maybe zoom out a little bit. Let's see. "Create a flow with an HTTP listener and a transform message that outputs the hello world string as text." Okay, let's try this and see if it follows the best practices of creating the global elements in another file. Oh, wait. But I didn't add the rules. Okay, well at least this helps as a test — it created the thing here. It didn't add properties, right? So, okay, let's reject this. And I'm going to add the rules. How did I add the rules again? Was it here? Yeah, here. Add context → Cursor rules → Add new rule. So, I'm going to call it "MuleSoft best practices." And now you have the thing here. So now if I go to this other file, I can just copy this whole thing and paste it here. Save. Okay. And now let's ask this thing again. Send. And now let's see if it follows the best practices. Okay. "Would you like me to make these changes to follow best practices?" Yes. ### Testing the best practices Okay. So it created the `global.xml`. Cool. And it added the thing. And then it's going to create the properties file. And then it changed the main file to remove the thing that it moved to the global properties. Okay. So let's take it one at a time. So it edited three files. Oh wait, it forgot the dev properties. But it created the `global.xml` and it has the configuration properties, `env.properties`, it matches here, global property `env.local`. That's what I wanted it to do. The HTTP listener with the host and port. Yes, this is what I wanted. Cool. One down. And then we have the testing cursor rules XML. And it just has a flow with the HTTP listener path `/hello`, and a set payload value "hello world", mime type text/plain. Yeah, pretty much. And finally, it only created the local properties files, but it did add the properties as strings, which is one of the rules — somewhere here. Yeah, "All values in YAML properties files should be strings including numeric values. For example, use `port: "8081"` instead of `port: 8081`." So that was one of the rules and it followed it which is awesome. But you did not create — I mean, it makes sense that it created the local properties only. Let me see. Did I ask to have local and dev? "Configuration property should be stored in YAML files with the naming convention `${env}.properties.yaml`." I didn't ask it to do several. Okay, that's fair. That is fair because I didn't tell it that I wanted it to have also like a dev properties. But I'm sure if I ask it to do it — "Can you also add a dev properties file for when I deploy it to the cloud?" Yep. It created the dev properties. Didn't change anything else. Just that. So, accept. And here's the properties. It has the same thing as the local. So, cool. This works as it should. Well, it pretty much is following best practices and I only had to use one — well, two, I added one for dev. I'm sure I can keep adding like the same thing just like QA or prod or so on. But it added the whole thing and I just needed one single thing. ### Fixing the set payload vs transform message So "Create a flow with an HTTP listener and a transform message." I did ask for a transform message and it gave me a set payload but that's fine. We can ask it to reconsider. "The output is a hello world string as text. Okay. You created a set payload component but I asked you for a transform message component." Okay. So now it changed it to have the transform message. All right. So we can see if this is what we wanted. `dw output text/plain. "Hello world"`. Yeah, that's pretty much what I wanted. ### The missing HTTP connector dependency Oh, I'm missing one thing though for the pom. I am using an HTTP listener and it was not added to the pom as a dependency. And I believe this is in the rules. Or not. Did I not add it to the rules? "Dependency." Yeah. "When adding dependencies to the pom, ensure the version is compatible." Oh, "Always verify connector." Oh, okay. It's not in the rules. Fair. Fair. Okay. So, let me tell it. "You added an HTTP listener component to my project, but you did not add the dependency in the pom file." Oh, close. But that's not what I wanted. I don't want this. I don't want this. No, I don't want this. I wanted this. Okay. No, reject. You know what? What if I try to run it and I will receive the error? Did I reject the files? Yes. Yeah. I'm going to try to run it. Then I will receive the error that the dependency was not added. Then I'm going to tell it to fix it. And then I'm going to tell it to add the rule if it's not there already. Okay. "Can't resolve this dependency or plugin. Maven missing." Okay. Let me just copy the whole error and put it here. Okay. "See the error." "The issue is that we need to add the HTTP connector dependency to the pom file." Yep, that is what I wanted. But I don't think that is compatible. Or maybe it is. "I use version 1.8.0 of the HTTP connector which is compatible with Mule 4.4.0, the same version specified in the mule-artifact." That is not true though. Nope. "The error you were seeing was because the HTTP connector dependency was missing which is required blah blah blah." Okay. "Why do you say Mule 4.4.0 is the runtime version I'm using though?" Yep. "Could not find." Oh, okay. No, you're wrong. I'm going to accept the pom change, but I'm not going to accept the mule-artifact. "So the mule-artifact.json file is located in the root folder not inside src." "Mhm." Okay, it's already in the — blah blah blah. "The correct HTTP connector dependency." "Okay, can you add this information to the rules file?" It might or might not add it to be honest because when I was trying it, sometimes it added it and sometimes it didn't. But we can just — oh, editing. Yes, we can just add it ourselves if it doesn't actually add it. "I see that the rules file already contains the relevant information of Mule runtime and version." Okay. So, "Error includes not matching Mule versions between pom and" — let me just tell it — "mule-artifact.json located at the root directory." Yeah, I can add that. Okay. So then why didn't you follow the rule if it was already there? But okay. All right, that's fine. Let's — let me try to run it as is and see if it's actually compatible. And if it isn't then I will have to go back because maybe the version 1.8.0 — yeah 1.8.0 — maybe the version 1.8.0 is the minimum required. So let's see if that is the case because maybe it is a minimum required because I've been using 1.10.3 which is the latest version but maybe 1.8.0 is a minimum required and then you can change it. ### The Java version issue All right. So it was able to build it. Let's see if it is able to deploy it. And it failed because it is not supported. "Extension HTTP does not support Java 17. Supported versions are 1.8 and 11." All right. So, let's tell our agent about this. Or like can it just go — "The deployment failed. Can you see what went wrong?" "Mule-artifact specifies Java 17, but we haven't configured the Java version in the pom." Ah, I don't think that's the case, but okay, fine. I'm going to let you do whatever you think you need to do. Added Java 17 configuration, added Mule runtime version configuration, added required dependencies. No, that's not it. Okay, so you're not smart enough to go and check the compatibility. That's fair. I mean you did add — wait what if I — "When adding dependencies to the pom ensure the version is compatible with the Mule runtime version specified in mule-artifact.json." Let me remove this. "And is compatible with the Java version specified in the same file." And let me add this — "whenever Mule runtime version is added, mule-artifact" — is there any other mule-artifact? No. Okay. All right. So as you can see you can play around with this and see whatever works best for you. For now I'm just going to let it be and I'm going to modify the version myself. Project properties. I'm sure MuleSoft is going to be releasing an MCP server soon, so you won't have to do all of this manually because it will be smart enough hopefully to already know all of these things. But yeah, 1.10.3. And you saw there, right? "Connector version is not compatible with project's Java version." So, let's just select that one. Apply. And if I check my pom now, I should be able to see my — where is the thing? — dependencies, 1.10.3. But wait, Maven plugin version. Oh, yeah, that doesn't matter. Okay. ### Success Let's try this again. This should work now because I did it myself, but okay. And yep, it was deployed. So now I'm going to tell it to call this app with a curl command. Yep. And we do receive the "hello world". So perfect. We made it work. And as you saw, we had to do a few changes, but I'm sure MuleSoft is going to be releasing an MCP server soon. Don't quote me on that. I'm not 100% sure, but it would be really cool. And since everyone is doing it, I don't see why MuleSoft wouldn't do it. But yeah, I asked one thing which was to create this flow with the HTTP and with the transform message. It did create a set payload component instead of the transform message, but that's okay. I just told it to change it and it changed it. And then the HTTP listener global configuration was added immediately. Well, not immediately. I had to tell it to follow the best practices, but it was added to the `global.xml`. And then it was also smart enough to follow the best practices of creating the global property `env.local`, which is what I told it to do in the rules. And then the configuration properties to follow the best practices that I had told it to follow. And here is the HTTP listener. And it also was smart enough to move the HTTP host and HTTP port into the properties. So it did create only one `local.properties.yaml` file because I told it to use YAML instead of properties. And also here once it added the values instead of leaving them as numeric it changed them to strings because otherwise it would fail which I already knew about. And that is why it's on the best practices thing. And then it didn't create the dev properties because I didn't add that rule to my best practices thing. I only said to create the local properties. So I just asked it to create the properties and it added it with the same configuration. ### Way better in summary Way, way, way better. And if we can take one moment to take a look at this: "All the global element configurations have to go inside a global XML file unless a different name is provided for this file. Properties like host and port should not be hardcoded in the XML files. These values should be referenced from a YAML or property file. Whenever a Mule runtime version is added to the pom.xml file, this version has to match the version from the mule-artifact.json file located at the root directory." This is what I added because apparently didn't know what the mule-artifact was. "Always make sure the properties files added to the project are being correctly configured with configuration properties global element. When adding a dependencies to the pom.xml ensure the version is compatible with the Mule runtime version specified in mule-artifact file again located at the root directory and that it's compatible with the Java version specified in the same file." It did not listen to me on this one. Well, it wasn't there. I didn't have the Java version. So I didn't test that. Hopefully that works. "Always verify connector compatibility by checking the official MuleSoft documentation before adding or updating dependencies." I don't know if it actually did this, but in one of the tests that I had previously done before recording this video, it actually went there. So hopefully it will continue to go there. "When determining the latest compatible version of a connector, always check the MuleSoft Maven repository to find the actual latest available version. Do not assume version numbers or make up version numbers" — because it was doing that before, so I had to add that rule. "Configuration properties should be stored in YAML files with the naming convention `${env}.properties.yaml`." And there are two examples. "The configuration properties module must be added to the pom.xml with the correct version." I think this one is not needed. No, I think this one is not needed. Anyway, "The configuration properties element in global XML should use the format" — because that was a thing that we had. I think it was using like the Mule 3 kind of configuration properties. I don't know. So, it just added this rule itself. "The environment variable `env` should be set as global property in global.xml with a default value like `env.local`. All values in YAML properties files should be strings including numeric values." For example, this and this is also a rule that it added itself. And finally, "The `.gitignore` file must include entries for IDE specific files like `.vscode` and test sources like `src/test/resources/embedded` to prevent them from being committed to the repository." That is a rule that I told it to remember. So it added the rule there. ### Wrapping up Okay. So that was it. I think it was pretty cool. I think we made a lot of progress with that. So here's the file. I just went ahead and added the revisions that I just did. I can continue updating this with whatever I see that is needed. So you can always come and reference to this file. If you go to my actual profile in github.com/alexandramartinez, you can scroll down after all of these things. And here in Pinned, you will be able to see the "use this cursor rules files for your Mule project thingy." And here you can find the file that I was just referencing to. So, okay, there is where you can find it. And yeah, feel free to write stuff, leave me a comment of something that I should change or that works best for you. And that's all for this video. So, in conclusion, the GitHub repo didn't work for the best practices, but this rules file actually worked for implementing the best practices and telling it what to do. And the other thing that I was seeing is that because it is here in your repo in `.cursor/rules`, you have them here. You can list more of them if you want and you can just commit them to your repo. So they will always be there and anyone that uses your repo will be able to reuse these. So that's really cool. There's also a global file where you can get all of these rules in all of your projects, but I just figured it was easier to get it here because it's more accessible and more visible from the project. All right, that is all for this video. I will continue experimenting, I don't know if with Cursor, but with other AI stuff. So feel free to let me know in the comments if you want me to do something specific. Or I'm sure there will be an MCP server for Mule stuff. So I will very probably come here and try it out from Cursor because I'm really liking Cursor to be honest. I don't think I'm going to continue using VS Code alone. I think I'm just going to start using Cursor directly because this agent is really useful. It's like a little intern that you're telling what to do, but you might have to explain some things if it hasn't done this before. So, it's like your little intern, your little assistant. So, I like it. You still have to know what you are coding, as you saw, but once you have the rules set for yourself, this will be a way better and painless experience. All right, that is all for this video then. I will see you in the next video. Bye. --- ## CurieTech AI vs Real-World DataWeave Challenges – Did It Deliver? Watch: https://www.youtube.com/watch?v=bbHcjmtFdiI | Page: https://prostdev.com/video/curietech-ai-real-world-dataweave-challenges | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone. My name is Alex Martinez and today I'm going to guide you through this awesome technology called CurieTech AI. I haven't been able to explore a whole ton about it yet, but we have here different kinds of agents you can use from their platform online. If you go to platform.curietech.ai or curietech.ai, you'll see the landing page and what different things it can do for you. There are different kinds of products: coding agents, testing agents, documentation agents, code review agents, and migration documentation agents. Right now I've only been exploring the coding agents, but hopefully I can get to the other ones soon. You can do a test drive and sign up for free — you don't have to buy it or anything, you can just try it out. The one I think is the most awesome so far (because I'm a DataWeave coder at heart) is the DataWeave Generator. It's pretty much like you give it a sample input and a sample output and it will try to generate the code for you. I've been trying it out for a few days and it's really cool. I only tried the no repository option, but there's also one with repository. You can also upload from your computer or use a mapping table, which is cool — apparently it's an Excel file that you can use. ### Understanding the DataWeave Generator One of the things I noticed right away is that the sample input format is limited: you have XML, YAML, JSON, CSV, and EDI, which maybe are the most basic ones. I use text a lot, but it makes sense it doesn't accept text because that would be more complex. So let's start with JSON. To test it out, I wanted to go back to my DataWeave programming challenges series. I thought Challenge #4 would be cool because there are actually two different ways of solving this one, so I'm kind of curious what it will do. The other cool thing about CurieTech is you can generate different input and output samples — not just one — so it can kind of look at different examples of how it works. And it reminded me of this challenge because I actually added three different scenarios of it. ### The Hanoi Tower challenge The challenge is pretty much you have to solve the Tower of Hanoi kind of thing. You have three discs, you have to move them around to get from one point to another, but you can only put a smaller disc on top of a bigger disc. Here are the three scenarios for the input. In the first scenario we have three disks and they're all in tower A: 1, 2, 3. We want them to be moved to tower C. In the second one, we have four discs. They're all in tower A: 1, 2, 3, 4. And we want to move them to tower C also. Scenario three adds another one — now we have seven discs in the target tower Y. So in the first two we had A, B, C; now we have X, Y, Z. It will start in tower Z from 1 to 7 and we want to move them to tower Y. I thought this was a really nice one. Let me just put the output first. Sample one has to end up with seven moves — still three discs — and all three discs are in tower C. Oh, it removed the other scenarios when I added the next one. That sucks. If the product team is watching, maybe don't remove the scenarios just yet because someone created this article where you have the scenarios separated by input and output. Now I just have to go back and forth copying and pasting one by one. Scenario two ends up with the four discs in tower C, and the third one starts in tower Z and ends up in tower Y. I also have here the link for the DataWeave Playground. If you click on this, you can open the DataWeave Playground and move around the things so you can play around with it. Right now this is not solved — the idea is that you solve it and this is the output you have to get. We'll use this to test it out. Let me add some notes — let's give the explanation here and then the rules. Submit. And let's see what it does. ### Testing CurieTech's solution I've been playing around with the DataWeave generator before, but I hadn't actually created an account. This is the first account I created. I don't know if you saw my previous videos about using Cursor, but I kind of came here to try to use a code enhancer to see if it would apply best practices to my project automatically. It didn't, but maybe it's because I set it up wrong or something. But I would be very curious to try out the best practices thing with CurieTech. For now I'm just trying the DataWeave Generator because it's a really cool product that I really like. It is taking a while to answer, but the previous ones that I did did not take that long. But anyway, what I was telling you is that this is a cool challenge because if we go and see what Felix did here, this is like the larger solution that he did — we have a few lines of code — but then he went ahead and created a super short solution which is only nine lines of code. Okay, it generated it. It took 1 minute 39 seconds. But first let me try Felix's one. Felix is a DataWeave guru. So this is Felix's solution and the moves end up being seven as we wanted. Let me try the biggest one to see — I put here it was 127, but I'm not sure if that's the case. So this is the input of the largest one and the output was supposed to be 127. When we put this, Felix actually ends up with 127 with only nine lines of code. And that's why I thought this was a nice one — Felix was able to kind of cheat on it and just do a math operation instead of actually moving the things around with a recursive function or something. But that's fair. I mean, it works. It actually does the thing. So what did CurieTech do? It's still a few lines of code. It's not that much. First let's put it here. And yeah, we still have 127. So it feels like it does work. Now let's take a look at the code and see what it did. It just has the output application JSON and then it says calculate minimum moves using the formula 2 to the power of n minus one. Isn't that what Felix did? Yeah: two to the power of discs minus one. This is awesome. So it actually figured out how to do it. But Felix is just creating an update operator there and this DataWeave is just doing the mapping directly. Let me remove the comments just to see how many lines there are without the comments. Then preserve the number of disks. Yeah, this is fine. Felix is just not touching the discs. Then we have the target tower which is just a one-to-one mapping from target tower payload. Then we have the tower state. So this is doing payload towers map object. For each tower, check if it's the target tower. I feel like this is not necessary. It's putting out the towers and checking the key. If key as string equals payload target tower as string, then we're moving the discs. So: if target tower, fill with sequence one to n. Else if not target tower, empty array. That is fair. That is one way to do it. Definitely. What Felix did here is just take the start tower, put an empty array, and then if this is the target tower, you're just taking the towers from the start tower and moving them to the target tower. So this DataWeave instead of getting the towers from the start tower to the target tower, what it's doing is just generating from one to whatever number (in this case seven). I mean it works. I don't know if it's the most performant way to do it. It seems like Felix's one is way more performant. But it works. It does work. So cool. That was one test. ### Testing Challenge #7 All right, now let's try this one. Challenge #7: modify certain values from a JSON structure. In this one I think I only have one scenario. Yeah. But we can tell CurieTech how this works. So let's start another DataWeave thing: new DataWeave generator task. This is going to be a JSON. So input data is this and output data is this. All right, and then the explanation of the problem. Let's just copy and paste it. Create a DataWeave script that will update all the values to uppercase except the ones in which the field equals "thisName". For example, the first field name with the value "a" has to be transformed to "A". However, the field "thisName" with the value "def" should stay the same. Oh, I already know how to fix this. Well, yeah — I created the challenge. So we have A, B, C... there's a list, nothing here (this is just to bamboozle you), we have the DEF. DEF stays the same. This E will go uppercase, uppercase, uppercase, and lowercase, uppercase. Okay. So let's submit it. And in the meantime, let me close the other thing and open this in the DataWeave Playground. And let's see how much it takes for this one. This should be easier in my opinion, but we'll see. Okay, let's see. I feel like that was a lot of code. That was a lot of code. Oh my god. No. No. This is not — I'm not even going to try to understand that. No, that is not. I mean, does it work first of all? So we end up with A, B, C, DEF, E, F, J, H, I, J, K... and yeah, so this is the same. But I know for a fact there is an easier way to do this than doing this whole thing. So maybe for more complex examples this is not the best, but I can see kind of the mapping that it did and why it did it. Can I tell it this is not good or something? No, I can't. What's this library? No. Yeah, I cannot really tell it that this was not good code, but that's fine. Oh, there's a ladybug. ### Testing again without instructions Anyway, what if I try it again? I wonder — will it create a different thing? Let's see. JSON, input, output. And what if I don't give it any instructions? And I just love ladybugs. So this took 29 seconds and I think it generated a new code. Let me just put it here. "Transform value field name value any. If the value is a string and field name is not thisName, convert to uppercase." Fair. This is fair. "Function to transform objects recursively, handle different types of values." Yeah, this is better. This is 26 lines of code, but this is still not the best way to do it. So I wonder if there's a way to keep teaching it what to do. I don't know if this is reading the DataWeave modules and functions because I can just go into my answer here. Yeah, mapLeafValues. I used mapLeafValues — simple as that. Let me just remove all the empty lines: eight lines of code. So pretty much get the mapLeafValues function and then check if the path minus one dot selector (this means the name of the field from the value) equals "thisName". Then I'll put the value without modification, else put the value in uppercase. So that's as easy as that. Oh, also mapLeafValues — but using the dollar sign expression instead of using the value/path expression. So yeah, I don't know if CurieTech is just not reading the modules — for example, this is a tree module. Maybe that's it. But it would be a nice thing to teach it how to do it. ### Wrapping up and what's next All right, that is all for this video. In my next video I will go ahead and test one of the other agents here. I don't know — like the integration generation or the coding enhancer, or maybe we can try some of the other things. There's sample data generation, there's diagram generator, document generator. This is really cool. I will try some of this definitely, but in my next video. For now, I think this is really cool, but I kind of want to go back to my main focus, which was trying to get an agent to apply best practices to my MuleSoft project. So let's see if CurieTech can do it. We saw that Cursor can do it with rules if you tell it exactly what to do, which is nice — you don't have to teach it again, it can just go and read the rules. So if you didn't watch that with Cursor, just go ahead and check my previous video on my YouTube channel or on my playlist or on prostdev.com, and we'll see what else we can do with CurieTech. All right, see you in the next video. Bye. --- ## Is CurieTech AI Smart Enough to Pick the Fastest DataWeave Code? Watch: https://www.youtube.com/watch?v=J3A-3HhOAVk | Page: https://prostdev.com/video/curietech-ai-fastest-dataweave-code | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone. My name is Alex and today I'm going to continue testing CurieTech AI with some DataWeave stuff. I don't know if you're familiar with this post that I created back in 2024 — more than a year ago — in which I tried different ways to solve the same problem. We have this input, and I'm going to go ahead and put it in CurieTech. So here I'm in CurieTech AI. I go to DataWeave Generator. I have my JSON. This is my input data. And then my output data is going to be something like this in which we are going to change some fields from the payload to generate something different — but then we also want to filter out the objects in which the criteria is less than (let's say) three for this example. The expected output should be like that. Let me tell it: find the most performant way. And let's submit it. ### The 2024 experiment While we wait for that, let me show you the experiment that I did. I ran three different ways of achieving the same result, but here I tried doing it with 10,000 items instead of just two. The results were mind-blowing. We created filter-then-map — which is the fastest one — because you first filter the whole array and then you map it. But then we also created map-then-filter, which is the second most performant. So first it goes and does all the mapping to all of the objects and then it filters them out. Finally we also tried to use just reduce, which in theory should only be using one iteration of the whole array. In theory reduce should be faster because it's only going through the whole array once instead of two times or more than one. But that was not the case. As we can see from here, filter-and-map took like 55 milliseconds, map-and-filter took seven, and only-reduce took almost 1 millisecond. This keeps changing if you keep running the code — as you can see it's going to keep doing different times — but it always remains first filter-and-map, then map-and-filter, and then only-reduce. So here for example I have map-and-filter: first we're doing the map, then we're doing a filter. And then here, only-reduce: we are just doing the reduce and nothing else. And then in filter-and-map we are filtering first by the criteria and then we are doing the map. ### CurieTech's output All right, and it finished in 34 seconds. So let's take a look at the code that it did. It is telling us "the current year 2024 hardcoded to match the expected output." Yeah, because I did it in 2024. So first we have the filter: it is doing the filter first (criteria less than three), then the map, and then doing the whole mapping from whatever I had. This is pretty much what I was expecting. ### Regenerating with tweaked criteria And then let's say you made a mistake and instead of putting less than three you wanted to put equals zero. Let's do criteria should be less than two, I think, because what was the output? Oh yeah, the criteria was zero. So for example: criteria should be less than two instead of less than three. You can click on regenerate and this will again work on it. I don't know why it says 1 minute now. Turns out it says 1 minute because the time that is showing is a cumulative time from the point the task was started until now. So because I was taking one minute to do the whole thing and then do the second task, it took almost two minutes from that point to this point. But yeah, you can just take a look at what is happening here — here you can take a look at the comments. So here "criteria should be less than two instead of less than three" and it generated it. You just have to keep scrolling down and here you can see criteria less than two. Here's the new code, so we can actually test this. Let's go ahead and take the input that we had and then let's copy the code with the less than two. And here we have it, and it is working as we wanted it to work. ### Final thoughts Awesome. It was able to correctly create the filter first and the map second, so I'm happy with this. It actually followed the experiment that I did where I was able to see that filter-then-map was actually the most performant way. So if I had CurieTech back then, I would be able to see this way faster instead of having to manually do the whole experiment and timing them. Awesome. Thank you, CurieTech fam. I like this. The DataWeave agent is going places. All right, that is all for this video. I will see you in the next experiment I do with AI. Bye. --- ## Cursor AI vs CurieTech AI: Who Writes Better MUnit Tests for MuleSoft? Watch: https://www.youtube.com/watch?v=x0WgKgeH2kE | Page: https://prostdev.com/video/cursor-vs-curietech-munit-tests-mulesoft | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone, my name is Alex Martinez, and today I'm going to show you the differences between creating MUnit tests using CurieTech AI or Cursor AI. First let me show you the project we're working with. This is a main flow where we receive a queue, process some variables, do an until-successful, and call a flow reference inside that. If everything went well, it publishes to a second queue and acknowledges Q1. If there was an error, it propagates to an error handler that sends a NACK to Q1 so the message can be retried. Inside the `process-api-hr-flow` subflow, we have three different scenarios that can happen. First is the happy path: if the request arrives and everything is fine, it creates a successful response. If there's a connectivity error with the HR API call, it comes to the on-error-propagate handler and indicates it's a connection error — and because it's surrounded by until-successful, it will keep retrying until the retries are exhausted. The third scenario is if the error comes to the on-error-continue handler because, let's say, it was a bad request or something — not a connectivity error. It actually connected to the HR API but received an error back, so it will generate an error response. There are different things we have to test. We need to check that everything is processed in the main flow, that a connectivity error triggers the until-successful retries and eventually a NACK if it can't connect. If the call is successful — or if it's a bad request (non-connectivity error) — the flow publishes to the second queue and acknowledges Q1. We could approach this by creating separate tasks for each flow and see what happens. ### CurieTech AI — Task 1 (main flow) Let's start with the MUnit Test Generator in CurieTech. I'll upload our project, enable coverage, and select at most three MUnit scenarios. I have Java 17, Maven 3.9, and I'll select the main flow first. I'm going to submit that test. While that's happening, I can go back home, click MUnit Test Generator again, enable coverage, select again at most three scenarios — everything is the same except now I'll select the HR API call subflow. I'll submit that. If everything goes well, these two tasks will generate 100% coverage for my whole project. It's really cool because I didn't have to explain anything — I just selected the flows and hit submit. Let's see what it generates. For the first task, we already have an MUnit scenario generated. Looking at the details: - **Test scenario 1:** Happy path where a message is received from MQ, processed, and the message is acknowledged. - **Test scenario 2:** The HR API call fails with a connectivity error, triggering until-successful and eventually a NACK from the original queue — exactly what we expected. For the second task, we can see it generated MUnit scenarios: - **Test scenario 1:** Happy path where the HR API request is successful. - **Test scenario 2:** Throws a connectivity error and returns an error response. It's missing the third scenario where the HTTP request returns an error that's **not** a connectivity error, but we can fix that by adding notes. After 2 minutes, it generated my MUnit scenarios. Both tests passed and my coverage is 90%, which makes sense because it's not checking the third scenario — the non-connectivity error path — but that's okay; we'll fix it. The second task completed after 3 minutes 38 seconds. It generated the two tests and 27% coverage for the whole application (because this is just a subflow). We can see both test scenarios and the full XML. But now we know it's missing the three scenarios we need. That's totally fine — let's just upload the project again, enable coverage, select at most three scenarios, and now we'll add some notes. ### CurieTech AI — Task 2 (HR API call subflow with notes) Here are my notes: > Generate three different test case scenarios. One, happy path where the HR API call was successful. Two, connectivity error from the HR API call. Three, bad request error or any other error from the HR API call, which will result in the on-error-continue error handler and will create an error response. I selected the HR API call flow and submitted. While that runs, let's download the two tasks we know we want. I'll extract the generated zip file — everything is in there. I go to `src/test/munit` and copy the generated flows. Then I copy everything from `resources` into my project resources folder. Now I can run Maven clean test. I know I could technically use the testing part from Anypoint Code Builder, but I've been having issues with that — my problem, I don't know why — so I run the tests with Maven in the command line. Meanwhile, the third task is already generated. Looking at the scenarios: - **Scenario 1:** Happy path where the HR API call is successful and returns a valid response. - **Scenario 2:** HR API call fails due to a connectivity error, triggering on-error-propagate. - **Scenario 3:** HR API call fails due to a bad request or any other error, triggering on-error-continue and creating an error response. The behavior here is that it mocks the HTTP request connector to throw a non-connectivity error, like HTTP bad request or any other error — exactly what we wanted. Back in my repo, I can see the tests passing. The first task generated 90.91% coverage, just like CurieTech predicted. The third task took 6 minutes 13 seconds because it was generating three scenarios instead of two. The three scenarios all passed, and the coverage is 36%, which is expected. Now I'll download that code, extract it, and add it to my project — the flows go in `src/test/munit/flows`, the resources go in `resources`. Let's run Maven clean test again. Now we can see we have **100% application coverage**. Honestly, this was super easy to do. It only took me less than half an hour to generate 100% coverage for the whole thing. I only had to generate a second task once because it generated two tests instead of three, but that's fine — I just added notes for the next task and it generated exactly what I wanted. That's really cool. I give it 100 out of 100 and I would really recommend you try it out. I'll put this repo on my GitHub so you can take a look. ### Cursor AI — Attempt 1 (generic prompt) Now let's try a little bit of Cursor and see what happens. I'm going to discard all my changes. MUnit is empty, resources is empty. Let's start again from scratch. I'll open the main flow and let's see what Cursor can do. Let's start by asking something really generic and see if it can handle it: "Generate a unit test to have 100% coverage." It's trying to generate a new test file for the main flow for both success and error scenarios, but it's having some issues. It seems to be stuck, but to be fair I'll start the stopwatch — we gave CurieTech a few minutes, so let's see what happens. Oh, that was my bad — I was trying to start the stopwatch and clicked something. Okay, let's try this again. Send it again, click continue and revert. Now I'm going to add a stopwatch — starting now. Let's see what happens. It's also worth mentioning that I'm using the **business account** for Cursor now, so I'm not on the free account anymore. I actually have some power now. Let's see. It seems like it's trying to fix some errors. I don't see anything here, even though I see some code. Okay, I can see it's generating a flow. What? It's been 2 minutes. I'm going to stop this now. I am very confused. This is a flow. This is another flow. These are just flows. This is not going to run at all. Let me reject everything it did. ### Cursor AI — Attempt 2 (detailed prompt with mocking instructions) Alright, let's give it more detail. I'll tell it: > Generate a unit test to be able to cover the different test case scenarios from the Mule app. One, happy path where the HR API call was successful. Two, connectivity error from the HR API call resulting in a NACK. Three, bad request error or any other error from the HR API call which will generate an error response but won't result in a NACK — it will actually result in an ACK like the happy path. Make sure to mock all the external calls like the HR API call or the Anypoint MQ calls. I'm telling it to mock because Cursor is not specific to MuleSoft like CurieTech is — CurieTech just knows you need to mock things; you don't have to explain it. I'm starting a new stopwatch now. Alright. It's been 7 minutes. I even went for a coffee and everything. I'm going to stop it. So we have the happy path: it mocks a successful HR API response, mocks the Anypoint MQ publish operation, mocks the ACK operation. Connectivity error test: mocks an HTTP connectivity error, mocks the NACK, verifies the flow handled the error. Bad request error: mocks a bad request from the HR API, mocks the publish, mocks the ACK, and verifies behavior. It says: "Each test case sets up the initial event, mocks all the external calls, verifies the behavior. To run these tests, use Maven clean test." I already know how to run them. But I see **a lot of errors**. What is this? Why are there errors? There's just an `assertTrue`. What are you even testing here? Is this assert not null or something? And this is unknown, unknown. And what are you doing here? This is like a huge waste of my time. No — not using best practices. No. Let's just stop. ### Verdict Pretty much what we're seeing here is: CurieTech is generating the MUnits beautifully. It does take a few tries, but we're talking 6 minutes, 3 minutes, 2 minutes. It pretty much makes sense when you don't have to give a lot of direction. I only had to give direction to one task, and I only used like three sentences. I would definitely use CurieTech for my MUnits anytime. Cursor, on the other hand — I feel like CurieTech is like a senior developer that already knows what to do, and maybe you just have to give it some mild direction like "Okay, you missed this though," and they're going to be like "Oh yeah, you're right," and bang, here it is. But Cursor is like an intern, where you have to tell it exactly what to do. I mean, what are you even doing here? This is not even running. This doesn't even work. I'm not even going to try to run it because it clearly has a ton of issues. It needs a lot of direction. Maybe in the future Cursor is going to be ready for MUnit tests, but right now MUnit tests are so specific and I doubt that a lot of people use them — even MuleSoft developers. So there's not a lot of information for Cursor to see out there on the internet. So it makes sense that it's struggling with it. Cursor definitely cannot handle MUnit tests. This was just awful. I would advise you to just go to CurieTech. ### Closing thoughts Alright, you saw it. You can try it. I will put my repo with the tests from CurieTech out there if you want to check them out and verify that they are actually good. But just go ahead and try it — it's free. Just go to CurieTech.ai and try it out by yourself. Let me know if you make it work and what you did. I'm just really happy with CurieTech. We've seen DataWeave already, now we're seeing MUnit. This is just awesome. We'll see how it works for further tasks. Alright, that is all for this video. I will see you in the next video where we keep trying AI tools to make your life better. Bye! --- ## CurieTech AI Wrote My MuleSoft Best Practices — AND Opened a GitHub Pull Request! 😱 Watch: https://www.youtube.com/watch?v=vguu_u0OQNE | Page: https://prostdev.com/video/curietech-ai-mulesoft-best-practices-github-pr | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone. My name is Alex and today I am going to show you the Code Enhancer agent from CurieTech AI. From my previous videos we already did the DataWeave generator and the MUnit test generator. Now let's try the Code Enhancer. Before, I had usually been updating my project here from my computer, but now I have set up my GitHub repo and everything. So let's go ahead and select my Alexander Martinez test in the main branch and we can give a description of the task. ### Using MuleSoft best practices as the task description In this case, I am going to be using the MuleSoft best practices file that I had generated in one of the videos before for Cursor AI — because Cursor AI can use this file to make sure that it's using best practices for all of the projects. But for now, I'm just going to copy this whole file and I am going to use it here in CurieTech. So this is the description that I'm going to add — just all of the rules to make sure that it has best practices. And we're just going to click submit. I already ran this, so I am going to click on submit just so you can see what's happening and how it looks. It's going to tell you the task was submitted, this is what you added, and the task is going to be in progress. Once it's done, it's going to show you the differences. ### Reviewing the generated changes So I already ran it. Let me come here — diff generated. If I click on this to see the diff, we can see that from my project, it's just a hello world application. Really simple. But I am not following best practices. I have my HTTP listener configuration here in the same test.xml file. I have my properties like my port and my host right here. You can see them. And I don't have any properties file at all. So those are the main changes that I want — that are actually described here. You can see in the description of the video I have all the links so you can go check them out and read them or just pause and take a look at them. So what it generated was in the test file — in the test file we have the listener config that was removed because it was added to the global.xml, and here it was just some doc ID changes. That's fine. The pom.xml had just some new lines added. Not sure why but whatever. But here is a good thing: in the `src/main/mule/global.xml`, it has the global property `mule.env` set to `local`, which is what I described in my task. Then I have the configuration properties set to `env-properties.yaml`, which is also what I asked — it can be named different things but I selected that. And my listener configuration was moved from my test.xml to my global.xml. And the properties of host and port were actually taken and put in another file which is my `local-properties.yaml`. ### Properties file naming and YAML strings I also added in my rules that I want this file to be named `env-properties.yaml`. So this is what I selected. It can be named local.yaml or whatever you prefer. Just make sure to put it in the task. And the other thing that I also put in the task was that these properties have to be strings — because YAML is very specific with that, or MuleSoft, I don't know. But if you put here for example just `8081` without the quotes, then it's going to be an error because it's going to tell you that it doesn't accept numeric values, only strings. So that is why it's important to put quotes in values that can be numeric. Like if you had `ABC` for example it doesn't really matter because it's not a number, but `8081` can be interpreted as a number. So that is why it's important to have the strings. So it generated all of those things and then it asks you if you approve the changes or not. In my case I selected approve. ### The pull request (PR) magic So then it created the task completed right here. So if we open that, we can see that it actually went ahead and created a pull request (PR) to your GitHub repo — which is really, really, really cool and really convenient. That way you don't have to do anything else. So here you can take a look — the following, this is the description of the task that you added. In my case it was added based on the file that I had previously generated. Then it gives you a small summary of what it actually created — like the files that it added, the files that it modified, and the summary of the changes. So this is actually really convenient if you have the GitHub repo already there. Just tell CurieTech to create the PR with whatever changes you need. That's really cool because you also can keep track of all of the changes that have been happening on your repo. And then you can just merge it or comment or do a comparison or whatever. ### Pairing with the Code Review agent and GitHub Actions And the cool thing about that is if you come here, we also have a Code Review agent. So you can use it to review a pull request (PR) for the provided repo — which is also cool because they kind of complement each other. And if you really need it to run MUnit or make sure that the project runs before deploying or whatever, you can just set up your GitHub workflows — your GitHub Actions — and you can add them to any PR that is created. And that way, once CurieTech creates a PR for your repo, you can just run all the necessary validations that you need before actually accepting the pull request, which is great. ### Closing thoughts All right, that is all for this video. So again, CurieTech is not disappointing. CurieTech helped make sure that my project had best practices and it also created a pull request (PR) for me to actually go take a look at all of the changes. So this is really cool. Again, 10 out of 10 would recommend. So what are we going to do next with CurieTech? Let me know in the comments whatever you want to see or reach out to me and I will be happy to test it out for you. All right, that is all for this video. I will see you in the next video. Bye. --- ## Testing CurieTech AI's Code Insights: Single Repo, Multi Repo, and Code Review Lens Watch: https://www.youtube.com/watch?v=x1_aMbHr3w4 | Page: https://prostdev.com/video/curietech-ai-code-insights-single-multi-repo-review-lens | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone, my name is Alex and today we're going to see some code insights from CurieTech AI. I'm going to demo three modes: single repo code lens, code review lens, and multi-repo code lens. Let's dive in. ### Single repo code lens I've been trying out the Mac project and I created this repo. If you haven't seen my really detailed README yet, let's see what CurieTech can tell me about this. I've already set up my repos. If you go to your settings → repositories, you can connect your GitHub. I would recommend connecting via OAuth. Once you have that, you'll be able to use your own repos. So let's go to Coding Insights → Single Repo Code Lens and select my own repo. You can also upload from your computer; in my case I'm just going to select the Mac project from here. Let's ask, "What is this repo about?" It gave me a bunch of explanations. For example, key features: Agentforce integration. Here you can see the flows it's implementing in Agentforce — all the flows I have implemented. Then we have the configuration and setup, the MuleSoft configuration and properties, some documentation like the README and the OAuth example that I created for you to use in Agentforce, and it also gave me this nice summary table so I can see where everything is set up. Another cool thing: if you go to your settings → workspace and scroll down, you'll see coding guidelines. I have uploaded my own PDF file with the coding guidelines I want CurieTech to take a look at. You can upload yours, too. So let's go back and ask: "Are there any best practices or security violations I should be taking a look at?" It's asking me for a specific flow. I'm just going to select this flow. It gave me a bunch of stuff: - **Use of a hard-coded file path.** Yeah, I hard-coded that. I definitely need to change it. It even tells you why it's not recommended and how to fix it. - **Proper use of global configurations.** Thank you! I have all my global configs in `global.xml`. - **Properties management.** I am following the pattern — yay. - **Secure properties.** I definitely need to secure my properties. I have not used the `secure::` property placeholder for this, so yes, I need to do that. - **No error handling.** That is true. I have no error handling. - **No MUnit test for this flow.** Definitely not. So I also need to add my MUnits. And finally, it's letting me know: you need to review other flows, ensure that secure property setup is in place, and some other stuff I need to review. So that was the single repo code lens in Code Insights. I'll leave the multi-repo code lens for the end, so let's do the code review lens now. ### Code review lens I created this pull request (PR) where I am definitely not following best practices. For example, here you can see that I have my port and my host hard-coded instead of in the properties — I just changed them from 881 to 882 — and I'm also exposing some test Salesforce credentials. These do not exist, by the way. So let's copy the PR link, head to Code Review Lens, select the test branch, and ask a question. We can also edit the guidelines here. Let's just say "Review this PR." And yeah, it definitely told me everything: - **Review finding: security issues.** Remove hard-coded credentials and use `secure::` property placeholders. Salesforce username and password — definitely. - **Create a `global.xml` file** with configuration properties. Yes. - **Update the HTTP listener configuration.** Yes. - **Add to the properties YAML.** Yes. - **Missing proper configuration properties.** Yes, because I have no properties at all. - **Missing configuration properties module.** Yes, because I have no properties at all. So yes, it was definitely able to find everything. ### Multi-repo code lens Now, the third and final way to use this is the multi-repo code lens. For those of you that don't know what Anypoint Race or Anypoint Speedway is: Ryan Heck, someone in the community, created this integration challenge where you can create your own Anypoint race cars for racing on the speedway. Anyway, for season 1 and 2, I believe, I had to create two different repos. This is the whole diagram of how everything works. But in summary, I have the **race client** (which is this repo) and I have the **racer API** (which is this repo). I'm going to take those two repos and feed them into CurieTech to see what it does. So in Multi-Repo Code Lens, let's select the branch (main), select the repos — Anypoint racer API and Anypoint race API MuleSoft — and ask a question: "Can you tell me how these repos work together?" If we take a closer look at the diagram, they never actually talk to each other directly. The race client is talking to the endpoint race API which was managed by Ryan, and that one is orchestrating and managing all of the calls to both of these projects. But what they have in common is this **token queue**. Every time the racer API would create a token, it would publish the token to the token queue, and then the race client would consume the token from the token queue. So this queue, in theory, is the same. But because we don't have the Anypoint race API from Ryan, we can't really see all of these connections, I think. And here we have it: **Anypoint racer API** is the message publisher. Yes, as we saw, this is a message publisher, and we can see here where this is implemented and how this is being published. The queue destination is defined by the property `anypoint.mq`, and the MQ configuration is in `global.xml`. Then we have the **Anypoint race API MuleSoft** (which is the race client) consuming messages from the same Anypoint MQ. This is implemented here: the subscriber listens to the same queue `anypoint.mq` and processes incoming messages, triggering further actions such as HTTP requests and logging. So, summary table: the racer API is the Anypoint MQ publisher, and the race API MuleSoft is the Anypoint MQ subscriber. That is really cool. ### Recap and my thoughts So yeah, in summary, if you go to the Code Insights part, you have: 1. **Single repo code lens** — where you can get a repo or your project and ask any questions you may have. Like, how does this work? Is it following best practices? Are there any security violations? If you're having a bug and you need to figure out why, you can ask any kind of question regarding a single repo. You can even ask if your code is being efficient or if there is a way you can rewrite it so it is better. 2. **Code review lens** — where you can actually tell it to review a pull request from the repo that you provided. As we saw, you can upload your own file to follow best practices, naming conventions, or whatever you want. Kind of like the rules file that I had created for Cursor. You can also check if a PR is going to inject a bug or something, and it can check against the main code to make sure it's still working as expected. 3. **Multi-repo code lens** — I used it with some of the projects I had that are actually connecting through a queue. They're not even connected like an HTTP call or doing some sort of experience/process/system layer exchange. I feel like this is actually more complex because it wouldn't know that they're connecting to the same place through a queue. So I thought this was a really nice use case to see it in action, and it actually worked. Using this multi-repo code lens, you can also see the overall architecture of how your projects are being set up — how it's working. If you had experience, process, and system layers, you could ask it, "How are these functioning? Maybe you can give me a diagram of some sort?" So yeah, that is how you can use the Code Insights. So far, in the last videos that I've done about CurieTech, we've been seeing the coding agents, the data generator, the code enhancer, and then in testing we saw the MUnit generator. We haven't seen any of the documentation agents, but we have now seen all of the Code Insights. We also have not seen the migration agents. So let me know in the comments below what other agent from CurieTech should I test next. Let me know what you think. If you want me to do some specific experiments, I'm here for you — I am all up for it. Or let me know if you have some other tool that you want me to try out. But so far, CurieTech is definitely earning the number one prize for MuleSoft development. All right, see you in the next video. Bye. --- ## Generate OAS & RAML Specifications Instantly with CurieTech AI’s API Spec Generator Agent Watch: https://www.youtube.com/watch?v=U8KChXTgABw | Page: https://prostdev.com/video/generate-oas-raml-specs-curietech-ai-api-spec-generator | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone, my name is Alex and today we're going to see something really interesting. CurieTech just released a new API Spec Generator agent a few days ago, and I am more than excited to try it out. Let's do this. ### Converting RAML to OAS I already thought of a use case. I have this RAML specification that I've been using for other things — my to-do API specification: create to-dos, delete, edit, and so on. Because I already have it in RAML, I was thinking let's move it to OAS. I select OAS 3 and ask a question. You can directly ask any question you want to this agent and it will generate an API spec for you. In my case I'm just going to go ahead and select this whole RAML and tell it to take this RAML spec and create an OAS YAML spec. I paste it and press enter. This is the result: "This OAS YAML specification is valid and ready to use in your MuleSoft project." Let's see if it's true. I'm going to copy it. I already have my RAML specification open so I can compare them. In this I have `/todos` GET, POST, `/todos/{id}` GET, PUT, DELETE. Then I paste the new OAS version. I can see that I have my `/todos` GET, POST, `/todos/{id}` GET, PUT, DELETE — it looks like it has everything the other one has. `/todos` GET has four query parameters, which are the same that I have in the other one. Then we have the POST to create a to-do and then we have `/todos/{id}` to get, update, and delete a to-do. So it pretty much took the same specification I had in RAML and just put it out in OAS. We have a check on that — it really worked. ### Generating a new spec with full detail Now let's try to actually create a new specification from scratch. "Create an API to manage a list of to-do items. Each to-do has the following fields: ID (unique identifier), title (the title or description), checked (if the task is completed), due date (the due date of the task). When it retrieves a list of to-do items it should support optional query parameters for filtering like title contains, is checked, from date, to date." I'm giving it a ton of information right now because I want to make a test with it first. Let's see what happens. And this is what it generated. It has pretty much the same structure as before, but it does look a little bit different because it has some different things. For example, it has a 404 description like "Item not found," which I didn't have before. So it did add a few more things. And also if you take a look here at the schema, it changed the name to `TodoItem` — it didn't used to be called that. We have `TodoItem`, `TodoItemCreate`, and `TodoItemUpdate` — three different items, which makes a lot of sense because they're different things. ### Testing with less technical detail Now let's give it less technical information to see if it can still create something. I did go to ChatGPT to generate this whole thing, but here we go. It's pretty much just a paragraph explaining what it does: it updates tasks, it adds tasks, it deletes tasks. "My to-do API is a simple digital system designed to help people manage their tasks and to-do lists. It allows users to keep track of tasks, whether they're personal goals, work assignments, or daily reminders. The system lets users create new tasks, view their list of tasks, update them, and remove them." Let's try with this whole thing and see if it works. Let me see if I give it even less detail to see what questions it will ask me. Something really simple: "Generate an API spec to manage to-do tasks." Yes, now it is asking me for a few clarifying questions to generate a better spec. **Number one:** Do you prefer the specification in YAML or JSON? YAML. **Number two:** What operations do you want to support? All CRUD operations. Again, I'm trying to give it as little detail as possible to see what it comes up with. **Number three:** What fields should a to-do task have? It's suggesting ID, title, description, due date, and status. So let's just take that and put that. I didn't answer four and five because I don't know — whatever it wants to make. So let's take this code and check it out. ### Reviewing the generated spec Oh, this is awesome. We still end up with `/tasks` GET, POST, and then `/tasks/{taskId}` GET, PUT, DELETE. That makes me feel better with myself because the API that I designed first had those five methods. So the fact that CurieTech keeps creating the same five methods makes me feel better. We have `/tasks` GET to get all tasks — this is an array of items of a task. We have POST to create a task, so it will send a `TaskInput` and it would return a `Task`. Then we have `/tasks/{taskId}`. To GET it will be getting a task. For PUT, to update a task by ID, you send a `TaskInput` and you receive a `Task`. Then for DELETE, you simply send the task by ID. Now if we take a look at the schemas, we have the object `Task`: ID, title, description, due date, and status. Oh, this is even better than mine. And we can see which ones are required, which is ID, title, and status. Description and due date are not required. Finally, we have `TaskInput`, which is title, description, due date, and status — exactly the same thing as before except the ID. That makes a lot of sense because when you are going to create a task you don't really need the ID — you want the backend to create the ID for you. So this makes a lot of sense. ### Why this is better than ChatGPT This is perfect. It created exactly what I wanted and even better than what I had at the beginning. This is awesome. I am going to use this so often. Before, I was trying to do everything with the UI. If you don't know the UI, you come here, create a new API spec, guide me through it, and then you have this nice UI where you can just press buttons and configure stuff and you will get either the RAML or the OpenAPI spec depending on what you want. I used to use that because I like having a UI. Then AI came and I started using ChatGPT to tell it to generate stuff for me, but I would have to go back and forth to check in API Designer to make sure that it is actually a running RAML specification. It was pretty annoying to be honest. But now, I gave the same amount of detail that I would give ChatGPT, and it generated it so quickly. And I like that it doesn't assume what you want as opposed to ChatGPT — it asks you for some questions on what you want exactly. For example, I didn't answer all of them. I didn't answer number four or number five because it didn't really matter to me at this point. I just wanted to see the specification. So I answered with just those three numbers and it was able to generate the whole specification for me. If I want to keep adjusting stuff, I can just keep talking to it and it will just keep sending me more and more specifications. And because you have this handy copy button at the top, you can just copy this and put it in your API Designer. You don't have to drag everything and do something complicated — no, you just have the copy button. So pretty handy. And also as we saw in the first task that we did, we can also just use this to translate from RAML to OAS or the other way around, which is really cool too. ### Closing thoughts 10 out of 10. Again, awesome product. Awesome new agent. I love CurieTech. All right, that is all for this video. Tell me which of these agents have you used, do you want to use, or tell me if I should try any other agent from this list — please let me know. I am more than happy to try them out for you. I will see you in the next video. Bye. --- ## Why You Should Start Using Anypoint Code Builder (Even If You Love Studio) Watch: https://www.youtube.com/watch?v=dlODB4cjXyU | Page: https://prostdev.com/video/why-start-using-anypoint-code-builder | Series: Learn Anypoint Code Builder (ACB) ### Why switch from Studio to ACB I'll go straight to the point. If you're still using Studio, you're not alone. But here's the thing: MuleSoft is clearly shifting toward Anypoint Code Builder, and it's already unlocking tools and features you just can't get in Studio. In this video I'm going to show you why I made the switch, what it has helped me do, and how you can try it out without fully committing just yet. ACB isn't just a different interface — it's a whole new developer workflow built on Visual Studio Code. It integrates with tools like Cursor AI and GitHub Copilot. And because it's on VS Code, you can use source control or install other extensions like I have. Plus, new MuleSoft features like Agentforce topics are only available in ACB. Imagine what else will be available in ACB and not in Studio in the future. Look, I'm not saying ditch Studio right now. It's still the default for many teams, and ACB isn't fully loaded for enterprise workflows yet. But you should absolutely start playing with it. That way, when your team does move, you're already way ahead of them. Take a look at this poll. Only 10% of the people in it (130 votes) are using ACB daily. The rest are on and off, tried it once or twice, or haven't touched it. Statistics don't lie — people aren't really using ACB right now, but this will change eventually. Remember when we had Mule 3 and were switching to Mule 4, and Mule 4 was this scary new thing? That's life. We try the new stuff, and eventually it's the only stuff available. So why not try it now? ### Installing the tools (Git + VS Code + the Anypoint Extension Pack) First, install Git. Search for "git download" or go to the [downloads page](https://git-scm.com/downloads). It'll set you up for Mac, Windows, or Linux. Once you have Git, download [Visual Studio Code](https://code.visualstudio.com/). Again, you can download it for Mac, Windows, or Linux. Now go to VS Code, head to the Extensions tab, look for the **Anypoint Extension Pack**, and install the whole thing. Do not, under any circumstances, install each extension one by one. You have to install the whole pack. If you already installed them individually, uninstall all of them, quit VS Code completely, reopen it, then install the Anypoint Extension Pack. If you're having any issues — and I've seen a lot of people having issues because they installed the others first — do the same: uninstall all the extensions, quit VS Code (even restart your computer if you need to), make sure you have Git and the latest version of VS Code, and only then install the Anypoint Extension Pack. Once it's installed, you'll see the MuleSoft icon in the activity bar. Everything else is plain VS Code — if you're new to it, this is a whole new environment, so go check out some tutorials on getting used to VS Code first. Then come back and start trying ACB once you already know how VS Code works. ### Getting started — design, develop, or follow a tutorial Start with something small. Click **Design an API** and begin with your API specification. Maybe don't use the Agent topics just yet since you're getting started. Add a project name and project location (in my case I always use Downloads). Pick your API type — REST or async, whatever you're comfortable with — and your specification language (RAML, OAS, whatever). Select your business group only if you're logged in; if you're not, that's fine, just don't select anything. Click **Create project** and you've started with something familiar. If you want to see code but don't want to develop anything yet, go to **Develop an integration**. You can select a template or example project. There should be a Hello Mule or Hello World somewhere, but you can use any example — for instance, implementing a custom API policy in API Manager. Pick one you're familiar with, or open it first in Studio so you can see the code there, then come here and click **Add asset** to follow along in ACB. That way you don't feel lost, because it is a new thing, and we humans are naturally afraid of change. You can also check out my videos — I have a ton on ACB now, and I practically don't use Studio anymore. ### Using Cursor instead of VS Code If you'd rather go straight to Cursor instead of VS Code, you can. Cursor is technically another IDE (another application you install). Go to [cursor.com](https://cursor.com), download the IDE, and install it. You still have to install Git and the other tools. The Cursor UI is a tiny bit different from VS Code. So if you don't want to switch twice — learning VS Code, then redoing everything in Cursor later — maybe just start with Cursor now. You can still do ACB in Cursor. When you open Cursor, it looks like you can only open a project or clone a repo, but that's not true. Look at the top: you can open the Extensions view just like in VS Code, search for the **Anypoint Extension Pack**, and install it (same rule — install the whole pack, not one by one). One difference: the activity bar that's on the left in VS Code is at the top in Cursor. You can pin items to it. You can also show/hide the side view, the terminal panel, and the Cursor chat — which is the one I use most. Getting used to the Cursor chat is its own thing: you can add context, use agents, attach images, use ⌘L to pull context from a file, and so on. Honestly, I feel Cursor is better than GitHub Copilot — don't ask me why, it's just a feeling. You also have Cursor settings for rules, MCP servers, and more. And in the command palette (⇧⌘P on Mac) you can search for themes and change them — you can even create your own. This works the same in VS Code and Cursor because it's the same underlying editor. ### Wrap-up In summary, ACB is where MuleSoft is headed, and early adopters will shape how we build integrations tomorrow. You don't have to switch completely today, but you should start getting your hands dirty now. I'm here to help. If you have issues and you've already tried my troubleshooting tips, leave a comment and I'll help as best I can. You can also reach out on LinkedIn. Remember, not all Studio features are available in ACB just yet — this is just to get you started. Let me know if you need a tutorial specific to ACB; I'm happy to create it so you can make the switch more easily. That's all for this video — I'll see you in the next one. Bye! --- ## Getting Started with MuleSoft Anypoint Code Builder (ACB) in VS Code | Beginner Setup + Hello World Watch: https://www.youtube.com/watch?v=GgCUNWGDaS4 | Page: https://prostdev.com/video/getting-started-acb-vscode-hello-world | Series: Learn Anypoint Code Builder (ACB) ### Intro If you're new to MuleSoft or switching from Anypoint Studio, Anypoint Code Builder might feel like a big shift. In this video I'll walk you through everything you need to get started with ACB locally in VS Code — from installing the right tools to building your first Hello World integration and finally deploying it to CloudHub. I'll show you how to avoid common setup mistakes, explain how the ACB extension works inside VS Code, and help you get everything running smoothly. ### Installation (Git + VS Code + Anypoint Extension Pack) First, install [Git](https://git-scm.com/downloads) for your operating system. Then download [Visual Studio Code](https://code.visualstudio.com/), again for whatever OS you're on. Once you have those two — and only when you have those two — make sure you're on the latest version of VS Code; if not, uninstall and reinstall. Then go to the Extensions tab, look for the **Anypoint Extension Pack**, and install the whole thing. Do not install the extensions one by one. If you already did, remove them all first, then quit VS Code. It's very important that you fully restart VS Code — you can't just uninstall the individual extensions and reinstall the pack without quitting first. ### Get comfortable with the command palette and themes Once everything is installed, get used to the command palette: ⇧⌘P on Mac, ⇧Ctrl P on Windows. This is your best friend. You can search for MuleSoft, DataWeave, and a lot more — options come from your installed extensions or from VS Code itself. Try something simple like **Preferences: Color Theme**. Pick a theme you like, light or dark. You can browse additional color themes from the marketplace, too. In my case I use a MuleSoft Community theme. Finding a theme you like is your first task of the day. ### Opening projects vs. creating them in ACB In Studio you import a project or open a folder. In VS Code you can open a folder, clone a repo, or create a Java project. You'd use the folder/clone options to open existing work. But to create a new MuleSoft project, go to the MuleSoft tab (Anypoint Code Builder), where you have three quick actions: **Design an API**, **Implement an API**, or **Develop an integration**. A common point of confusion: I don't believe you can import a JAR into ACB just yet — you have to open a folder. (Update: it turns out you can import a JAR — see the dedicated correction video.) If you click **Open folder**, you can only open the root of a project (where the `src` folder, the POM, and the mule-artifact live). Note that "Open folder" comes from the Explorer tab, which is plain VS Code, not ACB. From ACB itself you use the quick actions, plus Help and Tutorials. ### VS Code extensions are a superpower Another great thing about VS Code: you can install any extension from the marketplace — Docker, Kubernetes, Postman, a database viewer, GitHub Actions, Kafka, Serverless, Markdown and XML helpers, and so on. None of that is ACB-specific, but it's very handy. For example, I can view my MySQL database, see my running Docker containers, and run Postman requests, all from inside the editor. ### ACB settings, login, and source control Click the Mule button, then **Settings** to open the ACB-specific settings: autocompletion, trigger delay, default versions, and more. Under **Edit defaults** (via the command palette) you can set the default Mule runtime and Java version that get pre-selected whenever you create a project — I always keep the latest. You can also set a default group ID, and add VM arguments here (for example, `-M-Danypoint.encryption.secureKey=1234` to pass values to the runtime). You can also set the control plane and the home directory, where things like the Mule runtime and Java version get installed. Note: **you do not have to download Java** — ACB downloads it for you. And in the future ACB will download Git for you too, so you won't have to install it yourself. (If you're watching from the future and Git is already bundled, great — check the comments.) Then there's the **Not logged in to Anypoint Platform** prompt; you can sign in from there or from the profile icon. For now I won't sign in — I'll show that later. There's also GitHub Copilot if you want it; I don't normally use it, but you're free to. VS Code also has a **Source Control** tab. Once you have a folder, you can initialize a repo, push, pull, commit, cherry-pick, and more — all from the UI. No more Git commands in the terminal. ### Building the Hello World flow Let's build a Hello World. In the ACB tab click **Develop an integration**. Remember the three options: Design an API is for designing a RAML/OAS spec; Implement an API is when you already have a spec (it goes to Exchange to retrieve it); Develop an integration creates an empty project or pulls a template/example from Exchange. We'll create an empty project. Name it `hello-world`, pick a location (Downloads for me), and select your Mule runtime. By default I have the latest. Because I don't already have this runtime on my machine, it'll download automatically on project creation. For the Java version I only have 17 right now; if I changed the runtime I might be able to pick Java 11. Click **Create project**. Once it's created, you can adjust, hide, or reopen the windows (the UI canvas and the code view). There's also Agentforce, which we'll skip for now. You can choose what shows in the Explorer — Open Editors, Outline, Java Projects, Maven, and so on. I normally only keep the Folders view; right-click to uncheck the rest. You can show/hide the panels and terminal with the buttons at the top, and use ⌘−/⌘+ (or Ctrl−/Ctrl+) to zoom in and out. At the top of the canvas you have **Add** (to add flows, subflows, and error handlers) and the **flow list** (ACB shows one flow at a time, unlike Studio). Click **Build a flow** to create a flow with a name. Then click the **+** — this is your new Mule palette. Instead of dragging and dropping like in Studio, you click + and pick what you want. You see the components you have installed, or you can pull connectors from Exchange. Go to Connectors → HTTP → **Listener** (or just search "HTTP" / "listener"). Notice it shows which versions are compatible with your runtime and Java (for me, runtime 4.9.5 and Java 17). You can double-check versions by right-clicking the XML and choosing **Project properties** — that view also has options like export to deployable JAR, export shareable JAR, and publish. Select the listener, then add a global configuration: click **Add**, set the host and port, and save. (Don't add properties yet — we're just learning how ACB works.) Set the path to something like `/hello` and press Enter (or click to save) — important, or it won't save. You can rename the flow from the canvas or the XML. Right now there's an error because the flow isn't complete — you can see it in the **Problems** panel ("content of element flow is not complete"). So add a **Transform Message**. Use an inline script with `output text/plain` and literally `"hello world"`. Then add a **Logger** — click the FX button to open the expression builder and log `payload` at level INFO. To keep the XML tidy, open the command palette and run **Format Document** (make sure you're on the XML file). Then save with ⌘S / Ctrl S — watch the dot in the tab turn back into an X. Always save, because it won't autosave. You can reopen the global config by clicking **Test connection** or right-clicking and choosing **Configure component in UI**. ### Running and debugging locally Head to the **Run and Debug** tab in VS Code. You should already have a "Debug Mule application" configuration; if not, click **Open launch.json** to see it. Press **Run / Start Debugging**. The logs show it was deployed. You can check Problems and Output (Mule Maven output, etc.). To test it, I use an extension called **Postcode** — a simple REST client, like Postman but without needing to log in. Send a GET to `localhost:8081/hello` and you get **hello world** back. Now let's debug. You can right-click in the flow to add a breakpoint (it adds it in the XML too), or add it directly in the XML. Since I'm already debugging, I just resend the request in Postcode and it hits the breakpoint. You can see breakpoints, the call stack, and add watches — for example watch `payload`. You can also inspect the Mule message attributes: listener path, raw request, relative path, method, scheme, and so on. Use **Step Into** to move to the Logger; now the watch shows `hello world` with media type text/plain. Press **Continue** to exit, or disable the breakpoint and resend to get an immediate response. That's how you deploy and try your app locally without ever leaving VS Code. The next step would be to deploy to CloudHub (you press the deploy button, log in, and continue), but I'll keep this video simpler and cover that in a future one. ### Wrap-up You've got Anypoint Code Builder running locally in VS Code and built your first Hello World integration. In the next video we'll go one step further and expose a proper REST API using API specifications and flow logic. If this helped, hit like, subscribe for more weekly MuleSoft tutorials, and drop your questions in the comments — I always read them. And if you get stuck, I've got ACB office hours you can book (link in the description). I'll see you in the next video. --- ## How to Design a REST API in Anypoint Code Builder (OpenAPI + VS Code + Exchange) (Part 1) Watch: https://www.youtube.com/watch?v=cki1MOLC1CE | Page: https://prostdev.com/video/design-rest-api-acb-openapi | Series: Learn Anypoint Code Builder (ACB) ### Intro Let's design a full to-do list API in Anypoint Code Builder. No flows, no integrations yet — just a clean, reusable API spec that you can mock, test, and publish. This is the foundation of any good MuleSoft project, and if you're new to ACB or switching from Studio, it's a great place to start. ### Creating the API specification From the ACB welcome screen, click **Design an API**. You can create an API specification or an API fragment — we'll pick API specification. We won't use Agent topics for this one; let's stick to the basics. Name it something like `todo-task-api`. API type is REST (not async). The specification language will be **OpenAPI 3 (YAML)**. If you're signed in you can select a business group; we can skip it. When you're not logged in you don't even get that option, which is totally fine — we'll log in later. Click **Create project** and it opens a new project with a basic API file where we'll define our endpoints. ### Basic info Start by updating the basic info: the title, the version, and a description. Change the title to something like **To-Do Task Management API** and add a description. This description is used when publishing to Exchange, so keep it clear and helpful. ### The /tasks resource (GET and POST) Add the first resource, `tasks`, with two methods: a **GET** to list all tasks and a **POST** to create one. Give each a summary and a description. For the GET, we'll use query parameters to allow filtering. Under `parameters`, specify where the parameter lives — in this case the query. The first parameter is `completed`, of type boolean, with a description of what it filters. Add a second query parameter for `dueDate`. You can add more filters; we'll leave it there. Tip: use Ctrl+Space to see options based on your current indentation. Add a `200` response with a description and `application/json` content, and define a schema of type array whose items we'll define in a moment. For the POST, add a **request body**. It has to be `required`, otherwise we can't know what we're posting. Content is `application/json`, and we'll reference the schema in a moment. Then add a `201` response — "task created successfully" — also with `application/json` content and a schema reference we'll fill in later. Notice the breadcrumb at the top shows which level you're at, so you don't have to scroll up to see where you are. ### The /tasks/{taskId} resource (GET, PUT, DELETE) Now define the second resource, inside `tasks` → `{taskId}`, so we can filter by one specific ID. It will have a **GET**, a **PUT**, and a **DELETE**, with summaries like "get a task by ID", "update a task", and "delete a task". Add the URI parameter once, at the top, instead of on each method. With Ctrl+Space, set `in: path`, `name: taskId` (matching the path), `required: true`, and a schema of type string. For the GET, add a `200` response (description + `application/json` + schema ref, to be filled in). Because we use a URI parameter, also add a `404` with description "task not found". For the PUT, add a required request body (`application/json`, schema ref later) and two responses: `200` and `404` (copy the 404; give the 200 a description and schema ref). For the DELETE, two responses: a `204` and a `404`. The 404 is "task not found"; the 204 is "task deleted successfully" — we return 204 because the operation succeeded but there's no body to return. ### Data models — Task and TaskInput Now define the data models. Under `components` → `schemas`, the first schema is **Task**, of type object, with properties like `id` (string, example `"1"`), `title` (string, example "buy groceries"), `description`, `dueDate`, and `completed` (with a default of `false`, since a new task hasn't been completed). The required fields are `id`, `title`, and `completed` — we don't always need a description or due date, but we always need an ID, a title, and the completed state. Because we don't have an ID when **creating** a task, we can't make `id` required there. So create a second schema, **TaskInput**, with the same properties minus `id`. There are cleaner ways to avoid repeating yourself, but we'll keep it simple for this example. Now wire the schemas into the references. When getting a task we already know the ID, so use **Task**. When creating a task we don't have one, so use **TaskInput** in the request — and we respond with **Task** (the result that comes back with an ID). For everything under `{taskId}`, we already have the ID in the URI, so respond with **Task**. ### Testing in the API Console Now that the spec is written, let's test it without writing any Mule flows. Save with ⌘S / Ctrl S, then click **API Console**. This shows the full documentation: the endpoints (`tasks` GET/POST and `tasks/{taskId}` GET/PUT/DELETE), code examples, query parameters like `completed` and `dueDate`, and the shape of each response. For methods with more than one response — like GET by ID — you can see both the `200` and the `404`. Click **Try it** to get a feel for it. It's a mock, so when you send a request it returns the example you defined regardless of the input. For example, sending `taskId` 123 still returns the example ID 1. The query-parameter section lets you show/hide optional parameters like `completed` and `dueDate`. The DELETE returns a `204` with no content. And the console validates: send something that isn't a boolean and you get a bad request telling you it expected a boolean but found an integer (or a string). Try sending a request without a required `taskId` and you get a bad request. Great for quick validation. There's also API Governance, which is a more advanced topic for a future video. ### Publishing to Anypoint Exchange Final step: make the spec reusable across your team by publishing it to **Anypoint Exchange**. You need to be logged in. Either click **Not logged in to Anypoint Platform** to sign in, or click **Publish API project to Exchange**, or use the command palette (⇧⌘P / ⇧Ctrl P) and search "Exchange". It prompts you to log in — click **Allow** and follow the flow. Once logged in, select a business group. You may see a warning suggesting you add rulesets and run validation; for now we'll keep it simple. Add an API version like `v1`, set the business group, optionally review the project metadata, and click **Publish**. When it asks whether to implement it now, select **No** — we'll do that later. Now from Anypoint Platform → Exchange, you can see the **To-Do Task API** published, with the same endpoints we saw in the API Console, which you can try out there too. Anyone in your org can now discover and reuse this API, or scaffold flows from it later. ### Wrap-up That's how you design, mock/test, and publish a complete OpenAPI specification in Anypoint Code Builder — no coding yet, but you're already ahead of 90% of Mule devs still starting with XML. In the next video we'll scaffold the flows from this exact spec and implement it step by step, so make sure you're subscribed. And if you had any issues, you can book a free one-on-one session using the link in the description. I'll see you in the next video. Bye! --- ## Scaffold an API Spec in Anypoint Code Builder (ACB) + VS Code Project Tour & Tips (Part 2) Watch: https://www.youtube.com/watch?v=SIcxoyL5NtY | Page: https://prostdev.com/video/scaffold-api-spec-acb-project-tour | Series: Learn Anypoint Code Builder (ACB) ### Intro In the last video we designed a complete API specification using OpenAPI. Now it's time to scaffold it. In this video I'll show you how to take that spec from Anypoint Exchange, generate a full Mule project in Anypoint Code Builder, and get it running locally. But we're not stopping there — I'll walk you through what actually gets scaffolded, how to explore your flows in both XML and UI views, how to fix common project-setup issues, and how to start working like a real dev in VS Code with Git, Postman, error handling, and all the ACB essentials. By the end, your API will be running and you'll actually understand what's going on. ### Scaffolding from Exchange We finished our API spec and published it to Exchange — and remember there was a message asking whether to scaffold, and we selected No. So let's get that going now. In the MuleSoft tab, click **Implement an API**. (It doesn't matter whether the spec is on your computer; it's in Exchange.) Name the project something like `todo-api-impl`, select the project location, and since we're already signed in, search for "todo API", press Enter, and select your **To-Do Task API**. Click **Add asset**, leave the Mule runtime and Java version as default, and click **Create project**. ### What got scaffolded Once scaffolding finishes, you'll see all your flows. Note the **error handling** that's already there — this is the default for common errors: bad request, resource not found (404), method not allowed (405), not acceptable (406), unsupported media type (415), and so on. Each sets a target payload and an `httpStatus` variable. These error messages are created by default — you don't even define them in your API spec. There's a **listener** with a connection configuration and a path starting with `/api`, running on `localhost:8081`. We'll keep that for now and save best practices for a later issue. At the top is the **flow list** — one flow per method (the two GETs, the DELETE, the POST, and the PUT). There's also an **API Console** flow you can leave or ignore (a lot of people don't use it). You also get a Log4j config, test resources with some embedded sample data, an auto-created `.gitignore` (handy when you start using Git), the `mule-artifact.json` (don't edit it directly), and the POM. For any file under `src/main/mule`, right-click and check out **Open Mule project properties** — it's important: it shows what's in your `mule-artifact.json` in a nice UI, and it'll download runtimes or Java versions you don't already have. For example, I only have 4.9.6, but if I select another and click Apply, it downloads it. On the **Connectors** side you can see everything in your `pom.xml` (APIkit module, HTTP connector, sockets connector) — all added automatically by scaffolding — check available versions, and see which are incompatible with your runtime or Java. ### Cleaning up: .gitignore and source control To get rid of the embedded folders (they only create sample data), open the `.gitignore` and add `src/test/resources/embedded` (and anything else named "embedded"); save, and they disappear from the tree. To start a repo, go to **Source Control** and click **Initialize repo**. All files and folders show up (minus the embedded ones we ignored). Another folder you may not want in your repo is `.vscode` — right-click it and **Add to .gitignore** (it adds one file at a time, so it's easier to just ignore the whole folder). You don't have to commit right away, but you can see all the changes happening. ### Touring the canvas buttons Minimize the panels to see the canvas. At the top: **Open Agentforce** (skip for now), **Open changes** (related to Git/source control), **Open code editor** (to see the XML — you can close the canvas and only see XML if you prefer), and **Deploy to CloudHub** (not yet). If you have the XML open instead of the UI, you'll see an **Open canvas** button instead. A great tip: by default, opening a Mule config file opens the **UI** rather than the XML. To change that, click **Configure editors**, find `source.main.mule.xml`, and set it to default to the XML view. To go back to the canvas, set it to `mulesoft.pr-file.canvas`. (This tip was brought to you by Ryan Carter — thank you, Ryan!) Right-clicking a file also gives you: **Open canvas and code editor** (both at once), **Open Mule project properties**, **Deploy to CloudHub**, **Publish to Exchange**, **Export to Mule deployable JAR**, and **Export shareable JAR**. The difference: a deployable JAR can be deployed to CloudHub but not opened in Studio; a shareable JAR can be opened in an IDE to see the code. ### Running it locally Without changing anything, press the run/debug button to start the application and confirm everything is set up correctly. Once it's deployed, test it with the **Postcode** extension (or Postman, or any REST client). A GET to `localhost:8081/api/tasks` returns a `200` with an empty body (we haven't implemented anything yet). `localhost:8081/api/tasks/1` also returns `200` with nothing. Check the terminal logs and you'll see the GET requests coming through — the app is actually running. Then stop the application. ### Wrap-up We've scaffolded our API, explored everything ACB gives us out of the box, and you've got a much better feel for working inside VS Code with MuleSoft. We didn't touch any flow logic yet, but now you know how to find the flows, view and edit them in both UI and XML, set up your default views, run the app locally, and get started with source control and helpful extensions like Postcode and Postman. In the next video we'll start implementing real logic, beginning with the GET and POST methods, then the rest. If this helped you feel more confident in ACB, give it a thumbs up and drop a comment — I read every single one. And don't forget to subscribe so you don't miss the implementation video. That's all for this one — see you in the next video. Bye! --- ## Apparently I Lied: How to Import JARs in Anypoint Code Builder (ACB) Watch: https://www.youtube.com/watch?v=5xgb0bbzWBs | Page: https://prostdev.com/video/import-jars-anypoint-code-builder | Series: Learn Anypoint Code Builder (ACB) ### A quick correction: you CAN import a JAR Hello, hello everyone. A very quick announcement: I lied to you. I'm so sorry — it wasn't on purpose, I just didn't know better. When I told you in an earlier video that there's no way to import a JAR in VS Code, that was a complete and utter lie. My bad. There **is** a way to import a JAR into ACB — you don't have to use the "open folder" approach. ### Exporting a deployable archive from Studio First, in Studio, right-click the project and select **Export**. Choose **Anypoint Studio project to Mule deployable archive** and click Next. I'll rename it `test123` so I know which one it is. Select **Attach project sources** and leave **Include project modules and dependencies** unchecked, since we won't be deploying to CloudHub. Note the message: a lightweight package generated without modules and dependencies won't be deployable to CloudHub, but it *can* be imported into Studio (and ACB). That's exactly what we want. Save it in Downloads as `test123` and click **Finish**. This generates a JAR file. ### Importing the JAR into ACB Back in ACB / VS Code, open the command palette and select **Import a Mule project** (just like the docs say). Go to Downloads, pick the `test123` JAR, then choose a project folder (Downloads again) and select it. Once it loads, open the project. ### Why import instead of just opening the folder This is actually one of the reasons it's better to **export and import** rather than just opening the folder. For example, look at the Java version: in Studio this project is on Java 8 (on purpose, to demonstrate), with Mule server 4.9.7 — a combination that isn't possible from ACB but is fine in Studio. If you just opened the original folder, ACB would modify the `mule-artifact.json` of **both** projects, because you'd be pointing at the same files. So especially if you're just trying ACB and don't want to touch your day-to-day enterprise files or your real GitHub repo, importing is better: it lets you see the differences in a fresh copy instead of modifying the original. In Studio we have 4.9.6 and Java 8, which isn't really possible in ACB. So when I click **Set versions**, it tells me the runtime is 4.9.6 but the Java version is unsupported — so I select 17 and click Apply. You can also go to **Connectors** and update as needed (in this case just one). Now if I open `mule-artifact.json`, the imported project shows Java 17, while Studio still shows Java 8. Even if I refresh, they stay independent — because choosing a new project folder made a copy. The imported project lives in Downloads; the original stays in the Studio workspace. Likewise, the imported project isn't tied to the original repo, so you'd initialize a new repo (or reference the old one) if you want. ### Recap To recap: in Studio, click **Export**, select **Anypoint Studio project to Mule deployable archive**, click Next, uncheck **Include project modules and dependencies**, leave **Attach project sources** checked, save it somewhere, and click Finish. Then in VS Code, open the command palette, select **Import Mule project**, pick your JAR, and choose where the new project should live. ### Bonus: the CurieTech AI plugin for Studio One more secret — well, it's not a secret anymore, there are articles and videos about it: there is a new **CurieTech AI plugin** you can use in Studio now. Go check it out, it's really cool. You can apply everything from a panel, see your tasks, create new tasks, and even undo things. You can move it wherever you want — I like it on the side, but that's up to you, and you can close and reopen it easily. That was my quick update. I'm sorry I lied to you — again, not on purpose, just ignorance. See you later. Bye! --- ## Start Implementing APIs in Anypoint Code Builder (ACB) with MySQL + Docker (Part 3) Watch: https://www.youtube.com/watch?v=uV_ncdIoJQI | Page: https://prostdev.com/video/implementing-apis-acb-mysql-docker | Series: Learn Anypoint Code Builder (ACB) ### Intro Hello, hello everyone. In this video we're going to start actually implementing some things. In the last videos we first designed, then scaffolded, and learned all the tips and tricks about scaffolding your new Mule flows. In this one we'll start implementing — but just for one of the methods. Here's the plot twist: you're going to have some homework so you can get used to ACB and do some things yourself. I'll show you how to do one method, and you'll do the other four during the week. I'll come back next week and show you the solution I came up with. We're also going to learn how to run a MySQL database in Docker so we can connect to it and start saving to-do tasks. It doesn't have to be MySQL — it could be MongoDB, or even an Object Store (though that needs an enterprise account). I just figured a local MySQL database with Docker would be easiest. Let's get started. ### Running MySQL in Docker This is where we left off in the last video. First, I'll remove the API Console since we won't use it. Then I created a `docker-compose.yml` file that has all the information needed to run the Docker image. I also have the **Containers** extension (really the Docker extension), which gives a Containers tab where you can see images — I have `mysql:latest` — and running containers. You can see the image, the environment (admin), the name `mysql-db`, the user and password, and the port `3306`. With this extension installed it's easy: open the compose file and click **Run all services**. It downloads what it needs and runs the Docker container on the port you asked for, with all that information. After that, I installed a **database** extension where you can connect to any local database. Click to add a configuration — MySQL database — with host `127.0.0.1` (localhost) and port `3306`, plus the database name, user, and password. At first you won't have any tables, but I already modified a SQL script to create a `tasks` table with `id`, `title`, `description`, `due_date`, and `completed` — same as in our OpenAPI spec. Click **Run** and it generates the table. ### One config file per flow Now for the actual work. There are different ways to organize your flows. If you keep them all in one XML, you'd see them together (like in Studio) — but ACB only shows one flow at a time. Since this is a small project, I recommend creating a **new configuration file per flow**. I'll do just one — the **GET tasks** — and you'll do the rest. Right-click `src/main/mule` and choose **New Mule configuration file** (not "New file", which creates a literally empty file). The configuration file adds all the XML boilerplate. Name it `get-all-tasks` (you don't even need to add `.xml`). Then select **Build a sub-flow** (we don't need a whole flow, we'll reference it). Name it `get-all-tasks-subflow` and press Enter (very important, or it won't save). Add a logger so there's no error, and save. Back in the **get tasks** flow, remove its default logger (Delete key, or right-click → Delete component), then add a **Flow Reference** pointing to `get-all-tasks-subflow`. Save. ### A global config file We're not done. You can keep the global configuration here or move it to a `global.xml` — I prefer a global file. Create a new Mule configuration file, select **Global**, and press Enter. From the code of the other file, cut the APIkit config and the listener, and paste them into `global.xml` (in the code view). Save both. Open the command palette and run **Format Document** so the XML looks pretty (personal preference). You'll notice there are no properties yet — all the configuration is inline. That's okay for now. ### Implementing the GET-all-tasks logic Back in `get-all-tasks`, open the logger and set the message to "start get all tasks". Save (⌘S / Ctrl S). Add a second logger at the end with "end get all tasks". Now we have a start and an end so we can follow the logs. We have a MySQL database, so click **+** and select **Database → Select** (search for it or find it under Connectors → Database). Save, then open the POM. Change the database connector to version 1.14.2. You'll see the new dependency. But we need more: add a `mysql-connector-java` dependency, version `8.0.29` (the latest, 8.0.33, gave me an error, so let's keep 8.0.29 — yes, it has one vulnerability, but that's okay). That's still not enough: at the top, in the `mule-maven-plugin`, the configuration is empty, so add a **shared library** with group ID `mysql` and artifact `mysql-connector-java`, matching the dependency below. Now create the DB config. If you type `DB`, you can pick **DB MySQL config** directly (best), or **DB Config** and then choose a MySQL connection inside. Add the values directly: host `127.0.0.1`, port `3306`, user `user`, password `password`, database `mysql-db`. Save and click **Test connection** — valid. (At first I typed `username` instead of `user`; fixing that, the test passes.) If you didn't have the shared library, dependency, and POM config set up, this test would fail — I demonstrated that: without everything in place the connection test fails. Back in the select, set the query to come from `payload` — we'll build the query in a transform message. There are no input parameters, so leave that and save. ### The DataWeave files (request query + JSON response) Under `src/main/resources`, create a `dw` folder. Inside, create `get-all-tasks-request.dwl` and `get-all-tasks-response.dwl`. Add a **Transform Message** before the select. Instead of an inline script, select the DataWeave file `dw/get-all-tasks-request.dwl`. Rename the step to "request" and save. After the select, add another transform — "response" — using `dw/get-all-tasks-response.dwl`. You could also name these "create SQL query" and "to JSON response" to be more descriptive. **The request DataWeave (building the WHERE clause):** ```dataweave %dw 2.0 output application/java var completed = attributes.queryParams.completed var dueDate = attributes.queryParams.dueDate var whereClause = if (isEmpty(completed) and isEmpty(dueDate)) "" else " WHERE " ++ ([ "completed=$(completed)" if !isEmpty(completed), "due_date=$(dueDate)" if !isEmpty(dueDate) ] joinBy " AND ") --- "SELECT * FROM tasks$(whereClause)" ``` The `$(...)` interpolation works without parentheses when you have a single variable; for a more complex expression you'd use parentheses (e.g. `$(default something)`). We build an array that contains the `completed` string only if `completed` isn't empty, and the `due_date` string only if `dueDate` isn't empty — so we end up with one or two items, never an empty array (because the top-level `if` already handled that case). Then `joinBy " AND "` stitches them together. The result is a WHERE clause with `completed`, `due_date`, or both joined by `AND`. **The response DataWeave (rows → JSON):** ```dataweave %dw 2.0 output application/json --- payload map { id: $.id as String, title: $.title, description: $.description, dueDate: $.due_date, completed: $.completed } ``` Keeping it simple so we can see all the fields returned. Note the database `id` is an integer, so we coerce it `as String`. ### Running, testing, and debugging Run the application. Open Postcode (or Postman) and send a GET to `localhost:8081/api/tasks` with no query parameters — we correctly receive an empty array, and the logs show "start get all tasks" and "end get all tasks". Now send query parameters: `completed=true` and `dueDate=2025-01-01`. Still nothing yet (the table is empty), but since the app is running we can debug it. Add a breakpoint before the select to see what's coming in, and one after to see what's coming out. Run again. The Mule message shows `SELECT * FROM tasks WHERE completed=true AND due_date=...` — exactly the date we sent. Continue, and the message is empty because nothing was returned, which is what we wanted. Send only one query parameter and you can see the new query: `SELECT * FROM tasks WHERE completed=true`. Note there are two spaces — that's fine, but we can remove the extra space in the DataWeave (we had one after `tasks` and one before `WHERE`). You can also use **Hot Deploy**: change a file, save, click Hot Deploy, and it sends just that file to the runtime — faster than stopping and restarting. For example, add a "hello" log, save, hot deploy, and you'll see "started app". Send the request and the console shows "hello". Interesting test: with the extra space removed in the DataWeave file and breakpoints back, send again — even though the DataWeave is in a *separate* file (not this XML), the change is picked up: `SELECT * FROM tasks WHERE completed=true` with just one space. We didn't have to stop and restart everything. Confirmed it's working, so stop the application. ### Your homework — recap of the changes Here's a quick recap of everything we did for just **one** method, so you can do the rest: - Added a `global.xml` (you don't redo this — it already has the DB config). - Added the whole `get-all-tasks.xml` file. - Added the `get-all-tasks-subflow`, which only holds the implementation so it's easier to navigate. - Removed the API Console (do this once). - In the **get tasks** flow, removed everything and left just the flow reference to the subflow — you'll do that for all the methods. - Created the per-method XMLs (because it's a small project) and the flow references from the four remaining flows. When you build the other flows, look at the **Database** module operations: **Delete**, **Insert**, **Select** (we used it), and **Update**. Use those four. We also created two DataWeave files — one for the request, one for the response — to keep the XML cleaner (no inline code). Run **Format Document** if you like the XML pretty. And **always, always save** (⌘S / Ctrl S) — it won't autosave. Finally, the POM changes (do once): the plugin version, the configuration with the shared library, and the `mysql-connector-java` dependency (the JDBC driver used by the DB connector — don't quote me on that exactly). We did it — we wrote some code in ACB, and it wasn't that bad. For the next video, come back and we'll compare notes on the rest of the implementation. If you have Git set up, you can also see your changes from the Source Control view, which is handy so you don't have to track changes in your head. That's all for this video — see you in the next one. Bye! --- ## MuleSoft Anypoint Code Builder (ACB) Full CRUD Tutorial | MySQL + Docker + Debugging (Part 4) Watch: https://www.youtube.com/watch?v=_GY38ZvP8fI | Page: https://prostdev.com/video/acb-full-crud-mysql-docker-debugging | Series: Learn Anypoint Code Builder (ACB) ### Intro Hello, hello everyone, and we're back with the implementation of my To-Do Task Management API. Hopefully you finished the homework. In the last video we had the **get all tasks** flow: start → create SQL query (built with DataWeave) → select → transform to JSON → end. The homework was to implement the rest of the methods. Let me run it and walk through everything. ### Testing the happy path in Postman The app is deployed. In Postman I already have a collection with the GET, the create, the GET by ID, the update, and the delete. - **Get tasks:** with no query parameters I get ID 7, title "get groceries", completed false. Notice I'm not getting the other fields back because they're null. - **Create a task:** the body says "finish this video". Send it and I get ID 8, title "finish this video", completed automatically false. Getting all tasks again shows ID 8. - **Get task by ID:** task 8 returns the ID, title, completed false. - **Update:** for task 8 I send a body — I don't need to send the ID because it's in the URI. Change completed to true and send → 200 OK. Getting it again shows true; getting all tasks shows true. Correctly changed in the database. - **Delete:** delete task 7 → 204 No Content, nothing in the body. Getting all tasks now returns only ID 8. ### Known bugs (happy path only — no error handling yet) Everything works, but this is **only the happy path**. There's no error handling yet (I wanted to do that with you all). - **Get by ID** with a non-existent ID (e.g. 9) returns `200 OK` with nothing, instead of a `404 Not Found`. Bug — it should return an error saying it doesn't exist. - **Update** task 9 (which doesn't exist) just returns null and nothing gets updated, but responds `200`. Should be `404`. - **Delete** task 9 says `204 No Content`, but nothing was deleted because there was no task 9. Should be `404`. The query parameters for get-all-tasks do work: `completed=false` returns nothing (our task is true), `completed=true` returns the task. `dueDate` returns nothing when there's no match, which is correct. So those three (get by ID, update, delete) need 404 handling — that's what we'll fix in the next video. ### Reviewing what changed (Git diff) Let's look at everything we modified, using the Source Control view. - **POM:** removed the snapshot version (to `1.0.0`) and updated the To-Do Task API to `1.0.2` because I made changes in Exchange. - **The API spec / Exchange:** I changed from `1.0.0` to `1.0.2`. One change was to remove the required `completed` field in the create — I hadn't realized it was required, which would have forced sending `completed` on create. I wanted to send just the title and let the default be false. So I modified the spec, republished to Exchange (now `1.0.2`), and updated the POM. I also had to update the version in `global.xml` (you could make this a property so you don't edit it manually). - Added the **create new task**, **delete**, **get all task by ID**, and **update by ID** implementations. - Modified **get all tasks**: renamed loggers and renamed the DataWeave files to make more sense (optional). - In the **implementation file**, removed everything from the flows, leaving only flow references — same pattern as `get-all-tasks-subflow`. Also moved the global configuration to `global.xml`. - Added several DataWeave files: `create-task-input`, `task-id-uri-param`, `to-json`, and `update-task-query`. In `all-tasks` (response) I added logic to skip empty fields (so a null `dueDate` isn't shown), and in the all-tasks query I added some string coercions. Throughout, I open the command palette and run **Format Document** because I like pretty XML. ### Create new task The flow: start logger → transform message → insert → select (to get the auto-generated ID) → transform to JSON → end logger. The transform creates an **input payload variable** (click the X to remove the default payload, then Add → Variable, confirm to rename). It uses `create-task-input.dwl`. Everything sent to SQL has to be Java, so: ```dataweave %dw 2.0 output application/java --- { id: attributes.uriParams.taskId default payload.id default null, title: payload.title default null, description: payload.description default null, dueDate: payload.dueDate default null, completed: payload.completed default false } ``` I reuse the same DataWeave for the update (which *does* have an ID in the URI). For create there's no URI param, so `id` falls back to `payload.id` (also null here), which is fine because we don't send the ID. `title` is required so it's always there, but I add `default null` for safety on title, description, and dueDate. `completed` defaults to false. The **insert** is a plain string (not an FX expression): ```sql INSERT INTO tasks (title, description, due_date, completed) VALUES (:title, :description, :due_date, :completed) ``` We don't send the `id` — that's why it doesn't matter if `id` is null here. The **input parameters** come from the `inputPayload` variable. Because the table was created with `id INT NOT NULL PRIMARY KEY AUTO_INCREMENT`, MySQL handles the ID for us, so I don't have the new ID. To get it, I run a **select**: ```sql SELECT * FROM tasks WHERE title = :title ORDER BY id DESC LIMIT 1 ``` I take the title (the only required field) from the `inputPayload` variable, order by ID descending, and limit to one to get the newest. Then I transform to JSON with `to-json.dwl`, which maps mostly Java → `application/json`, takes the first index (a select returns an array), matches the ID and title, skips empty description and dueDate, and always returns `completed` (default false) and an ID. You'll see red squiggles because there's no sample payload defined — click **Define sample data** to fix that and even run the mapping, but I skipped it because I'm lazy. ### Delete Start logger → transform (create `taskId` variable) → delete → end logger. The transform uses `task-id-uri-param.dwl`: ```dataweave %dw 2.0 output application/java --- { taskId: attributes.uriParams.taskId } ``` Everything is Java because it's used in SQL. The **delete**: ```sql DELETE FROM tasks WHERE id = :taskId ``` The input parameters send `vars.taskId` (the object), and the query takes the `taskId` key — so `:taskId` equals the value. That's why we send an object instead of the raw value. No output mapping because it's a `204` with no content. ### Get task by ID Start logger → `task-id-uri-param.dwl` (same file as delete) → select with `LIMIT 1`: ```sql SELECT * FROM tasks WHERE id = :taskId LIMIT 1 ``` Then transform with the same `to-json.dwl`. We use `LIMIT 1` just to be tidy (it shouldn't match more than one). ### Update by ID The transform creates both a **payload** (with `create-task-input.dwl`) and a `vars.taskId` (with `task-id-uri-param.dwl`) — they're set at the same time in the same transform message. This is where `create-task-input.dwl` actually uses `attributes.uriParams.taskId`. Then the **update**: ```sql UPDATE tasks SET title = :title, description = :description, due_date = :due_date, completed = :completed WHERE id = :id ``` After updating, I run a **select** (same one as create/get-by-ID) to make sure I return what was actually saved rather than just echoing the input, then transform to JSON. ### Refactoring repeated logic into shared subflows You've noticed the same three steps (set task-id variable → select → to-JSON) repeat across subflows. So let's extract them. In `global.xml`, create a subflow `global-get-task-by-id` and move the select + to-JSON transform there. Then realize the **set task-id** step must run *first* (before the URI param is gone), but the update needs it in a different order — so make the task-id variable its own subflow too: `global-set-task-id-var-from-uri`. Now wire flow references: - **Get task by ID** → flow ref to `global-set-task-id` + flow ref to `global-get-task-by-id`. - **Delete** → flow ref to `global-set-task-id`, then delete (no get afterward). - **Update** → keep the `create-task-input` payload, replace the inline task-id transform with a flow ref to `global-set-task-id`, do the update, then flow ref to `global-get-task-by-id`. Now if we ever change how we get the task ID, we change it in one place. ### Re-testing everything Run it. In Postman: - **Get tasks** → works (with `completed` true/false too). - **Create** "be awesome" → ID 9, completed false. (We are awesome — should be true, so we'll update it.) - **Get by ID** 9 → "be awesome", completed false. - **Update** 9 → I found a bug: if I leave the ID in the body it breaks. Let's debug the update — breakpoints in. The `UPDATE ... WHERE id = :id` wasn't taking the ID correctly. The problem: I changed the variable to `vars.taskId`, but `taskId` is an object, so I need `vars.taskId.taskId` to get the actual value. (Because I modified a DataWeave file, I restart/redeploy.) Now ID 10, "be awesome", completed true → works. - **Delete** 9 → deleted (we still shouldn't get content back, but we'll fix that next video). A quick bug, solved quickly, and we saw how to debug along the way — a win-win. ### Wrap-up For the next video we'll add **error handling** so this works as it should — returning a `404` when something isn't found. That's also where our new global subflow comes in handy. We'll use **Choice** routers to check existence and send an error if not found, or a try/catch approach. Subscribe and turn on the alerts so you don't miss the next video, where we learn error handling in Anypoint Code Builder. I hope this was fun — see you next Monday. Bye! --- ## Handling 404 Errors in MuleSoft's Anypoint Code Builder (ACB) | Choice + Raise Error (Part 5) Watch: https://www.youtube.com/watch?v=_tCTfZ4YkH0 | Page: https://prostdev.com/video/handling-404-errors-acb-choice-raise-error | Series: Learn Anypoint Code Builder (ACB) ### Intro Our application is running, my MySQL container is running, and I'm connected to the database locally. Let's open Postman, check **get all tasks**, and make sure there are no tasks right now. We already had the happy path working from previous videos, but let's double-check. Create a new task with title "be awesome" → created with ID 15. Get all tasks → ID 15, "be awesome", completed false. Get task 15 → `200 OK`, "be awesome", completed false. Update task 15 with completed true → `200`, "be awesome", completed true; the previous get now shows true. Delete task 15 → `204 No Content`, literally no content. So the happy path works. The problem was when we used an ID that **didn't exist** and still got a `200 OK` or `204` — that's what we're fixing in this video. I've already modified the code; let me guide you through it. Now, trying to get task 15 (which I deleted) returns `404 Not Found`, message "task not found". Same for the update (`404`, "task not found") and the delete (`404`, "task not found"). ### The shared global flow: select → Choice → Raise Error In summary, we added a **Raise Error** component to make this work. Remember from the previous video that `global.xml` used to have two flows; now it has just one. When execution arrives, it first goes to a transform message to retrieve the task ID, using `task-id-uri-param.dwl`: ```dataweave %dw 2.0 output application/java --- { taskId: vars.taskId.taskId default attributes.uriParams.taskId } ``` We first check whether a `taskId` variable already exists — if it does, use it (don't replace it); if not, get it from `attributes.uriParams.taskId`. The attributes path is used by the **get** and the **delete**. The **update** is special: we call this flow **twice** — once to confirm the task exists (using the URI params), and a second time to retrieve the updated value (using the variable, because by then the attributes are gone). That's why we prefer the variable when it's present. After the DataWeave, we run the **select**: ```sql SELECT * FROM tasks WHERE id = :taskId ``` sending `vars.taskId` as the input parameters. Then a **Choice** router: - **When** the payload is empty (the select returned nothing — the task doesn't exist), do the `404` path. - **Otherwise**, run the transform message that maps the result to the JSON response format. In the `404` path, the transform message sets the payload to the "task not found" message in JSON and sets a `httpStatus` variable to `404`. (This should look familiar — the scaffolded main flow, `todo-api-impl`, does the same thing inside its `on-error-propagate` handlers for bad request, unsupported media type 415, 406, 405, etc.) There's currently a UI bug that hides the variable, so I show it via the XML. After setting that up, we **Raise Error**. The error **type** doesn't really matter here because we don't have a global error-handler element configured for it — but if you later need to handle a specific type with `on-error-continue` or `on-error-propagate`, make sure you know the type you're raising. For this case I just want to raise the error so it **stops the flow** and doesn't continue. So this global flow simply checks whether the task exists: if yes, put it in JSON; if no, raise an error and don't continue. ### Get task by ID This is the simplest one — it just runs the flow reference to the global flow. Nothing else. ### Delete First it checks whether the task ID exists (via the global check); if it does, continue, if not, raise an error and stop. Then: ```sql DELETE FROM tasks WHERE id = :taskId ``` using the `taskId` variable. ### Update (the tricky one) The update first sets up the **task input** (the payload for the SQL update via `create-task-input.dwl`). The problem: the global flow reference does a full get-task-by-ID, which **replaces the payload** — so by the time we reach the update, it would use the wrong data. To fix that, on the flow reference go to **Advanced** and save the payload into a `beforeUpdate` target variable. Now, because the global flow also creates the `taskId` variable internally and it flows out, we reference `vars.beforeUpdate` instead of `taskId`. So in `create-task-input.dwl` the ID comes from `vars.beforeUpdate.id default payload.id` (the latter for the rare case there's a payload ID). After it builds the Java payload, it runs the update (`SET title = ... WHERE id = :id`, the ID taken from this payload). Once updated, we get back something like "updated rows: 1". Then we need the actual saved task, so we run the **get task by ID** global flow again — and this is the moment from the intro: there are no attributes anymore, so the DataWeave falls back to the `beforeUpdate` variable to find the ID. It retrieves the task, outputs JSON, and returns it. ### Wrap-up In summary: in the flow everyone calls, we do a select; if nothing is returned we send a `404`, raise the error, and stop; if found, we transform Java → JSON and return it. Once you send the `404`, Postman automatically knows it means "not found" — you just set the `httpStatus` to 404 and the message to "task not found". Let me know if you want videos demystifying error handling — `on-error-continue` vs. `on-error-propagate` — in ACB (I don't do Studio anymore). Drop a comment with what you want, or any other content you'd like. That pretty much ends the whole to-do app implementation for this use case. I can add more — separating the API into experience, process, and system layers, for example. I'm happy to help you understand whatever you need from ACB. Have a great day, and I'll see you in the next video. Bye! --- ## Generating an OAS API Spec with CurieTech AI | MuleSoft API Designer (Part 1) Watch: https://www.youtube.com/watch?v=aAHpfYED3So | Page: https://prostdev.com/video/generate-oas-api-spec-curietech-ai | Series: From Zero to API with MuleSoft, CurieTech AI & Anypoint Code Builder (2025) ### Intro Hi, I'm Alex. You might be familiar with me because I did the whole MuleSoft from Start series. So it's kind of time to redo that, but better. I was thinking: what better technology to do that with than CurieTech AI? This is awesome. Let's learn how to develop a whole Mule application from scratch, but now using AI — and this time using CurieTech AI. I'm going to use the same use case as before, which was the to-do task management application. Let's get started. ### Generating the first OAS spec Here I am in CurieTech AI. Let's press on **API Spec Generator**, and in this case I'm going to select **OAS 3**. I normally do it in RAML, but let's keep up with the times. Let's start with something simple: "Create a to-do task management application API spec. Make sure to include all CRUD operations for the tasks." Now it's going to ask a few questions to make sure it got it right: 1. Do you want the specification in YAML or JSON? → **YAML**. 2. Should tasks be associated with users, or is it a simple single-user task list? → **Simple single-user task list**. 3. Do you require authentication? → For now, **no**. 4. Any specific fields you want for a task? It suggested title, description, due date, status, priority. → Let's just say **title, description, due date, and status**. 5. Should there be support for filtering or searching tasks? → **Yes**. Let's provide those details and see what it does. Because we're testing this on an AI, your results might be slightly different from mine — that's okay. I'll show you what we should all have in order to continue. This is also great because it makes you think about how *you* want it, instead of just copy-pasting everything I did. ### Reviewing the generated paths Let's take a quick look. We have `/tasks` GET, and it receives query parameters so we can filter by status or by due date. The response will be of type `task`. Then we have the POST to create a new task — it takes a `taskCreate` and returns a `task`. Then we have `/tasks/{taskId}`. We have a GET, with a path parameter for the task ID, returning a `task`. Then the PUT, again with the task ID path parameter, taking a `taskUpdate`. And the DELETE, which just deletes by ID and returns nothing — a 204 response. The PUT and GET were returning a 200, but let me give it some feedback: "Make sure you're using RESTful best practices." It fixed the status codes — 200, 201, 204. So now: getting all the tasks returns 200 with an array of `task`. We still have the filters for status and due date. The POST takes `taskCreate` and returns a `task` with a 201. Then `/tasks/{taskId}` GET returns a `task` with a 200, plus a **404 task not found**. The PUT takes the task ID path param and a `taskUpdate`, returning a `task` with a 200, or a 404. DELETE takes the task ID and responds with 204 or 404. ### Refining the schemas Now the component schemas. We have `task` with id, title, description, due date, and status — required are id, title, and status. That's correct. Then `taskCreate` for the POST: title, description, due date, status — required are title and status. Note the status is set as pending, in progress, or completed. Then we have `taskUpdate`, which I feel is redundant because it's the same as `taskCreate`. So let's give it more feedback: "`taskCreate` and `taskUpdate` are redundant because they're doing the same thing — please use only one. Change the status to be `completed` and make it a boolean. Default value should be false." Now the POST is a `taskInput` and the PUT is also a `taskInput`. Good. We have `completed` as a boolean, default false, example false; required are id, title, and completed. And `taskInput` has `completed` boolean, default false, example false; required title and completed. Perfect. One more thing I know is going to be an issue: the quotes. So I'll tell it: "The example value shouldn't have quotes, because it'll cause issues with the preview in API Console." You wouldn't know this — there's no need for you to know it. I just know it from doing this a lot of times. Now all the examples are plain text. The id might cause issues too, but we'll see when we pass it to Anypoint Platform. ### Bringing it into Design Center Let's bring it to Design Center. The title is "To-Do Task Management API." Go to Design Center → **Create new API specification**, add the project name, OAS 3 YAML, and create the API. Back in CurieTech, press the copy button so it copies everything, go back to the designer, and paste it in. I'm going to remove the servers since I don't have that information yet. But we have the endpoints. `/tasks` GET has the query parameters `completed` and `dueDate`. I do see the id as a number instead of a string, so let's change that. The id is required, the title is required, and `completed` is required with default false. The POST sends title, description, due date, completed — description and due date aren't required — and returns a 201. For `/tasks/{taskId}` GET, we send the task ID in the path and receive a whole task. The PUT sends the task without the id (the id comes from the path) and returns the updated task. The DELETE sends just the task ID and returns a 204 (no content) or a 404 if not found. ### Publishing to Exchange I think this is ready. Let's publish version one to Exchange. And it was published. We can take a look — we have `/tasks` GET and POST, and `/tasks/{taskId}` GET, PUT, DELETE. So congratulations — you designed and published an API specification using CurieTech as your AI assistant. Let's watch the next video to see how to start implementing this API and how CurieTech is going to help us with that. See you in a few minutes. Bye. --- ## Implementing the CRUD REST API in Anypoint Code Builder with CurieTech AI | MuleSoft App (Part 2) Watch: https://www.youtube.com/watch?v=NbjMNK5XUAw | Page: https://prostdev.com/video/implement-crud-rest-api-acb-curietech-ai | Series: From Zero to API with MuleSoft, CurieTech AI & Anypoint Code Builder (2025) ### Recap of Part 1 Hello everyone, my name is Alex, and in the last video we saw how to create a To-Do Task Management API specification using OAS from CurieTech. Just a reminder of what we did: we started with a very simple command telling it to create the app for us, it asked some questions, we answered them, and it created the first version. Then we did some back and forth — first "make sure you're using RESTful best practices," then "`taskCreate` and `taskUpdate` are redundant" and "change the status to be `completed` and make it a boolean." Finally we said "the example value shouldn't have quotes because it'll cause issues with the preview in API Console" — that's not a CurieTech issue, it's an API Console issue in MuleSoft. It took about 10 minutes, and we published it to Exchange. ### Installing Anypoint Code Builder Now the second step is to implement that API specification using — you guessed it — Anypoint Code Builder. In my last "MuleSoft from Start" series I was using Studio, but that's in the past. Now we're going to be using ACB. How do you install ACB? First, install Visual Studio Code. Then install Git locally, because I'm using the desktop version, not the cloud version. Then install the Anypoint Extension Pack from the Extensions panel. That's pretty much it. Once installed, you'll see the MuleSoft logo and the ACB homepage. Because we're going to connect to Exchange, we need to log in to Anypoint Platform. Click the button, click "Allow," sign in, and then open Visual Studio Code. Now you can see you're logged in. ### Implementing an API from the spec Let's click **Implement an API**. ACB gives you three quick actions: you can **design an API** (what we did in Design Center — same thing, just IDE vs. web), **implement an API** from a spec (RAML, OAS, or AsyncAPI), or **develop an integration** from scratch (no spec required). For our case, click **Implement an API**. Since our spec is called "To-Do Task Management API," let's call this project "to-do-task-management-impl" (impl = implementation). Because we're signed in, we can connect to Exchange — search for it and you'll see the organization, created by Alex Martinez. Click **Add asset**, select the Mule runtime (I'll select the latest) and Java 17, and **Create project**. After a few seconds the project is ready. ### Publishing the repo to GitHub Because you can use CurieTech with GitHub, I'm going to initialize and publish my repo to my GitHub account, which I'm already signed into. (If you click on Accounts, you can see I have my GitHub and my Anypoint Platform accounts.) First, initialize the repo. Here's everything that was created: a `.gitignore`, a `mule-artifact.json`, a `pom.xml`. I don't want the `.vscode` folder, so I'll add `.vscode/` to my `.gitignore`. Then we have the main Mule file (the flow), and the `src/main/resources` and `src/test/resources` for log4j. Let's add everything, commit ("first commit"), and click **Publish branch**. I'll create a public repo so I can also use it from CurieTech (a private repo works too, since it uses your credentials). If I check my repos, I already have "to-do-task-management-impl" there. Now go back to CurieTech, go to **Settings → Repos**, click **Connect via OAuth**, select only the specific repo I want ("to-do-task-management-impl"), and save. It asks for a branch — `main`. Now under indexes I can see my repo, and I can set Java 17 and Maven 3.9. I won't use a settings.xml for now — it's a very simple example. ### Asking the Code Enhancer to implement everything Now go to a new conversation and select **Code Enhancer with repo**. Select the repo on `main`, and in the description: > Implement the whole API with the CRUD operations based on the API specification. For better results, I'd really advise you to do each endpoint one by one instead of implementing the whole thing at once, like I did here. Because I did it all at once, I introduced some bugs that could have been avoided by letting the Code Enhancer focus on smaller tasks. So don't be me — do one small task at a time. I copied and pasted the whole OAS so it would know exactly what it is, and added instructions: > The flows have already been scaffolded into the main XML file — don't change these. Just add > flow references from the main file's flows to the subflows with the actual logic. Create a > different XML file for each of the five subflows: (1) get all tasks, (2) create a new task, > (3) get a task by ID, (4) update a task by ID, (5) delete a task by ID. The data is saved in a MySQL database. I created a MySQL container locally with a `docker compose`, ran it, and I'm using an extension to see my tables. I gave it the host, port, database, user, and password, plus the `CREATE TABLE`. I also noted: > When you create a task, the id in the DB will be auto-incremented. But note the id in the DB is > a number (`int`) and our API spec is a string — so make sure to transform this data in the > code. Follow best practices and secure the needed properties. Then I pasted the best-practices document I'd created for Cursor. I also uploaded my settings.xml just in case, since CurieTech runs the whole project before giving an answer (it might need my credentials). Let's run it. ### Reviewing the generated diff I had recorded this whole thing, but my computer crashed — sorry about that. It took around 20 minutes to generate the diff. Once you receive the diff, it asks if it's okay or if you need changes. I approved it, so you see "diff reviewed by Alex" and then "task completed." Clicking it opens the **pull request** it created for you, with all the details and the files it changed. I find it easier to get it onto my local so I can look at the actual code. Looking at my globals: I'd been manually externalizing a few properties, but CurieTech created all of them on its own — host, port, MySQL details, user, password, database name. It got everything right. I will have to change the host for dev, but for local properties this is great. Nothing changed on the main flow and the console — the only difference is the flows now have flow references. The get-all-tasks flow has a flow-ref to "get all tasks," the POST has a flow-ref to "create task flow," and so on. For the ones with a task ID in the path, MuleSoft had already created a Transform Message to save the task ID in a variable, and CurieTech just added the flow-refs exactly as I asked. CurieTech also added a Java module and a Spring module — I'm not yet sure why. The get-all-tasks subflow has a logger, a transform preparing the query (`SELECT id, title, description, due_date, completed FROM tasks` plus a WHERE clause), a Transform Message to map everything to JSON with the id as a string, and a logger. The create-task flow has a logger, a transform building the task input, the database insert, the SELECT to get the generated id, and a transform to JSON. ### Fixing the first errors I made a few changes and it almost ran, but there's an issue with the DataWeave it created — it has `if / else if / else if`, which isn't how DataWeave works. Instead of fixing it directly, I'll ask CurieTech to do that, since it would take a few brain cells. Some changes I made manually: I added the test resources embedded; modified the `pom.xml` (I removed a Maven plugin configuration I wasn't sure I needed, and removed the MySQL/Java/Spring dependencies because MySQL was throwing issues — I do have the DB connector, so that's probably good enough). I also moved the listener config and the APIkit config from the main flow to the global, since CurieTech forgot to. And I added a few properties I decided I needed. (Heads up: it turns out I *did* need those MySQL dependencies later — more on that in Part 3.) Commit and sync — this sends my changes to the `curietech-dev` branch. ### Switching to a local repo and fixing the DataWeave You can also add **local repos**. Import "to-do-task-management-impl," select Java/Maven, upload the settings, and now I can use the local repo with the active branch. New conversation → **Code Enhancer with repo** → select the local one: > Take a look at the get-all-tasks flow. The "prepared query" Transform Message has an error. > Please fix it and make sure the rest of the flow is correctly set up to get a list of all tasks > from the database. After about a minute and a half, it updated the DataWeave code and nothing else. Because we're using a local repo now, we can only download the generated code or the diff patch (no PR) — that's why we first used GitHub, since it created the PR for us. It's a straightforward change, so I just took it from the output. It says "expecting type Array but got Object," so we'll keep an eye on that — but it deployed successfully. ### Wrapping up We did a lot in this video, and we got it to run. In the next video we'll test it to see if it actually works and start troubleshooting the functionality. I know some of you will say it took a lot of time — but consider this: if I'm not familiar with setting up a database in MuleSoft, I'd have to watch or read tutorials to figure out the database connector by myself. Instead, it took about 20 minutes for CurieTech to generate functioning code, and it even fixed the DataWeave on its own after I told it about the error. As long as you have your best practices well documented in your workspace or instructions, it can do all of that for you. This gave me a working first pass in 20 minutes or less. I'll see you in the next video where we run and troubleshoot this working Mule project. Bye! --- ## Debugging MuleSoft API with CurieTech AI & Anypoint Code Builder | Local & Postman (Part 3) Watch: https://www.youtube.com/watch?v=6mhbcYTLaMo | Page: https://prostdev.com/video/debugging-mulesoft-api-curietech-ai-acb-postman | Series: From Zero to API with MuleSoft, CurieTech AI & Anypoint Code Builder (2025) ### Where we left off Hello everyone, welcome to the second part of the implementation. In the last part we were trying to get this thing going, and right now I have it deployed and I'm still testing it out. But guess what? Remember those things I removed that CurieTech gave me in the `pom.xml` — the ones I decided weren't good? It turns out they *were* needed. So I now have the configuration for the MySQL Connector/J and the dependency for it. Yes, I keep getting a transitive-dependency vulnerability warning; there's a workaround, but for now I'll leave it, because it's working and I don't want to mess with it anymore. Last time we also applied the changes it gave us for the DataWeave. So now we're just going to keep asking things in the Code Enhancer. One other tell I had: I was trying to test my SQL database connection from the global config (without running the whole thing), but my connection wasn't coming out as valid — until I asked CurieTech "my database connector is not working, please help me troubleshoot it." It gave me the correct configuration and dependency, and it worked. ### The 500 error on get-all-tasks Now my issue is that I'm trying to GET from `host/api/tasks` and I keep getting a **500 Internal Server Error** about bad SQL syntax. I know roughly where it's coming from: we set up a Transform Message to prepare the query and store a `whereClause` in a variable, but then we use that `whereClause` in a way I'm not sure is correct. So let's ask the Code Enhancer. Select the local repo on the active branch: > I keep getting this error when trying to run the get-all-tasks flow. It came up with a fix to how the syntax is set up — instead of `$(...)` it uses `#[...]`. It says it took 8 minutes, but it didn't — it took about 1 minute. (The timer counts all the approval steps and the time I was doing other stuff.) I applied the change, but when I try again I get another error. ### Using the single-repo Code Insights agent So I thought of another agent that can help: the **single-repo Code Insights** lens, where you can ask questions about a single repo. I asked "why am I getting this error in the get-all-tasks flow?" and pasted whatever was in the terminal. It was really insightful: the DB Select step dynamically builds the SQL query using a DataWeave-generated `whereClause` variable, concatenating values directly into the SQL string. When no query parameters are present, if the DataWeave logic produces a malformed clause, MuleSoft's DB connector may try to bind a parameter named `null`. If `whereClause` is empty, the SQL is valid; but if the DataWeave logic is off — like adding a clause with a missing value — the DB connector gets confused. Then it recommended **parameterized queries** and gave me the whole thing on how to do it. The only downside is I can't see the nice before/after diff like before, and I don't think this lens actually compiles or runs the code — but I can just copy and paste and see what changed. I copied everything from the Transform to the DB Select, formatted the document, and looked at it. It changed the Transform Message: now there's a variable `queryData` generating both `sql` (with the WHERE clause) and `params`. So we send the parameters as `queryData.sql` and `queryData.params`. I pressed add-and-deploy (hot deploy, no full restart). There were some issues here, so let me take this feedback and ask the **Code Enhancer** to do something similar against the repo: "Use parameterized queries." It returned a few changes on the Transform Message — variables `queryParams` and `sqlQuery` — and set up the DB Select with `sql` and the input parameters. I copied the whole thing, pasted it, saved, and hot-deployed one more time. Fingers crossed. Oh my god, it worked! It returned zero tasks because I don't have anything. I didn't even remember how to do parameterization with the DB connector, so that's really cool. ### Testing create, get, and delete in Postman Let's test the POST. From Exchange I copy the example request body, change the method to POST (after a moment of confusion forgetting to do that), and send it. It actually worked — it's creating tasks! I do see an issue where it returns id 1 here and id 0 there, so the id needs work, but it's creating the tasks. For the insert, it actually used parameterization from the start — it was only the other flow that wasn't. The problem is `SELECT LAST_INSERT_ID()` isn't working. In other news, I can search for each task by ID and retrieve it. Let's check the delete. Right now we have id 2, so let's remove id 2 → **204 No Content**, which means it worked. Yes! For the create's generated-id issue, what if we simply do `SELECT id FROM tasks ORDER BY id DESC`? That returns the last id, which I'll assume is the one I'm creating. Let's save and hot-deploy. The create now works. ### Debugging the update flow with breakpoints The update doesn't work — internal server error, "column 'title' cannot be null," even though it isn't null. Let's add a **breakpoint** and send again. Now we can see the Mule message: the payload is the whole thing — id, title, description, due date, completed. I notice the update isn't saving the original payload, because somewhere we're overwriting the whole payload. So let's add a variable `inputPayload` set to `payload`. That's the one we'll use. Then in "prepare the test data," instead of `payload.title` use `vars.inputPayload.title`, and the same for `description`, `dueDate`, and `completed`. Hot-deploy again. Send. Now we step through: first the Mule message has the application/java payload because we're preparing the task data; continue; now we're in the update with id, title, description, due date, completed. It looks like it's going to work. (Earlier I was confused because I was looking at `payload` instead of the `inputPayload` variable I created — that's why I couldn't see the updated value.) It actually got updated! Let me update again — "buy groceries by June 15," for example. Send (oops, forgot to remove the breakpoints — let me remove them). It's saved, and getting it by ID returns the updated task. It works. High five! ### Wrapping up So now our whole application is working — all five operations. And I'm just so happy about it. Thank you, CurieTech. It didn't take as long as I thought, because by the end of the day I didn't exactly remember how to use the DB connector — I hadn't used it in about five years. Asking CurieTech to do at least the beginning of it was really useful, because I'd be lost without the initial skeleton. And once I understood more or less what was happening, I was able to troubleshoot and debug on my own. Plus, if you didn't know how to debug in ACB — now you do. That's all for this video. We finished implementing our API. In the next video we're going to learn how to add MUnits to our project — and yes, using CurieTech AI. We're not going to have to create a single MUnit (I hope). See you in the next video. Bye! --- ## Generating MUnit Tests for MuleSoft APIs with CurieTech AI | Test Coverage (Part 4) Watch: https://www.youtube.com/watch?v=79x16XivYXQ | Page: https://prostdev.com/video/generate-munit-tests-mulesoft-curietech-ai | Series: From Zero to API with MuleSoft, CurieTech AI & Anypoint Code Builder (2025) ### Recap and what's next Hello everyone, my name is Alex. Welcome back to our awesome CurieTech AI playlist. In the last video we got our final Mule application — the To-Do Task Management app — fully working: get all tasks, create a new task (with the auto-generated id), get a task by ID, update a task by ID, and delete a task by ID. The next step is to create MUnits — and nope, we won't be creating them on our own. We'll rely on CurieTech AI. In CurieTech's **Testing** section there's an **MUnit Test Generator** and a **Sample Data Generator**. We'll select **MUnit → Start my own**. That reminds me: I merged the pull request CurieTech created before (the `curietech-dev` branch, "implement to-do API CRUD") that we used to push all our fixes. So everything we have now is on the `main` branch. ### Configuring the MUnit Test Generator In the MUnit Test Generator, select the repo, the branch (`main`), no VM args, enable coverage, and set "make as many scenarios as needed." Note the **number of MUnit scenarios: at most 3**. You can lower it to 1, but this is the *maximum* it can generate — it looks at your code and decides how many scenarios it needs, up to that max. It's very important to provide your settings.xml, because you might need access to your RAML or OAS (my case) and your Nexus credentials. If you don't know how to set up Nexus credentials, I have a few videos on that. Now the fun part: **you have to select just one flow** — it generates everything for one flow at a time. We have main, console, and the five resources (delete, get all tasks, get task by ID, create new task, update task by ID). We'll go one by one through the five, since those are the most important. Select **delete** and submit; add a new task, select **get** (the list), then the other **get** (by ID), and so on for all five. The cool thing is you can have them all running at the same time, which is what I did — and because we're using a GitHub repo, it creates a **pull request for each one** as soon as it's done. ### Reviewing the generated scenarios Take the PUT (update). We didn't add a description, so it only used the flow name. It shows what it's doing: - **Scenario 1** — successfully update an existing task (the happy path), mocking the database connectors, running the target flow, and asserting the logger is called and the DB connectors are called as expected. - **Scenario 2** — attempt to update a task that does not exist (the 404 path). For the POST (create), it generated only **Scenario 1** — the happy path. For get-by-ID, two scenarios (retrieve a task, and task not found). For get-all-tasks, just one scenario (the happy path). For delete, two scenarios (successful delete, and task not found). Some flows got only one scenario because it's just trying to cover 100% — and it decided one was enough. But as you and I know, **100% coverage doesn't mean the tests are correct**. If you want more scenarios — for example, get-all-tasks when there are zero tasks, or a database connection failure — you can add them in the optional **notes** when you select the flow. Just remember it creates at most three. When a flow finishes, you get a summary: it created the tests, both passed, plus the MUnit coverage report. Overall application coverage shows low (e.g. ~23%) because you're only doing one flow at a time. The report has a table of every flow and its coverage percentage — the PUT shows 100%, and so do the rest, including the get-all-tasks that only had one scenario (which is why it decided not to create more). ### Merging the pull requests In my GitHub repo, the five pull requests were added correctly. We can go branch by branch, but let's just merge them. Take get-tasks-by-ID: it has the full summary, the coverage report, and the two scenarios. Merge it, confirm, and delete the branch. Looking at the files, it added everything needed to the `pom.xml` for MUnit, the flow, and some expected-response JSON files so it can mock responses and assert expected vs. actual. For the PUT, the files include the pom, the flows, and mock data for scenario 0 and scenario 1 — so it's even testing an empty scenario. Now we have a **conflict** because it modified the pom. Open that branch, and by pressing `.` on the keyboard we can spin up the VS Code web editor and rename files. I want to keep them separate, so I'll rename the test suite XML and its resources folder to include "put," and rename the config name as well. Commit and push, refresh the PR — now there are two commits taking my changes, and it merges without conflict. Delete the branch. I'll also rename things on `main`: press `.`, rename the get-by-ID flow and do a replace, and remember to also rename the folder inside the resources folder (I forgot the first time and had to come back). Merge, push, and continue with the rest of the PRs. Once there are no open PRs left, click **Sync changes** to pull everything down. The git graph is funny — merge PR, then CurieTech 1 through 5 with all the renames. ### Renaming the test cases Another issue: all the MUnit tests have the same name, like `flow-test-0`. So I'll rename them by their scenario. For the delete suite, one is the happy path and one is "task not found," so I'll name them accordingly. Do the same for the others. Now in the **Testing** section we can see all the flows as test suites, and opening one shows how many tests each has — "happy path" and "task not found" — which is awesome. Now we can run them all. ### Final coverage I've been having an issue running the tests in my ACB — that's a *me* issue, not a CurieTech or MuleSoft one; I should probably reinstall everything. But since all my credentials are in my settings.xml in my `.m2` directory, I can just run everything with **Maven** instead. After it finishes, the new coverage is **83.93%**, because it's not covering the main and console flows. The summary table shows main at 0% and the API console at 0%. I don't really need the API console — deleting it was a best practice in some previous projects — so let's delete it and run again. Now we're at **87.04%**. To get the rest, we'd need MUnits for the error scenarios in the main flow (it's just doing the routing, plus error propagation / error handling for things like APIkit BAD_REQUEST and NOT_FOUND). There are about six of those, so you'd create two CurieTech tasks (three scenarios each) telling it exactly which errors to trigger. I'll leave reaching 100% as your **homework** — send me your results on LinkedIn, Discord, or alexpros.com and I'm happy to take a look. ### A reminder about automating the manual steps Quick reminder: everything I did manually (renaming, merging) I didn't *have* to do manually — I just found it easier to show you once. There are a few ways to automate it with CurieTech: 1. Add **notes** when you create each task, specifying your naming conventions for test suites, folders, etc. 2. Go to **Settings → Workspace** and upload a PDF with your coding guidelines, e.g. "make sure all test suites and folders under src/test/resources match the test suite name and are descriptive." 3. Use the **Code Enhancer** afterward to ask it to rename all of those things. ### Wrapping up So we already have a functioning application: we created an API specification on day one, debugged and implemented it in videos two and three, and now in this fourth video we're creating MUnits to get almost 100% coverage. Isn't this awesome? Let me know in the comments what else you'd like to see. See you in the next video. Bye! --- ## I Built a Full MCP Server in 10 Minutes Using CurieTech AI -- WHAT?! | YouTube API | MuleSoft | ACB Watch: https://www.youtube.com/watch?v=EE1QLCuzGVk | Page: https://prostdev.com/video/build-mcp-server-10-minutes-curietech-ai | Series: Adventures in MuleSoft + AI (2025) ### Intro Hello everyone. My name is Alex and today I'm going to show you how I created a whole MCP server for the YouTube Data API v3 using CurieTech AI. This is the first time I've ever built an MCP server in MuleSoft, and I did it in about 10 minutes with CurieTech — no docs, just the AI. ### Generating the YouTube API integration Once you're inside CurieTech AI (you can create a free account), I went to Integration Generator without any repo, just adding a description and selecting settings. In my case I selected Java 17, Maven 3.9, and Mule 4.9. I first asked it to generate a whole YouTube API for me, because I wanted the integration first to make sure it works before I create the MCP server on top of it. So I said: "Make an API that makes use of the YouTube Data API v3. Include all the possible query parameters including the key parameter for security. This is where your API key would go." I went to the YouTube Data API docs, copied all the parameters, and pasted them into the query so it would know what exact parameters to use. You can also just get the link and put it there — that also works. Then I said, "Make it as simple as possible." It took around two and a half minutes and got me the whole thing. I copied the code, went into Anypoint Code Builder (ACB), created a new project called `youtube-mcp-server-mule`, and pasted the whole code in. ### Testing the generated API After you start running the project — in my case I'm already running it — you can try it out. We're connected using the host `localhost` and port `8081`. There is no base path, and in our listener we have the path `/search`. So if I open Postman I can put `localhost:8081/search`, then we're going to use the `key` query parameter with the API key from your Google Cloud account. Make sure that you have the YouTube Data API v3 restriction set up. If you don't know how to do that, check out my GitHub repo in the description — I have the instructions in the README. Then we're using the `q` keyword to search for "MuleSoft". This is the same thing that comes from the Data API — we are just mapping all the parameters to make an API of our own. All of these query parameters are available for me to use from my MuleSoft integration. For now I'm just going to use the `key` and the `q`. And this is an example of what we receive. In this case we're getting 10 results because I added a few defaults. If I check the Transform Message — or rather, let me open the code — we have the required parameters which are `part` and `key`. But for `part` I am making a default; for `key` I am also adding a default. For `max_results` I'm adding a default of 10, for `order` I'm ordering by default by date, and so on. I added a few defaults just in case. So that is one of the defaults that is returning 10 results per page, but you can change that if you send the query parameter. Anyway, the API works. It is able to connect to YouTube, do a search, and return the results. Awesome. ### Pushing to GitHub and using the Coding Agent Once we verify that our API is working, we can go back to CurieTech. What I did is I created a GitHub repo and pushed the whole implementation. You can also do that directly from CurieTech — just click "Initialize repo" and then push, and it will connect to your GitHub credentials. So it just goes directly and puts it in your repo for you. After I created my repo, I clicked on "Repositories" in CurieTech. I am connected — you can click on "Connect via OAuth" and it will open another page. What I do is select only specific repos. You can select any one that you want. In my case I just added the repo that I had selected, then you click save and it gets you back to CurieTech. Once you have everything set up, you will see that you already have your stuff. I just make it a habit of coming here and selecting these things just to have them saved, because then whenever you open this repo from another agent you already have these settings. Once you have your API, everything is working, you put it in a GitHub repo. I just do it because I like that it creates a pull request for me automatically. But you can just go ahead and get it from a local repo or from just uploading your files — whatever works best for you. I went ahead and selected the Coding Agent. In the Coding Agent you can upload from computer, you can put no repo, or you can add a repo. So in my case I added my repo on the main branch, added the description, and it already has the settings. Just click on submit. For my case what I did was: "Take this API and turn it into an MCP server using the beta MCP connector." That is literally everything that I told it. It went ahead and did the diff and everything. ### Reviewing the generated changes Once you take a look at the diff that it generates for you, in my case it only modified the pom.xml. As you can see it added the dependency for the MCP beta connector, and then it went ahead and modified my whole flow to match with MCP stuff, which is pretty much what I needed. So I clicked on "Approve." Once you click on approve it will generate this PR for you. This is why I am using a GitHub repo, because I really love this feature. And now you can come here and take a look at the files changed. You can go to this branch to verify it on your own, which is what we're going to do now before merging. You can see here the pom and then all the changes. If you pay attention to the changes that it did, it added first a global element. We are not using best practices on this one, but we have the MCP server config. It's pretty much just setting up the SSE endpoint path and the messages path, which are the default values. Then it removed completely my HTTP listener configuration that I had here, and it instead added the MCP tool listener, which is exactly what we want. And not only that, it was able to generate the whole parameter schema for me including the description. It was able to know what this MCP server was going to do. So it added the whole parameter schema, and this is why I wanted all the parameters to be there explicitly. So if we take a look at the DataWeave that we had, we are matching up everything from the `attributes.queryParameters` exactly to the same name. We could have done this differently and just done the mapping automatically — just take all the query parameters and set them — but I wanted to have them here explicitly, because then this would help us to create this whole schema, because then now it knows what exact parameters it's expecting. So it was able to create all the properties — `part`, `key`, `q`, and so on — and it even generated descriptions for all of them because it knows what they do, which is awesome. After that, it just generates the MCP responses. In this case it's just doing the output in a JSON. That's fine. And we have the same logger that we had before. But then take a look at this: when we are mapping, we were mapping from the `attributes.queryParameters` to the actual thing. So now it's going to get this from the `payload`, because now this is no longer an HTTP request or like this is not a listener — this is an MCP listener now. So everything comes in the payload as opposed to the query parameters. So this is also why I wanted to have them visually there, because you can now see this change very clearly. And that's pretty much it. That's all the changes that it did. Let's remember that I had not done MCP stuff before. So this is the first time that I know how to actually do the changes, because I hadn't done an MCP server before in MuleSoft. This is super cool because I just went ahead, told CurieTech what to do, and it did it. The first one took like two and a half minutes, this one took eight minutes. That's 10 minutes of my life that I didn't have to go through all the docs. And you can argue like, "I'm not going to know what to do now," but I will, because I can see now the code and I can see now how this is working, how this is set up. So I can now replicate it myself if I need to, because I personally learn from code. Some people might learn from videos or articles, but I learn from seeing code. So this is super useful for me, because now I know what to do. If I wanted to do it from scratch by myself, I would be able to do it. ### Testing with supergateway and MCP Inspector Now let's run this application. So if I go to my GitHub repo, I already have these instructions in the README. We have to run supergateway first. So this is the command that you have to run — just make sure that this is 8081 or it matches whatever port you changed in the project. So I already have it here: `supergateway -s localhost:8081/sse` and then the path is `/messages`. We run that and this is just going to hang there like this. So this is just listening to that. It is pretty much taking the HTTP and connecting it to an output transport of STDIO, which is needed for the MCP stuff. Then the second thing is we can verify if this MCP is actually running. We cannot just do a call like we did before with Postman — that doesn't work like that. So we can use the MCP Inspector. Just run this thing. I already have it here: `npx @modelcontextprotocol/inspector`. Run that and you're going to see something like this. So you receive a session token and you can open the Inspector with the token prefilled. So this is what I'm going to do. I'm going to copy this whole thing with the token, and note that you have to have supergateway running. So I paste this in my browser. This is what I get. Now we have to do some setups here — you can just go to the README and check that out. So transport type is SSE and the URL is going to be this: `http://localhost:8081/sse`. Again, this is a port where I am running my gateway. And if we take a look here at Configuration, we can see that the proxy session token is already there because it took it from the URL that we put there. So that's it. Click on "Connect." And it didn't work. I just realized I forgot something else that I needed you to do. So I stopped the application, and then if you go to ACB, click on the settings — let me make this smaller — from here you can scroll down. You will see this that says "Mule Design Service System Properties." So copy that, or rather from here, and then go into your Mule Runtime Default Arguments. This is kind of a workaround that I'm working on right now because I had some issues with my ACB. I don't know if you will have the same issues or not, but just in case make sure you do this. So in your default arguments, go to the end and put `-Dmule.http.service.implementation=netty`, which is the same thing that we have from here. So save that. And you can also just double check in your — not here, here — click on this here. You should see this open, and this is how my `launch.json` looks like. If yours looks different, maybe try using this. Just make sure that you have the default argument thing there, or I think you can just add it here as well, like `-m -Dmule.http.service.implementation=netty`, if you don't want to modify your default arguments. But for me I'm just going to leave it in the default arguments. So let's run this again. And we are running now. So if I make this smaller, we see the deployed. There is no need to restart supergateway or the MCP thing. I think we can just come back here to the MCP Inspector, refresh. We still have the same URL, we still have the proxy session token, and now it works. So once we have this, you can go here to Tools and click on "List Tools." And you should now see the YouTube search, or whatever the name for your MCP server is. So we have this, and now we can see that we have the `part` snippet and it comes with a default. We also have the default for the `max_results`, for the `order`, and all of the other things that I had set up by default. So this is super cool because it's already there. We don't have a key. If we don't put a key, this is not going to work. So let's click on "Run." And we receive an error because it's forbidden, because we didn't add a key. So let's go ahead and add a key. And now if I scroll down and click on "Run Tool" again, I did receive an error because there's something wrong with my implementation. So if I come here and I take a look at the logs from my local, we can see here "kind" and then "evaluating expression" — so it's the response — "no read or write handler for kind." That's okay. So let's fix that. ### Fixing the response format For the sake of time, I'm just going to show you really quickly how to fix that error. So if we go to the tool listener at the beginning of the thing, we remember we had the name, the description, the parameter schema, and then here we have the responses. So in this case we are selecting text tool response content and we are just going to output the text as is — in this case, `payload raw`. That's it. It's just going to take everything from the payload and put it in a string pretty much. You can do this way fancier to make a few Transform Messages, but I'm lazy. I'm just going to leave it like this. So once you start running everything, you can go back to your MCP Inspector. You have the key — you put your API key — and then let's search for example, let's search for "MuleSoft." You can also add a channel ID or any other stuff, but let's just leave it like that. And once we run this, we will receive a successful response and we will receive some stuff. Like here's one video: "Explore Salesforce with Topic Center and API Catalog," and we can see the channel ID. I don't know who did this. Oh, channel title Salesforce Developers. Awesome. This is probably one of Akhairat's videos. Anyway, so this is working. This is connecting. Instead of using Postman to connect to your API, we are doing this to connect to our MCP server. Cool. ### Connecting to Claude Desktop Now let's see how to use it in Claude or any other IDE. So I'm going to show you how to do it in Claude. You can use Cursor or ChatGPT or whatever you prefer. In Claude, just go to Settings. This will open this window. Select Developer, and I already have it installed, but you can just go ahead and click on Configuration. This will show you where the file is located and you can open it in any editor of your choice. In this case this is where I have it. So this is a command. Again, you can go to the description of the video to see my GitHub repo where I have this command, so you can just copy and paste. And here you have the MCP servers. This is called "YouTube," and you have the arguments to run with supergateway. In this case it's in 8081. So that's what we want. That's fine. Once you have this installed, once you save the file, you will have to quit and reopen Claude. So in Mac it's Command+Q to close all of it and reopen it. In Windows I believe you can just close the app and reopen it and that's good enough. ### Using the MCP server in Claude So now we're going to get to the fun stuff. So let's try this example: "Search for the ProstDev YouTube channel and give me the last four videos they uploaded." And I totally forgot to send the API key. All right, so: "Use this API key for the call." FYI, don't worry about my API key — this will no longer be available once you see this video. So it searched for the ProstDev YouTube channel and got their last four videos. It first tried to look for "prostdev" looking for the type channel. For some reason it didn't return, but this is something that happens with the YouTube API. Sometimes it just returns zero — that's just how the YouTube API works, I guess, because it has happened to me a few times before. So it just tried again, doing now "prostdev channel channel." It should have worked with this one, but it was just a hiccup. So I just tried a different search and it did return just one result. And here you can see it returned my channel ID. It has a title, "ProstDev." So this is the channel. So by doing this it was able to gather the channel ID. So now it can use the channel ID. So now it's using the key, the type video, order by date, channel ID this, and max results four. So it already knows what to do with it. So it returned the four videos or whatever. For example, this gives you a summary of published video ID, description. You can say, for example, "Create a LinkedIn post to showcase the first video from this list." I mean this one. "Make sure to add the link to the video in the post." So now it generated this: "Streamline your API development with AI. Just discovered this fantastic tutorial from ProstDev," blah blah blah. It gives you what you're going to learn, and this is a part one of the thing, and then "Check it out. This is the link to the video. Have you tried," blah blah blah. And I mean you can just ask it to generate more stuff, change it, "Follow ProstDev," I don't know. This is one use case that I thought of, but you can use it for a lot of different things. ### Closing thoughts So this is really cool that I was able to generate the whole MCP server from scratch without ever having done an MCP server before in my life. And I didn't know how to start. I didn't know what to do. But CurieTech helped me figure everything out. I didn't even have to read the docs. I just went ahead and checked it out. And now that I know the basics of how to use this thing, I can do even more stuff. Because, for example, I showed you we were returning the whole payload instead of doing something nice for it. So if we come here we are just generating `payload raw`. We have all these types of responses: text tool response content, text resource tool response content, image tool response content, blob resource tool response content. So there's a lot of things that we can respond with. We can add audience priority and so on. So there's a lot of things that we can do with this. But at least now I know how to do the first thing. And this was so simple to do. It's just literally connecting to YouTube to do some search using the API. So we can do more stuff. Maybe we can do, I don't know, a Netflix thing to check if it's on Netflix or not. I know Netflix doesn't have an API, but maybe we can do a web crawler kind of thing or something like that. I don't know. The possibilities are endless. So yeah, this was really cool. Thank you CurieTech for making the MCP creation way easier. And I did have to learn a lot of things on my own, like how to connect it to Claude — CurieTech didn't give me that, but you can see how this is really cool. And they're already working on an extension for ACB or for Anypoint Studio that you will be able to use CurieTech from the IDEs. So I mean this is — well, I'm not speechless. I'm speaking a lot, but this is really cool. Anyway, let me know if there is any other use case that you would like me to showcase and I am happy to do it. Remember to subscribe and follow, because I will keep doing awesome videos like this and you better be here to watch them. All right, see you in the next video. Bye. --- ## Session 0: Planning the Outline | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=xzi8peU87v0 | Page: https://prostdev.com/video/mulesoft-from-start-planning-the-outline | Series: MuleSoft from Start: A Beginner's Guide ### Intro Hi everyone, my name is Alex Martinez. I am ProstDev's founder and current Developer Advocate at MuleSoft. This part right here is a recorded message for you, but the rest of the video is going to be what I livestreamed previously. There will be some additions here and there, there will be cuts and then you will be in another place, but don't worry about it. I hope it still makes sense on the video. If you want to join me on my live sessions, you can go to twitch.tv/devalexmartinez. Or if you just want to keep watching the recorded videos, you can just go to youtube.com/prostdev and see them there. Please subscribe. All right, enjoy the rest of the livestream. ### Why I'm basing this on the book I thought since I did write this book, then maybe we can kind of take this outline, this beautiful outline that we worked so hard to do. It's written by Ariel, Akshata, and myself. So I am going to base whatever I'm going to do on this series of streams on whatever the outline is for this book. I am not going to go through everything in the book. If you want to have more of a formal education, I would recommend you to go to the book, because I will not go into details of what's in the book. I am going to kind of just focus on a practical perspective, just going in and doing the things and figuring it out. I'm going to edit all of the recordings of the videos and put them in YouTube so you can see the clean version of whatever I did on the livestream. Because if you have been watching my livestreams so far, you will see that I'm never prepared. So I am going to edit them this time so you can have a clean version on YouTube of everything that I did. But if you join from Twitch, you will be able to talk to me in the chat, send me your questions, give me any suggestions of what you want me to do. So I would recommend you to join the livestreams on Twitch. And if you cannot make it because of the schedule or whatever, then you can just go to ProstDev's YouTube channel and you will be able to find there all of the cleaned recordings. Also, if you don't want to watch the whole livestream with me and you just want to watch the edited recording, that's also okay. ### About the book I don't know if any of you are familiar with the book or not. You can find it on Amazon: "MuleSoft for Salesforce Developers." It's not only for Salesforce developers. I also have it in my background right there. It's not only for Salesforce developers, it's for anyone that wants to learn MuleSoft. You can just go ahead. It does have at the end some of the final chapters with some specifics about Salesforce, but you don't have to go through that. You can just go through all of the other chapters and you will still learn MuleSoft from the beginning. So that's why I'm going to be basing on that outline. In this first session, I guess I'm going to kind of break down what we are going to do in the next sessions so we can kind of have an idea of what is going to happen. I can check the outline of the book and I can have something planned way better because you all deserve that, because you all are amazing. ### Setting up the GitHub repo I am going to create a new GitHub repo so I can keep all of the things that I will be doing here and you can just go reference the repo whenever you have questions or whenever you want to check something, you can just go there. So let me create a new one. Sweet, everything is set. In case you don't know about this IDE, whenever you are in your repo on GitHub (let me go back to my repo) — so I am in my repo github.com/alexandramartinez/mulesoft-from-start — whenever you're here and you are signed in and this is your repo, you can press on the dot on the keyboard, literally just a dot. Press on that and this will open the web editor for GitHub. So this is Visual Studio Code in an IDE version and you will be able to edit everything from there. Or you can simply use the same URL as we had before, just changing github.com by github.dev. As you can see here, github.com, github.dev. This only happens if this is your repo and you have access to modify it. Otherwise, I don't think it's going to work. ### Planning Session 1: MuleSoft products and community Okay, so let's put the book over here. "MuleSoft for Salesforce Developers." Let's go to the outline. All right, so this is the outline from the book. Again, I'm not going to follow through everything that is on the book. I am just going to take this as an example to have kind of a structure of what we are going to review on these sessions. I don't think I'm going to do the introduction. I'm just kind of going to explain all of this on the go. But I do want to explain the different MuleSoft products. I'm not going to name this exactly. Let's just say "Listing MuleSoft products." And I'm not going to go through all of them. I am just going to give you kind of a list so you know what are different products, you can understand. Because if you are new to MuleSoft, these will be confusing. At the beginning, I also want to go through something that's at the end, because I don't want to leave it until the end. I want you to be able to know that beforehand. Where is it? Here. So the MuleSoft certifications, the MuleSoft training. I may not go through the MuleSoft training just yet, but I will go through the community stuff so you know how you can expand your knowledge. So community overview: this includes ambassadors and mentors — what are those? Meetups, help forums, ambassador forums. Yeah, that's basically it. And I may go through some trainings and certifications. Let's leave it like that. ### Planning Session 2: API layer connectivity and designing your API I guess I can kind of explain APIs, or well, you know what, I'm going to let you decide. You have one week to tell me if you want me to go through what are APIs, or if you think that's kind of understandable. Or maybe I can just go and explain it when I'm actually doing stuff and not just doing theory. Let me know if you are live, comment on the chat. If you're watching the recording, leave a comment in the video before I go through this next session next week. I guess I can explain API layer connectivity because that is kind of important. I guess. Yeah, API layer connectivity. So you can let me know before May 17, 2023. All right, I guess that's good enough for a first session. And I'm also going to be updating the outline in case this keeps changing. This is just to give us an idea, you and me, of what we are going to be going through when we are doing this. All right, then designing your API. So basically yes, that's everything that I'm going to do on the second session: designing your API. I was thinking of an example of kind of like a restaurant, because it goes with the analogy of what is an API. So maybe we can create an API of like, imagining that we have this restaurant, we have the clients, we have the waiters, we have the cooks. And so a client comes and then asks for the menu and then gets the menu. Yeah, that sounds like something interesting. Design an API: restaurant. And we are going to start from creating the idea of what the API is going to do, like shaping it, and then actually creating it in a MuleSoft product. So that's what we're going to do. So I think: write down requirements, design the API spec in Design Center, test with the mocking service, publish to Exchange. I don't think I'm going to go through all of this. I'm not going to explain the difference between SOAP and REST, for example. I'm going to be very practical. I'm not going to be super theoretical. So again, if you want theory, it's better if you go to the books. And also another thing that I wanted to tell you: if you just want to get started and do some stuff, let me search for MuleSoft Tech Zone. So if you go to MuleSoft Tech Zone, you will be able to find Shravan's videos and stuff. Shravan has already some playlists: Mule 4 tutorials for beginners, MuleSoft for beginners. There are 13 videos right now and he is currently doing more videos. So if you just want to get it over with, just go with Shravan and see whatever he created on these videos. I'm sure it will be super helpful. It has a ton of views. For example, the session one has 369K views from two years ago. So that's a lot of views. So you can just do that if you want, if you don't want to wait. Again, just giving you some more options. All right, so query parameters. Yeah, I'm not going to go through the theory of that. I'm just going to show you how to do it practically. API design, naming conventions. Yeah, I'm kind of going to tell you all of the tips and tricks and all of the best practices while I'm going through the things. I'm not going to tell you, "okay, these are the best practices blah blah blah." I'm just going to be like, "yeah, we're going to create this API specification, we're going to do this, we're going to do that, and oh by the way, this is how you name the things," stuff like that. I'm not going to go through the specifics of OpenAPI spec in RAML because I'm going to be using the visual designer. So if you want to go deeper in that, I suggest you check other videos. Getting started with API design, or maybe I will. I don't know, I'm just talking out loud here, thinking out loud. API design, yeah, we're going to do that. All right, and then once we have our API spec published, yeah, publish to Exchange. ### Planning Session 3: Implementing in Anypoint Studio I'm going to go now to Anypoint Studio: implement an API. So there is, you're supposed to first design your API and then implement it, but you can kind of just create an integration or implement your API directly without actually having to design it. So I'm going to show you different routes on how you can do that. Let's just hope that it works because the demos sometimes don't work in the livestreams for some reason. I'm not going to go through downloading and installing, but I'm going to show you some videos that you can — I'm going to kind of prepare before I come to the livestream. I'm not going to be just like, "here it is," but I'm not going to prepare the technical things. I'm just going to prepare what I'm going to go through so I can give you videos and stuff. Because I'm definitely not going to go through all of these: building, running, testing. Testing, well, testing locally, I guess. Implement an API. Yeah, we're not going to do a unit test here. We're just going to test that it works locally, like manually test it. Updating the thing in Anypoint Studio. Nice, I didn't write this chapter, by the way. Yes, okay, we're going to go through Studio, kind of give a perspective of everything that is there in Studio so you can understand what that is, how does that IDE work in Studio. And I can also give you a brief maybe summary of Anypoint Code Builder because it's in beta. It's still in beta. A lot of people don't like it because it doesn't work yet. Well, yes, it's in beta version. It's not the final version yet. I'm going to say that a lot because you need to understand that: yes, a lot of things don't work. Yes, a lot of things are broken. There are a lot of bugs. It's a beta version. It's expected. So just leaving it out there. All right, the core components. I'm kind of going to go through them here in Studio: core components. But again, just a summary of what they are. I'm not going to go through all of them practically. I'm just going to show you the ones that I'm going to need and I can quickly tell you what the other ones are for. So if you want a deep introduction in all of the components, then you can just see the book or see other articles, other videos from other people. I'm sure there's a lot of stuff out there. And if there's not, feel free to tell me. And as you can see here, Chapter 4 goes through it all. It goes through components like the logger, set transaction ID, endpoints, error handling, flow control, scopes, transformers, etc. ### Planning Session 4: Anypoint Platform, Runtime Manager, API Manager And then Anypoint Platform. Well, I'm kind of going to show you Anypoint Platform in the first session: MuleSoft products. Anypoint Platform, Anypoint Studio, Anypoint Code Builder (beta). I can kind of tell you about Composer, but I'm not going to go through it because I don't have access. And MuleSoft RPA, I guess I can also show you about DataWeave, and this includes extension for VS Code, Playground. Yeah, that's it. Okay, core components: brief summary. And API. So I'm going to talk about Exchange here because we're going to publish to Exchange. I'm also going to go through Design Center there in this session because we are going to design an API there. Runtime Manager. Yeah, we're going to do Runtime Manager, API Manager, and the rest on this session — no, on the next session, fourth session. And this outline can change. It's not written in stone. So if for some reason I decide to go longer on a session because there's a lot of information, then we can just move something from that into the next one and so on. I can also answer any questions that you may have for me online or live when we're live. So we're going to go through Runtime Manager, API Manager. And you know what, let's also do CI/CD here because I'm going to show you how to deploy. This is deploy API to CloudHub. I'm going to show you how to deploy this manually in different ways, but I also want to show you how to deploy using continuous integration, CI/CD pipelines. So we're going to do CI/CD with GitHub Actions because I've been doing a lot of content with GitHub Actions lately, so I'm kind of aware of what happens here. So we can set this up in that way we don't have to keep deploying the API manually every single time. So there's that. I guess if I do this, I will have to talk about Maven and secure properties, secured encrypted properties. I don't know if I'm going to have — again, because I'm going to follow all of those practices as much as I can — I don't know if I'm going to have specific things like username and password or stuff like that. So if I do, I will have to encrypt them and I will show you how. But if I don't, I'm not going to go through that. I'm going to try to go through it because there are a lot of things there. Access management, Visualizer, monitoring. I'm not going to go through those. Access management, I can kind of show you really quick what it is, but no. Access management is going to be here on Anypoint Platform. All right. ### Planning Session 5 (maybe): DataWeave deep dive All right, then we have the deep dive. There are two chapters for DataWeave because there's a ton of things to do with DataWeave. But I'm not sure if I'm going to go through all of that. I guess I'm kind of going to go through some DataWeave stuff while I'm implementing the API. Oh boy, we'll see. We'll see if this is going to be our fifth session. Well, you know what, yeah, let's do a DataWeave: just DataWeave. We'll see what we can do. I'm going to try to start from the beginning, assuming you don't know anything about functional programming or stuff like that, so we can all be on the same page. ### Planning Session 6: Testing and deployment Building your Mule application, deploying, securing. Well, I'm kind of going to go through that in the other ones because I'm not going so deep. I don't need to do a ton of that. Okay, sixth session. Now let's do a unit test: M unit manually and M unit CI/CD. I can also do Postman so we can kind of see, yeah, we can maybe create some tests from Postman as well. What was the other tool? Was it BAT? Yeah, it was BAT: functional monitoring with the BAT CLI. That would be interesting, but I don't know exactly how to use it yet. Like I went through some stuff in the playground, but I haven't used the CLI. I can kind of mention it, but I don't think I'm going to go through it. Okay, mention BAT CLI. All right. I also haven't used API Test Recorder. Maybe we can kind of try it out a little bit in M unit. And then we have Salesforce integrations, which I'm not going to go through. Connectors and use cases — yeah, also not. I'm just going to show you how to do a simple API from the beginning, like from not knowing anything about MuleSoft. I'm going to show you how to do that, but I'm not going to go through the advanced stuff. I don't think so. You can buy the book and go through all of them, or you can check out, for example, Shravan's videos. I think Shravan is also doing the beginner stuff only and he's creating the advanced videos just now, but he hasn't uploaded them yet. Best practices, and then we already did all of this. Okay, I don't think these are going to be six sessions only. I think we're going to start checking that we're doing more and more and more stuff. But we'll see. We'll see if we can keep following the outline or if we get stuck. ### Recap and feedback Let's take a pause and let's just digest what I just said. You can let me know if this is not what you were expecting, if you want me to do something else. Who knows, maybe we do only six sessions and maybe I have more time to do more stuff. But remember, each session is going to be just an hour per week, so it's going to be very little information but throughout six weeks. So that's going to be good. I'm going to tell you why in a moment. ### Why spaced repetition learning works There's this thing called spaced repetition learning, where basically, instead of — for example, you have the "MuleSoft for Salesforce Developers" book — instead of going through the whole book in one day or in one week, you can create spaced repetitions of what you are learning. In that way, it will kind of stick more in your head. I really like this topic about learning and how your brain works, how the chemicals work, how the neurons work. I don't know the super deep scientific things, but I like to learn about that because it's, for me, super interesting to understand how we as humans learn different things and how we have, for example, different methods of learning. I am a very visual learner. Like you just saw that I had to break down everything into bullets because that gives me a sense of groups and I can see the different groups and what we are going to do. I cannot read a whole paragraph and understand it. I just can't. I have to practice it or I have to make a diagram or I have to see it in a different way, because I'm very visual. There are other people that are very auditory and they like to hear the things. So maybe they put a video and they just hear, or maybe a podcast and they're just hearing things, and that's how they absorb information. I do not work like that. I cannot sit through a whole podcast episode. I try. I can do it sometimes, but I get distracted very easily. My mind just wanders around because if I'm not watching something, because I'm visual, I cannot understand it. I cannot digest it that easily. This all came from a book that I read about how we learn, which is super, super good. And it basically explains different perspectives on how we learn. It explains the scientific perspective, like the neurons, serotonin, blah blah blah. It explains the practical perspective, like creating different schedules to learn, going for a run, going for a walk, etc. etc. And there's also the teaching perspective. So if you're a teacher, how should you teach your students so they can retain as much information as they can? It's focused on teachers for kids, like in elementary schools or stuff like that, but it still applies. I mean, we all learn. Maybe we learn differently, but we all learn. The only difference between kids and adults is that supposedly — I'm not 100% sure — adults have a better attention span than kids. I have seen very distractible adults in my life, so I'm not sure if that's true. But that's the only difference. Everything else works the same. And basically, how you learn is based on neural networks, right? So you have different neurons that are connected and they emit signals, let's say. And if that link is connected a lot of times, then the information that is passing through that link is going to stick with you. So what happens in spaced repetition? Going back to the beginning of my point: in spaced repetition, you are learning different things on different days, but it's spaced. So for example, we're going to be doing this one hour every single Wednesday. One hour is great because our attention span is going to be around an hour, less than an hour, maybe 20 minutes. So I'm going to do the livestream in an hour, and then the edited version is going to be less than an hour. So for people that are watching the recording, it's going to be great because I can just watch the digested version of whatever happens here. And if you do that every Wednesday for six weeks, seven weeks, eight weeks, that is going to help you to retain the information not faster but stronger. Because you are not — for example, if you go through the whole book in a week and then you never touch that information again in your brain until a month after that or something like that, you're going to forget whatever you did because you acquired the information so quickly that you didn't have time for your brain to actually create the links or actually absorb all of the information to pass it from your prefrontal cortex to the neocortex or something like that. You have to pass it to where your long-term memory is and not just your temporal memory. Or you have to pass it from the RAM to the CPU, if you want to look at it in a different way. So if you keep doing this repeatedly but with spaced amounts of time, then you will be able to keep forming the links. Because you form the first link and it's weak and it's going to start fading, but then you form it again and it's going to start fading, but then you form it again, and so on. So every time that you keep forming it, it's going to be stronger and it's going to be there more time. Like you know that saying that once you know how to ride a bike, you never forget how to do it? You can just pick up and do it. That's supposed to be because you did it so many times for a long amount of time. If you do know how to ride a bike — if you don't, that's totally okay — you did it for so long that those links just stayed in your brain forever. So even if it's been 10 years, 20 years without using a bike, once you take it, it only takes you maybe half an hour, maybe less, to pick it up and go at it again, because your links are so strong in your brain that you already know how to do it. So you can just pick it up and do it. So that's what happens with spaced repetition. I'm not going to tell you that once you know MuleSoft you're going to know it forever, but it's going to help you definitely to have the spaced repetition to be able to learn those things separately to form stronger neural links. That's all I wanted to say about that. I'm really passionate about that topic, about learning and learning how to learn. It's, I really, really like it. ### Wrap-up That's all for this livestream then. Thank you so much for coming. I will see you on May 17, next week. Please, you have seven days to let me know if there is something that you want me to change from the outline, or if you just have suggestions, ideas, or comments for me. I am happy to take them into account. I hope that my throat is better. I really do, because this is just annoying. But I've been sick for like three or four weeks. I didn't want to keep postponing my livestreams. So that's all then. Thank you so much for watching, and I will see you next week on twitch.tv/devalexmartinez if you want to see me live, or ProstDev on YouTube if you want to just watch the recorded videos. You can also go to alexmartinez.ca to see some of my portfolio, my links, and my stuff. Or you can go to alexandramartinez on GitHub to follow through whatever I'm doing in these sessions. All right, that's it then. Thank you so much. Thank you for the follow, and I will see you in seven days. Bye bye. --- ## Session 1: MuleSoft Overview | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=I6BWPoD639A | Page: https://prostdev.com/video/mulesoft-from-start-mulesoft-overview | Series: MuleSoft from Start: A Beginner's Guide ### Introduction Hi everyone, Alex here. This is session number one of our MuleSoft from Start series. If you didn't watch the last video, that's totally okay because we were just planning the outline. We didn't go through any actual content. This video was run live and I have edited it so you can see just the best things that happened, so you don't have to go through the whole hour. I hope you enjoy it. We didn't go through any practical stuff yet, but I did go through a lot of very useful information for you: an overview of all of the MuleSoft products, an overview of the community, the MuleSoft community, what we are, what we're doing, and also a little bit about training and certification. So if you want to go through this video, you will learn a lot about the overview of the products and what we will start learning in this series of videos. ### Anypoint Platform Overview If this is your first time coming into MuleSoft, you might be wondering what is the whole thing? Is this just Anypoint Platform which is on the web or do you also have an IDE or what's going on? It can be overwhelming. So I am going to give you an overview of all of the MuleSoft products. We have Anypoint Platform, which is the one that is on the web. If we go to anypoint.mulesoft.com, as you can see here, you can create a new account. If you don't have an account, sign up. Just fill up your stuff. I'm not a robot. I agree. Accept and create. Before I create it, I want to mention something that is very useful for a lot of people. It's a trick. You can just keep creating free trial accounts as much as you want. Just use the same email, but use a different username. That's all. I have so many trial accounts. As you can see here, this is my 19th one. I'm just using the same email. The rest of it doesn't really matter. Just use a different username and you can keep creating free trial accounts. Now that we're here on Anypoint Platform, this is the first product that has a lot of products inside. So this is the product that we can access from a web browser. If I go here, you can see here the menu of all of the different products. There are more products that you cannot see in this list because this is a free trial account. If I were to use an enterprise account, I would be able to show you more products, but I'm not going to do that right now because you're just getting started. So you will have a free trial account just like me. ### Anypoint Platform Products Here is Code Builder. I'm going to go through that in a little bit. Just know that it is there. Design Center is one of the most used ones. You can kind of guess what every product does from their name. Design Center helps you to design stuff, APIs, but we will get into that. I'm just going to read through the list for now. Exchange is kind of an app store that you have here in MuleSoft. So you can download connectors, you can download APIs, examples and so on. It's just like an app store. We have DataGraph. So if you are familiar with GraphQL, this is our version of GraphQL. You can use GraphQL here on this product, which is DataGraph. Then inside Management Center, we have different products. Access Management is where we can control all of our users, all of our organizations, permissions, applications and so on. You can go here to manage all of that stuff. Normally just the admin of the account will have access to it. Because we are in a free trial account, we are the admins, so we do have access to it. API Manager is where you can secure all of your APIs. You can put different security policies. You can also add contracts. You can add SLA tiers and so on. Again, I'm going to go through all of them in more detail, but first I'm going to go through the whole list. We have Runtime Manager, which is where you run all of your APIs. So all of your APIs will be running here. You can connect them to API Manager to secure them and we will see how I think on the next session. We have API Governance, which is kind of a new product. In this one we are able to set standards for all of our different APIs. Then we have Visualizer and Monitoring. Visualizer is where you can see kind of architecture diagrams, where you can see how your applications are connecting to each other. I'm not very into Visualizer and we're not going to see it throughout this series, but it's important that you know that it's there. You can read more about it in the docs. Monitoring is where you monitor your APIs, like the logs, any spikes on CPU, stuff like that. More technical. And Visualizer is more architecture, kind of an overview. Finally, Secrets Manager is where you can keep passwords or keys. You can keep keys, you can keep important files, stuff like that. So that's basically Anypoint Platform. You can know more or less what you can do. Everything is in the cloud here. So this is the first product, Anypoint Platform, which has a series of products, and some of these products even have more products inside. Like Design Center, for example, has different products inside where you can create APIs, you can create a quick Mule flow, you can create Async APIs. So all of those are different products where you can create API fragments, API specifications, you can mock them, and the mocking service is a different thing. But all of this umbrella is Anypoint Platform. Everything that is on the cloud, everything that you can access through your web browser, this is Anypoint Platform. And again, it's located in anypoint.mulesoft.com. ### Anypoint Studio We also have Anypoint Studio, which is the IDE that you download into your computer in order to actually develop the APIs. So Anypoint Platform is more for management of your APIs, designing your APIs, securing your APIs, monitoring your APIs. But the actual implementation will go in the IDE, which is Anypoint Studio. And in the future, also Code Builder is going to be available. Anypoint Studio is where you implement your APIs. You can also design APIs here, but I really recommend you to use Design Center instead, just because it's easier. The UI is just way easier to understand. Here in Studio you kind of have to know what you're doing in order to design the API here, whereas Design Center is going to help you and guide you through it. So it's great if you're new to it. As you can see from the menu here, we can create a Mule project. We can create a Mule project, create an API specification, open a template from Exchange, create a Mule domain, open an example from Exchange, and Java stuff. We can do that. We can also import a project that already exists. So we can open it here or we can get an API specification from Anypoint Platform, put it here, develop our application, and then we can deploy it from here to Anypoint Platform with just one click. So it's great. ### Anypoint Code Builder The idea is that Code Builder is going to be the next IDE. As you can see, this is in a beta version right now, so it's not very stable. It still has a lot of issues, and we recommend you don't use it for production environments or for your actual work. It's more of a beta version. As of May of 2023, it is used only to play around with it so you can see the new capabilities that are going to be available in the future using Visual Studio Code instead of Eclipse. But I don't think we're going to go through it because it's in a beta version and you're just learning. So we are going to learn using Studio eventually, not today. That is Anypoint Studio or Studio for short. Anypoint Code Builder I just mentioned. Right now, Code Builder is available only from Anypoint Platform. So if you click on it from here, you're only going to be able to use it from your browser, which is kind of a bummer. But coming soon you will be able to use it in your desktop. As you can see here, Code Builder Desktop, this is not possible yet. But you can use it from the web for now. And in the future, it's going to be available both ways. So you can either use it on the web or you can use it on desktop, which is super cool. But I think until it's available on desktop, I am not really using it a lot because it makes things harder to do. But it's great to play around a little bit, check it out and see how this is going. ### Composer Then we have Composer. I'm not going to go through it because I don't have access to it, but just so you know, Composer is kind of like, I don't know if you're aware of IFTTT, IFTTT.com, if this then that. Here you can create flows to kind of go through different things, create integrations. For example, you can integrate Alexa, you can integrate your Philips with whatever, you can do different things, you can integrate a lot of stuff, and you don't have to code. This is a great thing. This is low code or no code. This is no code, actually. You don't have to code at all. You can just go and click stuff and create integrations with it. There's also SAP here that does basically the same thing. You have different products that you can use and this is more or less how it looks like. So you create flows and then you create a workflow that is going to run through different products. You can create integrations like that or automations as well. In our case, we have Composer, which you can access through Salesforce, through your Salesforce account. It does have to be an enterprise account. So you do have to have a contract with Salesforce or your company. It runs MuleSoft underneath, but on the outside you only see Salesforce and you are able to create flows and do stuff like that. If you do want to learn more about Composer, you can go to meetups.mulesoft.com, which I'm also going to talk about in a little bit. You can search for Composer and here you will see different events that already happened or there's a group for Composer and RPA. You can go check out the past events. A lot of them have videos, so you can just go through the past ones and see if there's something there. I'm going to go back here to the meetup site in a bit. ### MuleSoft RPA Then we have RPA. RPA is robotic process automation. MuleSoft RPA is kind of a new product as well. Salesforce bought Service Trace. I don't know if you know about that. Service Trace was an RPA product. So since Salesforce acquired Service Trace, it went through a redesign and now it's MuleSoft RPA. We're not going to go through it because I don't have access to it and because I believe it's only available for a Windows machine right now. So I cannot go through it. But there are some videos about people doing this. You can also go to twitch.tv/mulesoftcommunity and if you go to videos, if you go to the videos tab here and scroll down, you will get to the Composer and RPA section. And there are four videos here if you want to check out some demos about how Composer or RPA work. I think these two are Composer and these two are RPA. If you want to check out RPA, I would recommend doing this one, which is from people from the training team. That's going to be super useful. So there's that and also you can go to the meetup to see if there's something about RPA there. ### DataWeave What is DataWeave? DataWeave is a programming language that we have right here in MuleSoft. We use DataWeave for everything that we need. MuleSoft is a low code tool and it's not no code because it does have code and that code is DataWeave. If you are familiar with functional programming, then you're going to be great with DataWeave. It's the same concepts. They just translate from one programming language to another as long as it's a functional programming language. But if you come from an object-oriented programming or structured programming like me, then it's going to be a little bit harder for you to understand it because it's not just understanding the syntax of the language. It's also changing your mind, having a different mindset to be able to create code in this paradigm. So it is going to take you a while if you have never done functional programming before, but it is not impossible and I am the proof and obviously there are a lot of people that have done this. Inside DataWeave, there are two different products that you can use to develop in DataWeave besides the ones that we already said in Anypoint Platform and Studio, because DataWeave is also there. We have the extension for Visual Studio Code. So if you go to your Visual Studio Code and look for extensions, you can search for DataWeave and you will get to this. This is also in a beta, but it's pretty good. I use it almost all the time. There's also a DataWeave Playground. So if you go to dataweave.mulesoft.com here, you can find the links to different stuff from DataWeave. So you can see why DataWeave, what you can do with DataWeave, how it looks like. You can get started learning the fundamentals, executing in any terminal. Oh, I forgot about that. Wait, there's also the DataWeave CLI. So yeah, you can go here, go to the CLI, check it out. You can develop in Visual Studio Code as I told you. You can install the extension here and you can also join our community. So you can check out the questions that we have in Stack Overflow. You can ask any questions in Stack Overflow. You can also join the Slack workspace that we have for DataWeave and you can also raise issues in GitHub. So you can go check that out. Here are more links and that's basically it. ### Community and Ambassadors Then we have the community overview. What are ambassadors and mentors? Where are the meetups? Where are the help forums? I'm going to go through all of that. So ambassadors and mentors are our best people in the ecosystem. Let's go to the ambassadors. So you can go check out all of the current ambassadors and you can also get to know them and their channels. If you click on someone's profile picture, you will be able to find here their bio and their links. So if you want to connect with them, you will be able to do that through here. For example, Edgar. If you go here, you will see that Edgar has a lot of links because he does a lot of things. Here you can find his LinkedIn, Twitter, YouTube, ProstDev, and blog. So if you go to his blog, for example, you will be able to see everything that Edgar writes about and you will be able to find a ton of information from Edgar. Or you can just go through every single person from here, follow them on socials, on their blogs, on their YouTube channels, and a lot of them will have very, very useful content. Some of them don't focus on creating content like they don't create blogs or videos, but they are super helpful in the forums. So here are the MuleSoft forums. You can ask stuff. All of these people or most of these people that are answering you are people from the community because they love to answer questions and they love to help people. And a lot of these people that are top contributors to the forum are our ambassadors. So not all of the people from the ambassador side are going to be doing blogs or videos. They can also be helping newbies through the forums or through Slack or through different kinds of channels, but they are the ambassadors for some reason. They were chosen because they are top experts on what they do and they really love to help people. You can apply yourself. So if you want to become a mentor, which is the first step to become an ambassador, you can apply here or you can nominate someone if you think they're very cool and they're helping a lot with the community. You can nominate people. So there are going to be a lot of mentors and fewer ambassadors because ambassadors are the top people. Then we have the mentors, but everyone is so friendly and so good and we all just want to help you succeed. So there are a lot of mentors, a lot of ambassadors. You can also go to LinkedIn and search for MuleSoft ambassador, MuleSoft mentor and you might be able to get the people because a lot of them have that title on their LinkedIn description. So you might be able to find them there as well. ### Forums and Meetups Finally, the meetups. I already went through the help forums. So let's go through what are the meetups. So again, if you go to meetups.mulesoft.com, you will be able to find this beautiful page where there are a ton of groups to join and meetup nearby where you are. You can scroll down here, search for a city, country, region, or you can explore by region. A lot of these groups are still doing virtual because when it started, it was pre-pandemic and we were all doing just in-person meetups. But then the pandemic started so everyone was doing virtual meetups and now people sometimes have in-person, sometimes have virtual, sometimes have hybrid. So you can just go here and check them out. There are a lot of them that are online. So these ones never meet in person because they are all over the world. But you can also explore by your region and you can find whatever groups are nearby you. For example, I'm from Canada. I can select all of the ones that are in Canada and see if they are going to be hosting a virtual meetup. Then I can just join the virtual meetup. Or for example, I am based in Niagara Falls. I can join the New York chapter in the US and I will also be able to attend if they have virtual or maybe in person. Like who knows, maybe I can go. And so on. You can just join whatever groups you want. You just click on it. You get here and as simple as that, you just fill your first name, last name, email, click on join and you will start receiving notifications in your email as soon as upcoming events are posted. You can also check out the past events. If it says virtual meetup, it will most likely have a video. Maybe not. Yeah, there's a video. So you can also check out all of the recordings. And as I told you at the beginning, you can use this search bar here and just search for a topic or a meetup, a country, a city, whatever, and you will get your results. ### Training and Certification Now we are only missing training and certification. This is a tricky one. I'm not going to lie. If you go to training.mulesoft.com, you will be able to find different training certifications. So if you go here to courses, you will be able to find a course catalog. A lot of them are free. Anypoint Platform Development Fundamentals. Learn material at your own pace on your schedule. Yeah, you just have to sign up. You don't have to pay, I believe. So yeah, you can go through that training. For example, if you want to go through a very formal training that goes through all of the things, it's supposed to be 5 days, but assuming that you do like 8 hours a day, so it might take you a little bit more. And that's all right. You can use a free trial account for all of this. Everything will be available for a free trial account. So you can go there and try that. And then for the certifications, I think once you do that training and also if you follow everything that's on the book plus the training, it is most likely that you will pass your certification. The first certification is the MuleSoft Certified Developer - Level 1. So it's 2 hours. It's virtual. It is two hours, by the way. You may be thinking like maybe I'm not going to take two hours. Yes, it is two hours. It is like 60 questions. So it's like 2 minutes per question. You are going to take that long. So make sure you are prepared for that. The Level 2, I haven't checked that out, but I believe it is kind of hard. So make sure you go through all of the prerequisites. If you click on each certification it will have, oh yeah, and the certifications are not free. So you will have to pay for that. I don't know if the training is still giving free certification vouchers or not, but you can try to check that out. ### Quick Tour of Design Center, Exchange, and More Before we finish, going back to Anypoint Platform, let's check out. We're going to go through Design Center on the next session because we are going to start designing an API. So, oh my god, it changed. Anyway, once you are here inside Design Center, you can create a new API specification, which is like the contract that you have between APIs. I will explain that in the next session. You can create a new fragment, a new API, an Async API, or you can also get from an existing source. That's basically what you do with Design Center. I will show more examples in the next session. If you go to Exchange, as I said, it's kind of like an app store. So here you can filter whatever you want to find. So you can find connectors, DataWeave libraries, examples, policies, API spec fragments, REST APIs, RPA, activity templates, rule sets. This is used for API Governance. So APIs and templates. You can search for all of these. We are also going to be using more of these in I think the third session where we actually implement an API in Studio. But here you can find more stuff, kind of like an app store, and you can also go here and see the documentation. So that's really cool. Next we have DataGraph. As I said at the beginning, it's kind of like our version of GraphQL. You can start building your unified schema. There are some start guides, some tutorials and some concepts that you can go through. If you haven't used this before, you can also go to YouTube and find some channels to use this. I am not going to go through DataGraph in this series of videos because that's kind of more advanced or more to another topic. This is not on REST APIs, which is what we are going to be doing. So you can go check that out. Access Management is where we have all of our stuff like all of the admin stuff. I'm going to click on try new features because I like that. So you have the users, you have your teams, you have roles, environments, business groups, identity providers. If you want to add more client providers, connected apps and so on. So you can find here different things but mostly only the admins have access to it. API Manager, which we are also going to look into it more in our fourth session where we deploy to CloudHub and check out some more stuff. We're going to look more into it there. But here is where you have APIs and you can secure them. For example, here you can have automated policies, client applications, custom policies and this is to secure your APIs. Then in Runtime Manager is where you are actually running your API. So for example, here in Sandbox, if I were to deploy an application, I can just deploy it here and this will be running in CloudHub and I will be able to connect this API that is running in CloudHub or in Runtime Manager with the API that I create in API Manager so I can secure them and I am going to go through that in the fourth session so we can get more familiar with that. API Governance. I'm not going to go through it here, but it's super cool because this helps you. Or maybe I can go a little bit through it in the next session depending on the time. You can add some basic stuff like rule sets. Let's do super quick. For example, Anypoint best practices. So with this you can make sure that all of your APIs have all of these standards and that they are running exactly how you want them to run. So this will help. Async API best practices, OpenAPI best practices, required examples and so on. You can select or create any more rule sets that you want to apply and you will be able to see which APIs are compliant and which ones are not. So that's very useful. Visualizer is where you can see from an architecture perspective, as you can see here. So you can see the architecture view, troubleshooting view and the policies view, for example. I don't have anything running so I have nothing here. But this is basically how it looks like. Monitoring is where you go and check if it's running properly. If it's not, I don't have any resources running right now, so I'm not going to be able to see them. You can also visit the documentation to learn more. And the Secrets Manager if you want to create stuff. Let's do test. You can have key stores, trust stores, certificates, shared secrets, TLS context, so on. So that is useful for that. I'm going to remove this and I'm also not going to go through it in this series of videos. But that is your summary of all of the products. ### Wrap-up So we finished the first session. Yay. All right, we made it. And that's all for this video. We did it. That's all for this video. I hope that is helpful. That is all for the first session. It was mostly theory as you can see, but it will kind of give you an overview of what is MuleSoft and what you can do with the different products or where you can find the different things. Of course, it's not everything, but it's a nice first guide for you to have so you are not that lost while you are navigating this confusing ecosystem. So on the next session then it's going to be on May, I'm going on vacations, so it's going to be on May 31st. So not this Wednesday, the next Wednesday. And we are going to learn more about designing an API. What is an API? What is API connectivity? How to design an API? We're going to design a restaurant API. Write down the requirements. Design the API spec in Design Center. Test with a mocking service and publish to Exchange. Remember to go to github.com/alexandramartinez/mulesoft-from-start so you can find the resources that we have been talking about or doing in this series of videos. So the next session, May 31st, 1:30 p.m. Eastern, we will start doing actual stuff. All right. Thank you so much for coming and I really appreciate it. And if you're watching this from YouTube, you rock. Thank you for watching. All right, I will see you then on the next video. Bye. --- ## Session 2: What is an API? and API-Led Connectivity? | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=M4gYW2o9IKc | Page: https://prostdev.com/video/mulesoft-from-start-what-is-an-api | Series: MuleSoft from Start: A Beginner's Guide ### Introduction Hello there, Alex here! So you are here because you want to learn more about APIs, and that is exactly what we are going to be doing in this video. We are going to learn the basics of APIs without getting too deep into it, and also learning what is the API-led connectivity approach from MuleSoft. Now remember, this is a live stream and the only thing that I did was edit it down for you to make it shorter, more compact, and easier for your digestion. So I hope you like this video and I will see you in the next session. Now let's watch it! All right, so if you go to my blog prostdev.com and then you go to the part here that says blog, you will find the list of all of the things that I've done. If you go here to others and scroll down, you will find the Understanding APIs series. So this is the series that I'm going to go through right here. The first one, and I'm also going to put the links there so you can have them and follow with me if that's better for you. So the first part is basically just getting started from the beginning. What is an API? If you have absolutely no idea what an API is, this first part is basically trying to give you some basic idea but not really getting into the details. So I'm going to go kind of quickly through all of these posts because I just want to show you the bigger picture of what we are going to be doing, but again you can go through them on your own and that is completely fine. ### What is an API? In summary, this is how I look at it. We have here the implementation of the API, or the API if you want to just look at it that way, and this implementation can be created in a bunch of different languages, right? In MuleSoft, in Ruby, Python, PHP, Node, you name it. So this API, for example, the API that we're going to create is going to be created on MuleSoft, but it can be created in a number of languages or tools. And the fun thing about APIs is that you don't know and you don't care what they are made of when you are using them. So for example, if I were to use the Twitter API, I don't care if it's created on Java, on Python, on Ruby, whatnot. I am just going to be sending something and receiving something, and that's the fun part. All of these APIs can be created and managed however they want, and the only thing you need to worry about is how to call it and what you're going to receive in return. So here I wrote that it basically has like four different aspects to it. It has inputs, so you send something to the API, or it has an input that it receives. Something inside the API, it has operations, which is what you want the API to do with the input that you sent. It has outputs, what is going to return to you, and it has data types. So what kind of data are you going to be sending or receiving back? It has of course a lot of different things, but this is just a quick intro to it. So for example, we have request and response. Whatever you send to the API is a request, and whatever you get back from the API is a response. So now putting it all together, I came out with this diagram that I like very much because it's simple. And if you already know about APIs, this is way too simple, but if you are new to APIs, this is perfect because you're just getting started. So this is the way that I look at it. You, please feel free to create any other diagram that makes you feel better if it improves how you look at it. This is just how I look at it. So we have the implementation of the API, or the API itself, whatever you want to call it. Here you have the request. You send the request into the API and you get a response back from the API, right? So in the request you send your input along with the data type and the operation. We're going to see some examples a little bit, or analogies, so this makes a little bit more sense. So you send your input data with the data type. It can be a JSON format, a CSV format, a number, I don't know, whatever. And the operation, which is what you want the API to do with that information. And then you get a response which is the output with the data type. So this data type can also be JSON, CSV, XML, a number, or it can also be like nothing. It's just like something that comes back to let you know if the operation was successful or not, or like what happened inside the API because you cannot look into it. ### Analogies and Examples Part two is the analogies and examples. So let me put that in the chat. All right, so this is a diagram that we came to, and then the first analogy is a restaurant. For example, here we have another diagram. You are the client, right? And you arrive to a restaurant and they give you a menu, or you take the menu and you take a look at the menu to see what is available for you to request, right? You cannot request anything that is out of the menu because they won't have it. They are literally listing whatever is available for you to order. So you take a look at it and then you make a request to the server and say, "Hey, I want a hamburger," for example. And the server is going to do their thing. Like, you don't know what the server is doing. You don't know if they're going back to the kitchen, if they're writing it down, if they are just telling it, if they are inputting it in a computer. You don't know what the server is doing, right? And you don't care. You just care that you requested a hamburger and that you get back exactly what you requested, which is a hamburger. So you make a request with your input and your operation, I guess. You make a request. You say, "I want a hamburger with, let's say, pickles," or "no pickles," "with no onions," "with extra cheese." I don't know, you give it options to be personalized to you. The server is like, "Okay, I got your request, I will be right back." They go, and then they come back with your dish. And that's it. That's an analogy of an API because you are making a request and you're getting a response, and the response is hopefully the same thing that you asked for. And you also get the menu saying, "This is what's available for you to order." That's pretty much it. So the server would be the API, and you are requesting something and getting something in return, which is a response. Then we have a calculator. You would also probably wouldn't care about what the calculator has on its inside. You don't really care about what happens when you push a button or what happens when you put some batteries on it, whatever. It functions in its own way and you don't necessarily know how it functions on the inside. You just know that if you press some buttons, you get something in return, right? So in this case, let's say that you input 2 + 2. So two and two would be the inputs, and the operation would be the plus because you're telling it what to do with these two inputs. With 2 + 2, you can also say 2 - 2, and it's the same input, just a different operation, right? So this is what you are requesting to the calculator, and the calculator, once you press equals, it will output a four. And in this case we also have data types because two, two, and four are numbers, and that is our data type. Our request data type is a number, and our response data type is also a number. And as we saw from previously, we have input with data type plus operation. We have inputs, data types, operations, and at the end we have the output with data types, which is output, and the data type is a number. Here's a request, here's a response, and we don't care what's inside the calculator. We just care that we send something and we get something in return. That's pretty much it. Then we have some like more real life examples, I would say. So for example, if you have an HR system and you have listed everything in a CSV format, I hope you can see, you have everything listed in a CSV format. You have the employee ID, you have the employee first name, the employee last name, and then you send all of these as a request and say, "Get the employees' first day of employment." So then the API returns this CSV, which also contains the same things that you sent in the request, but it adds a new field at the end, which is the first day, and there you have the dates. So that is another example. Here's another example. That was a CSV. This is a JSON. So here you send this as a request. This is your input, this is your data type, which is JSON, and the operation is, "Can you please get the employees' first day of employment?" which is the same but with a different data type. And the API will respond another JSON with the same information that you sent: employee ID, employee first name, employee last name, and the first day of employment. So this is a JSON instead of a CSV. It's exactly the same operation, it's exactly the same input or data, it is just in a different data format. Now some real life APIs, if you want to check them out: Twitter API, Slack API, Google Maps API, or Facebook APIs. ### HTTP Methods Part three: what are HTTP methods? As I said, I'm going to kind of rush into this because I just want to get the basics done before we start explaining some more other stuff. So we had the calculator analogy and we broke down the four aspects, which is the inputs, the data type, the operation, and then we have in the response the output and the data type, right? And then for the human resources API, we did the same thing. So we had here the operation is an addition, right? And then on the HR API we have here the request, response. Inputs is this, data type is CSV, and the operation is a GET. So this is where it gets interesting. This is a GET operation. Then the output, we have the same thing, and then it's a CSV. Remember that the operation only goes in the request, not in the response, right? Because you are just telling the API what you want the API to do with your input. So there are some different HTTP methods that exist in real life for the APIs. We have these examples: GET, POST, PUT, PATCH, and DELETE. Those are the most important ones, I would say, but there are more. So if you want to take a look at the whole list, it's going to be there in the post. In summary, GET is used here to read or retrieve data from the server or API. So we used GET in the Human Resources API example because we are just telling it, "Get some information for me," or "Get some information for me." It's also good that the names are kind of straightforward, right? I'm going to skip this in a little bit because I just want to go to DELETE. So GET and DELETE are the easiest, right? Get information, delete information. Simple as that. But the fun stuff is when we have POST, PUT, and PATCH, because it really depends on how you design your API is how you are going to use these methods. I have seen APIs that simply use POST for everything. They never use PUT or PATCH. That depends on the architecture of your API. But if we follow what MuleSoft is telling us here about designing your API, is that POST should be used when you want to create new data. So for example, if you want to create a new employee in your Human Resources API example, you would use POST to create a new employee. Then PUT is mostly used to update or replace existing data, but it can also be used to create new data. And that is where the confusion starts. The difference with POST is that PUT is an idempotent method, which means that sending the same information more than once will have the same effect. So what's the difference then? If you try to create, for example, a new employee using POST and you send it once and you send it twice, like the same information, and you send it three times, you will be creating three different employees with the same information, which is a duplicate, right? But if you use PUT, again according to design principles, you will not be creating duplicate data. If you send the same information three times to the API but using PUT, you will only be creating one new employee because it's the same information and we don't want duplicates. That's the main difference. Again, you can code it however you want. I have seen APIs that only use POST. There's no right or wrong here. It's just how you prefer to do it. If you prefer to stick to MuleSoft standards, then you should follow this. If not, that's all right. Then we have PATCH, and this is used to update or modify parts of existing data. So the difference between PATCH and PUT, because PUT is also used to update, is that with PUT you would normally send like the whole information, for example the whole data of your employee. And with PATCH you would only use a small part. So if you were to change, for example, the name of your employee because you had a typo, if you use PUT you would have to send the name, the last name, the ID, the first day, whatever more information you have. But if you were to use PATCH, you could only send, for example, the ID and the first name, and it would update using the ID of your employee. It would update just the first name that you sent. That is the difference. And PATCH can also be an idempotent method depending on how your API works. ### What is a URI? Part number four: what is a URI? So here we have an example using a joke API. If you click on it, here I have a URI which is official-joke-api.appspot.com/random_joke, and I get a random joke. "When do doctors get angry? When they run out of patients." Haha, okay. So there's an example, and we are now going to take a look at what is this URI. If I go here, so the complete URI is https://official-joke-api.appspot.com/random_joke. The protocol is https, which is Hypertext Transfer Protocol Secure. It can also be HTTP. It can also be a lot more, but we are just focusing on the REST APIs which are using HTTP. The host, the host for this URI would be basically everything that is in between the double slash and the next slash. So official-joke-api.appspot.com, or also like whatever has the main domain. Then that is your host. And then we have a path, which is /random_joke, because if you change the path to something else, maybe we have something else here. Yeah, so for example, this is /random_joke and this is /jokes/random. So that's different. And then here we have, for example, /random/10. That's a different path. Or here we have /jokes/10, which is also a different path, and so on. Like, the host is always the same, official-joke-api.appspot.com, which is where your API is living. And then depending on the path that you use is what you will be accessing or what you will be getting back. ### Query Parameters Intro to Postman and query parameters. I'm not going to go through Postman right now because we don't need it, but if you go through the article that I just posted, then you can try it out to see, to follow through everything that's here and see if that works better for you to understand. So this is creating a new... this is Postman. It's basically to send requests to APIs and to be able to see the response in a better way. So for example, you saw like I opened my browser calling the random joke and I received a weird format that if you are not familiar with JSON, then you might have a little difficulty trying to understand it. If you use Postman, for example, this is how it looks like. You have the HTTP method, which is a GET. Let me zoom in a little bit. And then here you have the URL, which is official-joke-api, blah blah blah, /random_joke. And then you click on send here, and it will send this request and receive a response, which is this one. And this is a different joke. So here you can see, for example, that there's an ID, there's a type, there's a setup, and there's a punchline. That's why Postman is, well, it has a bunch more neat things to Postman, but that's mainly why you would use it if you're just getting started. So what are query parameters? For example, if you go to Google and put google.com/search and then there is this question mark q=cats, that is the query parameter. So we have the protocol https, the host is google.com, then the path is /search, and the query parameter. The name of the query parameter is q and the value is cats. So basically we are using the search path because that is where Google is using, I don't know, its logic, and then we are sending the query, which is cats. So we are looking for cats. What are the elements of a query parameter? We have the question mark, then we have the equals, and if we were to have more query parameters, we would be using this sign. I don't know what that name is. So for example, here we have whatever this path, and then we put a question mark, which means that the query parameters are going to be started here. We have query parameter number one, which is cats, and then we have the ampersand symbol, I guess. Then we have query parameter number two equals dogs, and query parameter number three equals squirrels. This is just an example. It won't work, of course. And then here's a more complex example. If you go to expedia.com and search for a hotel, after you click on search, you can take a look at how your browser looks like, and you will notice that it has query parameters. So for example, here, this is just an example. I searched for adults=2, and then d1, day one I guess, is 2020-11-23, and then day two is this, destination is Toronto, Toronto and vicinity, blah blah blah, and date is this, regionID is this, rooms is one, and so on. Like, you can see that there are more query parameters being sent. So you can also put this in Postman so you can see the different query parameters, because in Postman you can just go here to the Params tab and it will list them for you in a way better way. So you can see adult, day, date one, date two, destination and date, rooms, and so on. So this is easier to read. ### HTTP Status Codes Now what are HTTP status codes? This is the last post that I created. I think I just ran out of time when I was creating them, so I wasn't able to continue the series. But there are more things to it, like headers. Actually, just headers. Headers are basically like parameters. The difference is that headers don't go into the URL. They just go in the request, but they don't go into the URL. It's kind of hard to understand, so if you download Postman you can check it out. I think, yeah, here, for example, you have parameters, authentication, and headers. So if you go to headers, you can see that there are eight headers being sent that you are not sending. It's just Postman that's sending them for you. I don't know, stuff like what is the data type that you're sending, or where are you sending this information from, like from Postman, for example, or more information that would be needed by the API. So what are HTTP status codes? We have been focusing on the request, right? We have seen the inputs, the data types, the operation, which is the HTTP methods, and we saw the query parameters and a little bit of the headers, which all goes in the request. But then what goes in the response, apart from the data type? HTTP status codes. So let's see. There are a ton. Here is a complete list of HTTP status codes. Like, a lot. But they are separated by like the first digit. You have, if it starts with one, it's just informational. If it starts with two, is a success. Means that it's a success. There are different types of successes. If it starts with three, it's a redirection. Four, client error. And five, server error. And here you can see all of the different error codes. Again, these are standards that you could be using for your APIs, but you can honestly do whatever you want with your API. I have also seen APIs that only have 200, 400, and 500, and that's it. It really depends on what you want your API to do. And there are also some stars here, which means that these are the most popular APIs, and HTTP status codes to use in your APIs. Like, if you want to give a success back, you can send 200 OK, which is the one that I have seen the most. You can also send 201, which means created, or 204, which means no content. So like, something was done. Like, 201, if I understand, is created and returns back the information that the API created so you can see. And 204 is, it means like it was successful, it was created, but there is no content for you. So just know that it worked. For example, 400 is a bad request. It means like you, from the client side, you did something wrong with the request. So for example, when you were ordering in the restaurant, you ordered something that did not exist, so they are not able to do that. But it was your fault, not their fault, basically. 401, unauthorized, means that something in your, like for example your username and password that you use to access the API, are not correct. So it's unauthorized for you to get into the API. 403, Forbidden. I don't remember exactly what it means, but that's for some reason you weren't able to access the operation or the resource that you were trying to get, or just you couldn't do the operation. I don't know, something like that. I don't know them all like by memory, but they're there. You can just consult them. 404, not found. That is, I'm sure you have seen this when you are trying to access something that doesn't exist. So for example, let me take this URL and put something weird at the end. And once I access, it's going to tell me that it doesn't exist. So we couldn't find this page. If I were to send this from Postman, you would see that it would return a 404 error because it doesn't exist. And so on. And then the 500 errors, like 500 internal server error, means that there was something wrong from the API side, not from your side. So you did everything great. You sent the right authorization, the right request. Everything you sent was okay. There was just an internal error in the API that they couldn't solve, so they will send you a 500 to let you know that there was something wrong from their side, from the API side, and not from your side. And I put a joke here because it's nice to lighten the mood a little bit. A parent tells the teenager to clean the room. The teenager responds 202. And the parent goes away. Some hours later, the parent comes back to see that the room is still dirty and asks, "Why isn't it clean yet?" The teenager answers, "I said 202, not 200." I'm sure that's not funny, but it was funny for me. ### API-Led Connectivity Anyway, so we have this image. This is basically what we call an application network representation, right? For example, like, let's say you have Salesforce APIs that you connect to MuleSoft APIs. You have Workday, you have Twitter. You can connect to your car, connect to your smartwatch, to SAP, to Facebook. Like, you name it. There are different APIs in the world and you can integrate them or connect them using MuleSoft, right? So now what is API-led connectivity? This is basically it. It's just an approach, an architectural approach that you can take, or that we tend to do with the APIs that we design using MuleSoft. Maybe you will start taking a better look at how this all looks when we start implementing it, because they will start like talking to each other and so on. Or maybe we won't even get to use API-led connectivity in this case because it's just a simple API and we are not connecting like a bigger system. But basically there are three layers to this approach. We have Experience APIs, Process APIs, and System APIs. Experience, Process, System. And what happens is that the ones at the top, the Experience APIs, are the ones that will be connecting directly to the client. So for example, here we have an API that is for connecting to a mobile application. Or here we have a web API that is connecting to a computer. We can have another one for your smartwatch, another one for your car, and so on, right? So these APIs are the ones that are going to be created to talk to those specific devices, for example. Then in the Process API, we are going to be handling all of the orchestration. Or better, if I go to the System APIs at the bottom, these are the APIs that are connecting directly to external systems, like FedEx or a database, and I don't know, an SQL Oracle database, SAP, Salesforce, or even just like an e-commerce database or an e-commerce system. All of the APIs that will be located here are the ones talking directly to these systems. And that is all that they are doing. They are taking the information from the Process API, they are translating it to match the other APIs, the external APIs. And then when the external API comes back with their answer, the System API is responsible for transforming, or it also depends on your architecture, but that's all these System APIs do. They just connect to the external systems. So in theory, you should have one System API per external system. And the Process APIs that are in the middle of the two are basically the ones that create all of the orchestration, translation, whatever needs to be done. For example, from the mobile API, you talk to the shipment status, you talk to the order status, and you talk to the order history API. And then from the shipment status API, you talk to FedEx and you talk to the call center, it says right here. Or the order history talks to customers and talks to orders, and so on, right? So this is how the API-led connectivity approach works. It's basically just, again, a guide for you to design better systems. ### Wrap-up That's all from me. I hope you like it, and I will see you then on the next session on June 14 at the same time to start designing our API then. All righty, see you then. Later, bye! --- ## Session 3: Design an API Specification in Design Center | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=XIrCqwmTPQs | Page: https://prostdev.com/video/mulesoft-from-start-design-api-specification | Series: MuleSoft from Start: A Beginner's Guide ### Introduction Welcome back! My name is Alex, and in this video we will learn how to design an API from scratch — how to write down the requirements in a note and then bring those requirements to life in our API specification using Anypoint Platform's Design Center. ### Writing Down the Requirements Here is where we're going to be doing the requirements for our blog API. Since we're just getting started, we need to design it first. This is my process, but you can use your own process, whatever you prefer. First, we need to identify our resources. Resources are basically the nouns or the things that we can do a CRUD operation on: Create, Read, Update, Delete. We want to do all of those things with these resources. Our list of resources includes: - **blog** (we'll have one blog, so I'm not sure if this is a resource or not yet) - **articles** (plural — we'll have several articles where we can create an article, read an article or several articles, update an article, or delete an article) - **writers** (same operations) - **series** - **categories** - **comments** Notice how all of these are plural because we're going to have several of each. Maybe blog will just be the largest object that will contain all of these inside. Let's leave it like that for now. ### Creating the API Specification I'm going to create a new account so you all can do this with me if you want to. Once you sign in, you might see this as your first screen — just go right over here and select Design Center. Remember in session one we saw an overview of all of these products, and now we are going to just focus on Design Center for this session. Let's click on "Create new API specification." This is the beautiful thing about this UI — you don't have to know RAML or OpenAPI Specification language to design your API. For the project name, let's use "blog API." Now, how do you want to draft the API spec? Here you can select "I'm comfortable designing it on my own" and choose whatever language you want, or you can select "Guide me through it — use a visual interface." The scaffolding can generate both RAML and OAS (OpenAPI Spec), and that is what we want. Let's select "Guide me through it" and create the API. Here on the left we can see the different things that we can do — we can import stuff from Exchange, and we can create security schemes, resources, data types, and so on. Here on the right you have the code that is going to be generated once we start creating stuff. Here in the middle is where we'll work. At the bottom you can see that you can select RAML or OpenAPI Spec — you can see whatever language you want and even copy and paste this code if you need it. This is a free tool, and you can create as many free trial accounts as you want. Let's set up the basic information. Our title is "blog API," version 1.0.0, protocols let's select both, media type is going to be application/json. For base URI, we won't have any parameters. Description we can leave blank. Secured by — nothing. This blog will not have any security because we trust people (said no one ever). ### Defining Data Types Before we create the resources, we should probably start creating data types. Let's think about what each one will have: **Article:** - ID - title - content - writer ID (this will link to the writers list) - category ID - comments **Writer:** - ID - name - bio - articles **Series:** Actually, wait — what's the difference between series and categories? I think I don't need series actually, so let's remove it. I think we only need categories. **Category:** - ID - name - list of articles **Comment:** - ID - content - author - article ID (because an article can have many comments, but a comment can only be in one article) Now let's create these data types in Design Center. Starting with **article**: - ID: required, type number - title: required, string - content: required, string - writer ID: required, string — actually, let's make all IDs numbers - category ID: number - comments: not required, this is going to be an array of comment (we'll come back to this) Let's do **comment** next so we can link it properly: - ID: number - content: string - author: string - article ID: number For the example: ID 1, content "This was a nice post," author "Alex Martinez," article ID 1. Now back to article, the comments property is already showing the inherited example. Perfect. Let's add an example for article: - ID: 1 - title: "The Importance of Regular Exercise" - content: (example content) - writer ID: 1 - category ID: 1 - comments: (inherited from comment) Now **writer**: - ID: required, number - name: required, string - bio: required, string - articles: not required, array of article Example: ID 1, name "Esmeralda," bio "20 plus years of experience in IT and very smart," articles with the example article including the comment from Alex. Finally, **category**: - ID: required, number - name: required, string - articles: not required, array of article Example: ID 1, name "Mental Health," with the articles array. We also need an **error** data type for error responses: - code: number (example: 404) - description: string (example: "Not found") ### Building the Articles Resource Now let's create the resources. We already have our requirements, and I don't think we're going to do blog because it seems like all the information we need is already covered. Let's start with articles. Create a resource at `/articles`. Here we have the HTTP methods we can select. Remember from our previous session we talked about HTTP methods — the main ones were GET, POST, PUT, PATCH, and DELETE. **GET /articles** (get a list of all articles): - Response 200 OK: body is an array of article - Response 404 Not Found: body is error ("There are no articles to show") - Response 500 Internal Server Error: body is error ("There was an internal error") Now we want to retrieve just one article. We'll use a slug pattern (like prostdev.com/post/programming-challenge-seven). Let's add a slug property to the article data type: required, string. Create a nested resource `/{slug}` with a URI parameter: - slug: string, example "importance-of-regular-exercise" **GET /articles/{slug}** (get one article): - Response 200 OK: body is article - Response 404 Not Found: body is error ("There are no articles with this slug") - Response 500 Internal Server Error: body is error URI parameters are like variables in the path (https://host.com/posts/1, where 1 changes). Query parameters come after the question mark (like ?q=cat). Headers are not in the URI at all. ### Adding POST, PUT, and DELETE Let's check our CRUD operations: - Can we create an article? Not yet. - Can we read one article or the list of articles? Yes. - Can we update? Not yet. - Can we delete? Not yet. **DELETE /articles/{slug}:** - Response 204 No Content: nothing returned - Response 404 Not Found: body is error ("There are no articles with this slug") - Response 500 Internal Server Error: body is error I don't want to allow deleting all articles at once — that would be awful. Just one at a time. For creating and updating, I'm going to use POST to create and PUT to update. This is my personal preference, but you can do whatever feels better for your API. **POST /articles** (create a new article): Body to send (the API will generate the ID): - title: required, string - content: required, string - writer ID: required, number (you have to create the writer first) - category ID: number (not required) - slug: required, string Responses: - 201 Created: returns the created article - 400 Bad Request: body is error (if something's wrong with the body, like missing title) - 409 Conflict: body is error (if the slug already exists) - 500 Internal Server Error: body is error **PUT /articles/{slug}** (update a specific article): Body to send: the full article object with all properties (since it's already created) Responses: - 200 OK: returns the updated article - 400 Bad Request: body is error - 500 Internal Server Error: body is error So now we have completed all CRUD operations for articles: - Create: POST /articles - Read: GET /articles (all) and GET /articles/{slug} (one) - Update: PUT /articles/{slug} - Delete: DELETE /articles/{slug} You can design this however you want. For example, you could use POST to update a specific article and create a new one if it doesn't exist. In my case, I don't want to create new articles if I'm trying to update — I just want a response saying "you're trying to update something that doesn't exist." ### Homework For your homework, you will need to continue what we started. We only did articles today, but you saw what I was doing. You will have to do writers, categories, and comments. At least we finished the data types, so you only need to do the resources. I will also try to do that before the next session so we can all start on the same page, and I will send this to the repo so you can compare your response to mine. You are free to make any changes you think are necessary for your API because this is your design. You are designing it however you want. If you want to use POST, PUT, or PATCH, you are free to do so. If you want different status codes — 400 Bad Request, 500 Internal Server Error, or whatever else — you are free to do them. This is your stuff. I will see you in two weeks on June 28th, so we have enough time to do our homework and finish this API specification. Bye bye! --- ## Session 4: Test & Publish the API Spec to Exchange | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=ho5GQJD8Hxo | Page: https://prostdev.com/video/mulesoft-from-start-test-publish-api-spec | Series: MuleSoft from Start: A Beginner's Guide ### Introduction Hi there, Alex here. Just a quick summary of what you're going to see in this video. This is a pre-recorded live stream and I edited a few things so you don't have to watch the whole thing. In this video we did the API design or API specification for our API. We finished that, we tested it, we published to Exchange, and we started creating the implementation using APIkit. That is a MuleSoft product that helps us to get all of the API specification and start creating code instead of having to do everything from scratch. MuleSoft does this for you, including error handling and some logging. I hope you enjoy it. ### Reviewing the Homework Solution The homework was to create the resources for writers, categories, and comments. I added the solution to the repo before this session and here is my whole solution if you want to read it and go through it. I'm going to do a quick summary. Here are the requirements if you want to see them. First of all, I didn't continue using the visual designer when I started checking all the resources and everything because you kind of have to go—for responses, click on Add, and so on. I kind of just wanted to copy and paste stuff, so I ended up just going to Design Center and creating a new one. I created the Maxine's Blog API. I didn't want to remove the other one just in case we needed to check something from there, but I ended up creating a new one. You can continue using the UI if you feel better with that. This is just my personal preference. I ended up with this RAML which I think looks easier. Then you can just copy and paste and do different stuff. You can also use traits and other kinds of things. These are the endpoints I ended up creating: `/articles`, `/articles/{articleId}`, then `/articles/{articleId}/comments` and `/articles/{articleId}/comments/{commentId}`, and finally I created `/writers`, `/writers/{writerId}`, and `/categories`. I didn't create `/categories/{categoryId}` and I'm going to explain why. But this is basically the API specification that I ended up with. You can do whatever you want—there are different ways to design your APIs. It doesn't have to be my way. ### Data Type Changes Here are the changes I ended up doing. First of all, with the article data type, I changed it—we previously had `writerId` as a number, I changed it to be just `writer`. So here, instead of having `writerId` I actually have the whole writer, so `writerId`, `name`, and `bio`, so everything is going to be there. Same thing with `categoryId`—I actually changed the category to be just a string instead of it being its own data type. So that is why we don't have the category data type here. I ended up just making it a string. So this is kind of going to be like if everything is going to be saved in an array of strings and then each string has to be unique. If there's something different then it's going to become another category, kind of a deal. So it's going to matter the spelling, the lowercase, the uppercase, and so on. Then the writer data type—I removed `articles`. So here in the writer data type we used to have the article property and I removed it because then I would end up having, since I moved the whole writer thing into the article data type, then I would have like article, writer, article, writer, article, and so on. So I just removed the article from writer. And then if you want to find out what articles a writer has written, then you will have to create a query parameter or something. In my case I'm just not doing that. I think I have the article and in the article we have each writer, not the other way around. I removed the category data type because I just created a string with it. The comment data type—I removed `articleId`. So if I go to comment I just have `id`, `content`, and `author`. I don't have the article listed to the comment anymore, but if I go to the article data type then it's going to have different comments. So for example here I have writer, array of comments is an array, and then this is an array of different comments. So all of the comments will be under an article instead of being on its own. ### Extracting Data Types into Files I created the new API specification. I created a `types/` folder with the data types just because I didn't want to have all of the data types listed in the main RAML. I did this: `types:` and then inside here you can just write the name of the type and link it to the file that has it. In this case everything is under `types/` and then `error.raml`, `comment.raml`, and so on. To create it you basically just go here, create new folder, create the `types/` folder, and inside the types folder here you can just click on it and create a new file. But when you create a new file, make sure here that you select a data type for this specific case because we will be creating a data type. We are not creating a specification, we are not creating a resource, we are not creating a library—we are creating just one data type. So do that, click on Create, you can change the file name, and you end up with this. For example here, `comment.raml`—as you can see here, this says data type, it doesn't say specification or resource, it says data type. And now we can just list the properties that we had before. The `id` and example is `1`, the type is going to be `number` and the format is going to be `int`. Then we have the `content`—example "Hey, this was a nice post", type `string`. And then `author`, example "Alex Martinez", type `string`. This is just going to be the name and that's it. So you do that with all of them. We have `error`, we have `writer`, and we have `article` which has more stuff. And as you can see here, for example in article where we have writer, then in the type we are linking it to the `writer.raml`. Same thing for the comments—we have the comments and then items and we are including the `comment.raml`. This is an array. That is how I decided to create it. You can do whatever you want with that. ### Traits and Nested Resources Then I created some traits because I noticed that all of the responses will have 400 and 500. So I created a trait called `commonErrorResponses` and these are the two—just 400, 500, and the body is going to be error. That's it. And basically in all of the resources you can just go and say `is: [commonErrorResponses]` and then everything that is going to be here is going to have the same common responses. I could have also clicked on new file and trait and put it there—that also works. But I had a little bit of trouble doing that for some reason, so I decided to just put it here directly. But you can also just put it in another file if you want to make this even better. You can also create, as you saw, a new file—you can create examples, you can create user documentation. You can create a lot of different things to make this RAML easier to follow. I thought this was pretty good. We have the `/articles`, we have get, post, and then `/articles/{articleId}`, we have the URI parameters, get, put, delete, and so on. I removed all of the descriptions as well because I just wanted this to be easier to read, but you can also leave all of the descriptions. If you don't know RAML, I would advise you to learn it first if you want to do something like this, because there are so many different ways to make your RAML easier. So I decided to leave it like this, but it's up to you if you want to just keep extracting pieces of the specification into different files. Because at one point, if you have a huge project, this RAML—if you put everything in this RAML, then this RAML would be like, we have right now 150, 160 lines. So imagine you can end up with 5,000 lines or something like that if this is too complex. So it's better to put everything here in files and folders to make it easier to read the main RAML. But in this case I thought it was not that bad. I can clearly see where the traits are. I decided to extract the types, but other than that I think that looks pretty good for me. Again, you can do whatever you want. Then the other change that I decided to do on the design was, as I told you, we have `/articles` and inside `/articles/{articleId}` we have `/comments`, because I didn't want to have just `/comments` like a resource for comments. Because all of my—at the end of the day all of my comments are inside the article data type. So what's going to happen here is that I'm going to have to do `/articles` and then `1` for example and then `/comments`, and this is going to return all of my comments because it's first going to extract the article ID and then it's going to send me the comments. Same thing here—if I want to see just one single comment I decided to do `/articles/{articleId}/comments` and then the comment ID. But you can decide to manage this differently if you want. I think in categories—yeah, in categories I created a query parameter in the get. So if I go to `/categories` I will be able to see all of the categories that have been created, but if I send the query parameter called `categoryName`, type of string, then I will get just the categories that match this category name. So this is a change that I decided to do instead of doing `/categories/{categoryName}`. I'm just doing `/categories` and I can just send a query parameter. So you can do the same with the comments if you wanted to do that—to just do `/articles/{articleId}` and then send the query parameter that is like comments or comment ID or something like that and it will return that. You can also filter it. Normally the query parameters are used to filter things, but you can use it however you want. You can also send headers if you want. Remember, the only difference between using query parameters and headers is that the query parameter is on the URI—you can visually see the query parameters in the actual URI that you send. And the headers are not visible from the URI. You have to kind of go inside the request or the response to see them. The only thing that I would advise you is that all of the responses should have at least a 200 response—it can be 201, 202, or whatever, but that starts with 2. That it has a 400 response, whichever you want, just that it's a 400-and-something because it means that there was an error from the client side. In the 500, which is an error from the server side. Those are the only three responses that I would advise you to always have. Outside of that, all of the things that you want to do—you'll notice that I just have post and put but I don't have patch. I just decided to do it that way. I decided not to use patch, but you can also use patch if you want to. And that's it. All of them have get. The delete I decided to use it only on the singular resources. So I can only delete one article at a time, one comment at a time, one writer at a time, but I cannot delete all of the writers, all of the comments, and so on. So far this is my design. You can decide to take that or do your own, whatever you prefer. If you want to take mine you will find it in Sessions 3 and then `homework-spec.raml`. The `in-session-spec.raml` is the one that we did during the last live stream. The `homework-spec.raml` is what I just did. ### Testing with the Mocking Service Go here and click on "Try it". You will be able to send the query parameters if you have any, any headers or anything you want to send. This is just a get—I'm not sending anything, so I can just click on Send and it will return me an example like "200 OK, everything went right" and this is how your response is going to look like. If I wanted to try for example the categories, let's see categories because we had a query parameter there. If I click on "Try it" I can just not send any query parameters. Oh, I made it required. Well, that's wrong. I should—that shouldn't be required. Hold on. If I check this again, try it, yeah, okay. Now this is not required so I'm not going to send it. Send. Okay, I didn't get anything back for some reason. I should have gotten—well, I got a 200 OK. String. Oh, maybe I don't have any examples. Oh yeah, I don't have any examples. Okay, so I can add an example. We have example here that you can send, but I'm not going to send it. If I click on Send I receive a 200 OK and I receive an array of strings. That's what I wanted, with the different categories that we have. Yeah, I like the design of my API. I can also try how it's going to look like if I go to comments. Get. Oh, I think I have to have different IDs. Well, it's nice that we're trying it out. So `/articles/{articleId}` is going to be that, and then `/comments`. And let's change this to `commentId` URI parameters, `commentId`. So now, yeah, `/articles/{articleId}/comments/{commentId}`. I'm going to change everything so it matches: `/articles/{articleId}` here and here, and then `/writers/{writerId}` here and here. Okay, so now `/articles/{articleId}/comments/{commentId}`. If we go to the get, click on "Try it", now we have the different URI parameters. So article ID 1, comment ID 1. Comments. Ah, here is why. Comment, comment, comment, copy pasting, and then comment ID, comment, comment, comment. And that is why it's good to check everything. So comments, get, try it, article ID 1, send. Now we receive the list of comments. And if I go to comment ID, get, try it, article ID, comment ID, send, and now we receive just one comment. Okay. All right, I'm glad that I checked this before continuing. That would have been awful. All right, so we have this. It's all good, we tested it, it works. ### Publishing to Exchange Now we can publish it. So click on Publish. Preparing to publish as a version 1.0.0. API version 1 and let's do stable because we want to be able to use it. If we do development I think we won't be able to use it, so let's do that. Let's leave it in stable. Publish. All right, so this is done. Close. And then we have it in Exchange. So we can go to Exchange now and see that it's actually there. Yes, Maxine's Blog API. And then if we go to Summary here we will be able to see everything that I just showed you, all of the specification. And then on the Endpoints we can do the same thing that we just did. But this is—I think this is better, like visually speaking. I feel like this is better to see than the one we have in Design Center. So yeah, you can just check this, everything. You can also use this part here to test it out just as we did before. You can add query parameters if you want. You can click on Send and you will receive the 200 OK and how this response would look like. So you can also try it out there if you want. And that's it. We have it on Exchange, which means that we can start designing it in Studio. ### Scaffolding the Project in Anypoint Studio (APIkit) If you're new to the whole thing: mulesoft.com, Studio. You can also do that and you will get to this part and you can just download it—Downloads, Anypoint Studio. So here's the overview, here's the release notes, download and install. Here are also for Windows, Linux, and Mac. I'm going to do Create a New Mule Project. I think it was Project name. What was the name? Maxine's Blog API. So Maxine and Blog. Maxine's Blog. Maxine's Blog. Okay, I just put it as Maxine's Blog. And then here we have the—let me zoom in—API implementation. So here you can see that it says "Add an API implementation to your project to automatically set up an APIkit Router and create placeholder flows for each resource method." So that's what we want to do. This basically means based on the API specification that you have, I'm just going to create the basic structure for your project so you don't have to, which is awesome. So "Scaffold flows from this API specification"—yes, that's what we want. Import a published API, import RAML from local files. So even if you don't put it in Exchange you can also just put it here, or download the RAML from Design Center. So go to Design Center, you download it, and you put it here. No, it puts it there for you, I think. Anyway, we're just going to do Import a Published API because we want to be able to use the one that we have in Exchange because we are assuming that it's done, right? So we're going to select that: Import a Published API. And then Plus. From Exchange or from Maven. Let's select from Exchange and now we can search for it here. Maxine's Blog. Maxine's Blog API. This is the one that I created. Publisher is Twitch because that's the organization that I created. Click on Add. Now you have it here in Selected Modules and you can now click on Finish. ### Running It Locally So we have this and now this is the beautiful, beautiful thing. Maxine's Blog API. We have the APIkit Router. This is the beautiful, beautiful thing about MuleSoft: APIkit Router basically gets the request here and then based on the URI or the path that it receives, it automatically sends to the corresponding flow so you don't have to do that. And it already creates—oh, I think I can zoom in here. Yay. It already creates the flows, the error handling flows for you. So for example, if there's a bad request it's going to automatically send this. You don't even have to create any implementation at all to send the bad request. If there's a bad request it's going to be here. If there's a not found it's going to be here. If there's a method not allowed, not acceptable, unsupported media type, not implemented—they already have the error messages for all of that. And then here you have the full flows for everything. Here you have the put. Here at the beginning you can see put and then we have `/articles/{articleId}` and this is what you're going to—you can just change this to go to the desired flows. If we go down we have put `/articles/{articleId}/comments/{commentId}` and so on. Right-click on the canvas here and click on Run Project. This will start running it locally and then you can run it in the terminal or in Postman or whatever you prefer. My case, I'm going to do terminal. So here's my terminal. If I do `curl localhost:8081` will be able to run this. Okay, this is running now. Let's run `localhost:8081`. Oh, sorry, let's do `/articles`. No, `/v1`. It probably added there. Okay, see here Path—it says `/api/*`. So this is the path that we're going to have to use before sending the requests. So `localhost:8081/api/articles`. It didn't return anything. I guess it only has a logger or something. Yeah, it just has a logger. It doesn't have anything here, so it did log something. If I click on word wrap right here it will show me everything in different lines. So it did run something here. So if I run it again you will see that a new line is going to be added there. There you go. And if I keep running it, new lines are going to be added because it's logging something. ### Wrap-Up All right, I hope you like that and I will see you on the next video. Remember to follow me on Twitch or subscribe on YouTube if you're watching the recorded version from YouTube. All right, I'll see you on the next session then. Bye. --- ## Session 5: Develop the API in Anypoint Studio | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=K9ntwKz9vds | Page: https://prostdev.com/video/mulesoft-from-start-develop-api-anypoint-studio | Series: MuleSoft from Start: A Beginner's Guide ### Introduction Hi there, Alex here. If you haven't been following all of the previous videos in this series, this is the recorded version of the live stream I did. I just cut down certain scenes so you don't have to watch the whole hour and you can just watch a few minutes to see what we did in the session. Let's watch it. ### Anypoint Studio UI Tour Everything you have here in the middle is your canvas—you will work on this canvas in the middle. On the right we have the Mule palette. On the left we have the project structure, so all of the folders and files you'll be able to see from here. I mostly use this panel for MUnit, but I'm not using MUnit right now so that's fine. Here on the bottom, every time you click on a component or connector you'll be able to see the Mule properties. If you click here on Console you'll be able to see all of the output console of whatever you're doing in the application. If you start a new application you'll see all of the output from the console here. Then we have Problems. Here I already have three warnings. If you have issues, warnings are normally in this warning sign icon. If you have any problems they'll be in red—those you do want to fix. Inside `src/main/mule` is where we have our flows. You can see here in parentheses that it says "close"—all of this is called a configuration file. It's an XML file. If you open it you'll be able to see all of the different flows. In this example it's not empty, but if I click on New and then select Mule configuration file, I can create a new one. I'm going to name this `test` and click Finish. Here there's an empty canvas. If I wanted to add a flow, I would be able to go here to the Mule palette and select, let's say, Core. Here you can see all of the core components are already installed. In the Scopes we have, for example, Flow. I'm not going to go through all of these—I'm sure you can figure out some of them. I prefer to teach with examples and not just theoretically. If I go to HTTP, all of these are modules. If I go to the HTTP module and I take a Listener from here and drag and drop it to Source, then I'll be able to have an HTTP Listener that is listening to a URL. Every time you call that URL it's going to come here, and then here is the process of what you want to do once it gets inside this flow. For example, I can add a Logger. I can use the search bar here and type "logger." I already have it in favorites, so I can drag and drop it into the process. ### Global Elements and global.xml Now let's change our Mule project to implement some best practices like global elements and properties. Here we have the Message Flow view, here we have the Global Elements view, and here we have the Configuration XML view. If you click on Message Flow you'll be able to see the graphic representation of the XML code. If you click on Global Elements you'll be able to see these global configurations. For example, I have the configuration of this HTTP Listener that is listening on localhost or 0.0.0.0 IP and port 8081, and it has a read timeout of 30,000 milliseconds. This is a global property that can be reused in different flows or in different configuration files. It doesn't have to be one per file—it can be reused over the whole project. Configuration XML is where you keep all of the XML of this file. If you're more comfortable seeing code, you can just go to the XML view and work on the code, or if you prefer the graphical view you can go to the Message Flow tab. The first thing we're going to do, as a best practice I've seen in other projects, is go here to `src/main/mule`, right-click, select New, and then Mule configuration file. We're going to create a new file called `global.xml`. Once you do that, click Finish. In this file we won't have any flows—we will only have global elements. You saw the global elements we have in the other file: we have an HTTP Listener configuration and an APIkit Router configuration which is used for APIkit. If you right-click on it you can click "Go to XML" and it will show you where the HTTP Listener code is located. We're going to copy both of those—or cut them—and then go to the `global.xml` file. If we go to Configuration XML, now we're going to paste those two inside the `` brackets here. Now it's in red because I haven't saved the other file. Right now it thinks this configuration exists in both files. If I go to the previous file and save it, now it will give me errors because there's no configuration file there. But if I come back here to the global elements, now we have them here. If I save this, now this problem goes away because now we have this file here, and now this problem goes away too because now it can reference the other file. This way you can keep all of the global elements inside this global file and you don't have to keep searching for them in every single file. If you end up with, let's say, 30 configuration files here with different flows, the last thing you want is to go into each specific file to look for the global elements. If you have just a `global.xml` file, you'll always know that the global elements are located here. ### Property Files per Environment Now for the properties. You saw that I had the HTTP Listener configuration, and it says this is 0.0.0.0 and it's using port 8081. But what if, for example, this is only in my local environment running on 8081, but in my dev environment it's running on 8082, or in my prod environment it's running on 80? There are different ports you can do per environment. Another best practice is to create properties for all of these settings and to create a different property file per environment. In this particular case I will not have a prod environment. I will just have a local environment and a dev environment—local is what I'm running here in my computer, and the dev environment is when I put it in CloudHub. If you go to `src/main/resources`, right-click on it, select New, and then just new file. I am going to name this `local.properties`. You can also use `local.yaml` and keep all of your properties in a YAML file instead of a properties file, but I'm just used to the `.properties` extension so I prefer that. Click on Finish. Then let's create another one that is going to be `dev.properties`. Now we have `dev.properties` and `local.properties`. Now, for example, I want to create: ``` http.host=0.0.0.0 http.port=8082 ``` I'm going to do 8082 just to change it a little bit. I can save this, and then go to `dev.properties`. In `dev.properties` I do want to do 8081, so I'm going to save it like this. Now let's take the HTTP host property and go to `global.xml`. If I double-click on the HTTP Listener config, now I can change this to be dynamic. To make it dynamic and use the properties, I'll do dollar sign, curly brackets, and then the property `http.host`, and then close the curly brackets. We can also have the port be dynamic: `${http.port}`. All right, we did that, but that's not all. How will this program or app know which configuration file to use? We need to create another variable. If I click here on Create in the `global.xml` file, in the global elements—because I'm going to create a global element—I click here on Create. Let's search for "property." You'll see the global property. You can click on that. Here in Name let's do `env` and value is going to be `local`, because every time you run this anywhere that is not CloudHub, the default is going to be local. So name `env`, value `local`. If I run this in my local it's going to have the value `local`, and if I run this in CloudHub I'm going to put a property there that says `dev`, for example. Click OK. Now, how will it know exactly which file? Let's create another global property. Search for "configuration properties." Double-click on it and it will ask you for the file. Here is where you will connect the file that we have here—either `dev.properties` or `local.properties`. Since we already said the environment, the `env` property is going to be `local` for this case, then it's going to take the `local.properties`. For that we're going to do: dollar sign, curly brackets, and inside we put `${env}.properties`. If we run this in local, the `env` is going to be `local`, so this file is going to be `local.properties`. If we run this in CloudHub, where I'm going to put a `dev` property, then this is going to be `dev.properties`. That is how it will know which file specifically it will take. ### Implementing GET Articles with Object Store Now let's start implementing stuff. I'm thinking we can use the Object Store that MuleSoft provides because I just like it. We can also try to implement our own database, but I feel like that's going to be more issues because in local maybe you just have your local instance of a database like SQL, but then when you send it to CloudHub it's going to have to connect to a service where you have your database in the cloud. I don't have the infrastructure for that right now, so I feel like the easiest way to do it with just the few resources we have is to use Object Store. Let's do `/api/articles` GET. Let's do the GET. We have GET articles—that is the first flow we're going to try to connect. We're going to leave the Logger, but I'm going to change the logger to show me the start, for me to know that it got into the flow. I'm going to change the message here. I'm going to put "Starting GET articles," for example, because that makes sense for me. Here on the display name I'm going to put "Start." I'll probably do another logger here for the end—the same thing that I put in the start, I'm just going to put "Ending" just so I know where it ended and where it began. Now we're going to add the Object Store. Right now we don't have any module for Object Store. We just have Core, APIkit, HTTP, and Sockets. We're going to click here on Add Module and let's see if Object Store is here. It is. You literally just have to take it, drag and drop it to the left side, let it go, and now you have Object Store. If there is a module that is not here but it's in Exchange, you can also click here on "Search in Exchange" and it will get you to this other screen that we've seen before. You just have to sign in with your username for Anypoint Platform and add the module from there. I want Retrieve. Now we see here that we have an X saying that the key is required. You can also see in Problems, "Required attribute key is not defined in Object Store." Now we're here on Retrieve. The key I'm going to name `articles`. Then Object Store—here this is a dropdown for the configuration of Object Store. I have not done that. Instead of creating the global element from here, I will instead go directly to the `global.xml`, go to Global Elements, and create the Object Store global element. If I look for Object Store, we have this. Double-click on it. It just asks for a name. If it's persistent, max entries—I don't think I need any of that. I think that's good enough. That's all. I didn't really add anything. I just created the configuration. Save it, and then I go back to where I was, and now I can select it from here. ### Implementing POST Articles (Retrieve, Append, Store) Now let's do the same thing that I did with the loggers before. Now we are going to add the Object Store and we're going to select Store. It says "Stores the given value using the given key," which is exactly what we want. Let's put the store there. The key is going to be `articles`, like the Retrieve. Here you start to wonder—maybe you should have a property to retrieve these instead of having to manually write it, in case there is a typo or something. I think we can do that. The value is going to be the payload, so whatever comes in the request is going to be added to the articles. We're not going to do any validation whatsoever. We just want to do a POC really quick. We select here our Object Store, and that's it. Now this way we are going to be overwriting the same thing over and over again. Let's create the property. Because this property is always going to be the same regardless of the environment, I am going to create in `src/main/resources` a new file and I'm going to put this as `default.properties`, because all of these properties I want to be available in all of the environments—they're not going to change. Here, for example, is going to be: ``` os.articles=articles ``` I just want to avoid the typo, that's all. Then I'm going to reference this property in the Object Store stuff. Save this, go back. Now here in the Store, I was doing key `articles`. Here's another way to call your properties: instead of doing the syntax as we were doing before with the dollar sign and curly brackets, when you have this FX button you can click on that and this will become DataWeave code basically. Here I'm going to do `mule::p()` parentheses, and inside in quotes I'm going to put my `os.articles`. You can also do this without the `p`, like `p('os.articles')`, but I have found that that syntax sometimes can give you trouble, so I'm going to leave it as `mule::p()`. I'm just going to copy this and put it in the other place as well. So we can reference that instead of having to manually write it. Now I click on the FX. This is going to be now `mule::p()`. Also, if you're not sure if you had clicked on the FX or not, you can see here there is a hashtag and then square bracket—that is telling you that this is a formula instead of being a literal value. The other thing we have to do is go to Global. The same thing we did here to put `${env}.properties`, but now we have to add our other file. First, we need to retrieve all of the articles so we can add the new article to what already was there, instead of removing what's already stored in the articles key and just putting the new one. We don't want that—we want to keep adding into the new stuff. What we are going to do is we are going to Retrieve. I'm going to use the same `os.articles` property or this key, and the Object Store is going to be Object Store. The issue here is that when we arrive here, what we have in the content of the payload or whatever we have from the request is going to be the request that we send. But then when we get here and we retrieve the articles, that is going to be removed. What we're going to do is, if we click here on Advanced, Target Variable, we're going to create a new variable called `articles` and the target value is going to be the payload. This way we will have the variable `articles` and the payload, and then we're going to put them together. ### Fixing the Default Value and JSON Output Now we can add the Transform Message here before storing but after retrieving. This is what we are going to do with DataWeave: ``` vars.articles ++ [payload] ``` My array plus payload, which is going to be the object that I receive from the request. That's all. I am just putting my new article inside the array of articles that I already had. Let's rename this here. Rename as "Append new article." So we retrieve the articles, we append the new article, and then we store that—which is the array with the new article—we store that, which is now in the payload, into `os.articles`. If you want to kind of keep things clean, you can also remove the variable that we used before. After this, if you want to help a little bit the garbage collector, you can do that. If you don't do this it's fine. It's not going to break. The variable is not going to stay there forever. As soon as this flow is done, that variable we created is going to be removed—which was `articles`. But I like to keep things tidy, so Remove Variable `articles`. You don't have to do this again. This is a very small program. It's not going to break just because you don't have this, but I like to see it. All right, and that's all. That should work. It was deployed. Let's call `/api/articles`. Oh, I tried 8081—it was 8082. Oh, I received an error because it doesn't exist. It doesn't contain any value for key `articles` and default value was not provided or resolved to a null value. Let's do that really quick. In the Retrieve—and now notice this, because we have here a Retrieve in GET articles, we have a Retrieve which is doing basically just retrieving `os.articles`, and we have another one here which is doing the same thing. The only difference is that here we're putting it in a variable. Maybe we could create a subflow to do these two things, but for now let's just leave it simple. Now I'm going to put the default value here. I'm going to do DataWeave and I'm just going to put an empty array: `[]`. I'm going to put it in the other one as well. You can maybe try to figure out how to create subflows so you can put all of these things in if you want. Saved. Run it again. Another thing we can also do is we can create different configuration files for each resource. We can have one with articles, one with writers, one with categories, and this way we can have all of the logic of the flows in those specific files. Here I can just do a flow reference from this part to that other part, so I don't have to keep working here on this file because it's so long. I can just keep working on smaller files. I think you can figure that out. You can figure out how to do that on your own. Maybe for homework you have to create different files here. I'm going to put all of the instructions in the GitHub repo as well so you can follow up. Here in `src/main/mule` you will have to create configuration files, and you will have to have there the flows that we just created. These three, for example, are going to have to be in a subflow in that other part, and here you will only have a flow reference. Now let's try this again. I think I know what that is. They want me to do `output application/json`, so this way this empty array is going to be application/json. Let's save that and change that in the other one as well. That is why we want subflows—instead of having to do this every single time in both of them, we could just be able to do it in one place and it would be reflected automatically in both of them. ### Testing It Are we done? Finally! Now we retrieve the articles and we are retrieving an empty array. I am in `localhost:8082/api/articles`. For the body I'm going to send Raw, and this is a JSON. Let's copy this and paste it. Send, and we received the 201 Created. We received this thing. Now let me try this again. It was created! Now let me do `id: 2` and I'm going to remove this whole thing so it doesn't look that big. "The Importance of DataWeave," and this was written by Esmeralda, category, and there are no comments in this case. Now let's send this. It was posted according to this. Now I can also do the GET. We have the two of them—we have the first one and the second one. Now we have one article, two articles! It's working. I'm glad I tested it. This is working. You have the first version. ### Homework Now you figure out how to do the rest of the things. Remember, create a new configuration file here for every resource so you don't have to keep navigating in this huge file, and you just have a flow reference. I think you can even select this whole thing and then right-click on it and Extract to Flow or Subflow. There are ways to do it—just try it out, try to see what works, what doesn't work. This will give you a lot of experience to figure out what's happening. ### Wrap-Up Thank you so much, everyone, for watching. I really appreciate all of you for joining me and for watching my videos. Please let me know if you have any suggestions or any feedback for me of what else you want to see, if this is working for you, or if you have any questions. Also, if you don't finish the homework by the 19th, I will be happy to help you. You can contact me on LinkedIn or on Slack or on Discord, and I can just help you with that to see what we can do, or just wait for the session to see how to do it. All right, that's all for this live stream. I will see you in two weeks, and I will upload the homework solution—my solution—on the 12th. All right, see you all. Thank you, bye! --- ## Session 6: Debug the Mule Application in Anypoint Studio | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=75IJ1WFa9iA | Page: https://prostdev.com/video/mulesoft-from-start-debug-mule-application | Series: MuleSoft from Start: A Beginner's Guide ### Introduction In the last session, session five, we started developing our API in Anypoint Platform. The homework was to create new Mule config files and add flow references to finish the logic for the Articles resource — just the happy path. I modified the homework last week because I realized I was asking too much. Originally, I asked you to finish the whole implementation, but when I was doing it myself, I noticed that was way too much if you're just getting started. So I changed it to just finish the logic for articles. Even that would be difficult if you don't know DataWeave, because you do need to transform some data. So don't worry if you didn't finish — I will walk through it. You can see my full solution in this pull request. I'm going to start doing pull requests so you can see all the things I changed. I added images, I changed the RAML specification to include POST and DELETE for categories. After you do that, you can simply rescaffold. All of the instructions are in the README, so you can check out my step-by-step and see exactly everything I did. You can also see all the code changes. ### Per-resource config files and flow references I modified the main file, which is Maxine's Blog API, and I created four new config files — one for each resource. This way, things are better organized. You can separate this however you want, whatever makes it easier for you to find the flows. For me, this works because if I need to modify something in the logic of articles, I can just go to that file. If it's something in the logic of writers, I go to that file. In the main file, I added all of the flow references to the flows that I created in these four files. All of the things you see here are just flow references to the other place. The router is going to route the request to the specific flow from these, depending on which one we're calling, and then it's simply going to reference one of the flows that I have in the other files. That's all this file is doing now. I do not have to modify this file at all anymore. All of my logic is in one of these four files. If I go to, for example, the comments file, I simply took whatever was on the main file and put it here in the flows. This contains all of the flows for each one of the operations. ### Naming convention and reusable subflows Notice the naming convention I used to name each one of the flows. I put first the name of the file, like `resources-comments`, and then I added a colon, and here I'm putting the name of the flow. This makes it way easier to find. If I go back to the main file and click on the flow reference, in the flow name it will ask me for the flow. If I didn't have this first part — the `resources-articles` prefix — then I would just see "update one article". It would be harder to find, in my opinion, because when you check the dropdown, it's huge and has all of the flows from all of the projects. With this naming convention, we have all of the flows from resources-articles grouped together, then resources-categories, resources-comments, and resources-writers at the end. That makes it way easier for me to find the flows I'm looking for. Even though I know which file it is in, when you need to reference them, it's going to ask for the name of the flow without the file. That's why I'm putting the file name here. Make sure you have loggers. If you have nothing in a flow or subflow, you won't be able to run this application. You cannot have a scope that is empty. I just added a logger to have something there so I would be able to run the Mule application. Let's go to articles. I created four subflows at the top that have common stuff that I'm calling from this same file over and over again. For example, we had "retrieve articles" in two different flows. If we modified something from the first retrieve, we would have to modify it from the second retrieve as well and make sure we remember to do that. You can never trust your memory. That's why I'm creating a subflow with the component, so I can just call this subflow from wherever I need to reuse these components. Same naming convention: `resources-articles: retrieve articles in vars`. This name you can just put whatever is descriptive for you. I'm doing the retrieve articles and the store articles in two different subflows. I also created two transformations that are reused in the rest of the file, so I added them here. One is to get the article ID variable — going to `attributes.uriParams.articleId as Number`. Very important that we are making this as Number, because even though we put number in the RAML, the URI parameter is always going to be a string. If you need to transform this URI param into Date or Boolean or Number, make sure to do that from somewhere in the logic. The other transformation is filtering the payload by article ID. This is basically taking all of the payload, whatever the request is, and then I'm just doing a filter to match the ID to the URI parameter. First, I'm retrieving all of the articles with the retrieve component we saw before, and then I'm just filtering them to match the ID that I sent in the request. There's a `[0]` at the end because that means it's getting the first item. ### The Mule event: variables, attributes, and payload Now we get to the logic part. All of the flows have a logger for the start and a logger for the end. That makes it easier for me to understand if there was an error — what flows happened, what flows didn't happen. You can put loggers or not, it's your preference. I just prefer to do that so I can debug this later in the console or in the logs. Before we continue, let me explain the Mule event. The Mule event is immutable. Every change to an instance of a Mule event results in the creation of a new instance. This is what is getting transported from one component to the other. The whole Mule event is this box, and inside the Mule event we have on one side **variables** and on the other side the **Mule message**. Inside the Mule message, we have the **attributes** and the **payload**. When we're retrieving the article ID, we're doing `attributes.uriParams.articleId as Number`. We are accessing the Mule message and getting the attributes. Then we have the payload. When I send a request to this application, the payload is going to contain the JSON that I send in the request, for example. The attributes could be the article ID, for example. The variables I'm going to create them inside Mule, like the article ID variable. If you see here, it says output and then it says `variable articleId`. This means that the variable named `articleId` is being created. You can modify this and you will be able to see: output variable, variable name `articleId`. You can make this an attribute or you can make this the payload. We have attributes, payload, or variables. ### Walking through the CRUD flows Let's walk through the logic. To **read all articles**, we have the flow reference to retrieve articles. You can right-click on it and select "Go to reference flow" if you don't want to just scroll and find it. It's retrieving all the articles, then it's setting the payload to `vars.articles`, and that's it. To **read one article**, it starts, then it gets the article ID variable, then it retrieves all of the articles in the variable `articles`. At this point, we have a variable `articleId`, a variable `articles`, and then we are filtering all of this. We have a Transform Message and we are saying: get from the variables `vars.articles` in order, default for now `vars.articles`, then filter and get all of the articles that match the ID that I put there. That's how we are retrieving one article. To **create one article**, first we are getting the article ID from the payload, because when we create an article, we are not sending any ID. We are not sending a URI parameter with the article ID as we are doing when we have `articles/{articleId}`. When we create an article, we have to get the ID from somewhere. In this case, we are getting it from `payload.id`. Then we retrieve all of the articles, then we append the new article to the articles, then we store the new set of articles, and then we filter the payload by article ID to return that. To **update one article**, first we are retrieving the article ID, then we're retrieving the articles in the variables, then we are updating the articles by article ID. We have the variable `articles` which contains all of the articles, and then we're doing a map. We're going article by article, kind of like a for-each or a for loop. We're saying: if the ID of this article that I'm checking right now matches the article ID that I received from the attributes, then return the payload. If it doesn't match, then just return the same article — don't do anything else. If it matches the ID, then it will put the request in that place instead of the article that was there. Then we're storing the new articles array, and then we're filtering again by article ID to return that at the end. To **delete one article**, again we retrieve the article ID, we retrieve the articles, and then we are filtering by not article ID. We're getting `vars.articles`, all of the articles, and then we're filtering by the ID that doesn't match the article ID from the variables. This is where I had trouble with the data type. Since I was filtering the ID that doesn't match the variables article ID, and the variables article ID was a string when the ID was a number, then nothing matched. This just wasn't working. It wasn't removing anything. I needed to make the article ID variable a number, otherwise this just wouldn't work, because the not-equals is comparing the data types as well. Then we store the new articles, and we finish. ### Building a Postman collection Let's go ahead and build the Postman collection. If you go to postman.com, you can download this from here, whatever matches your operating system. You can create a collection from this side. You can change the theme — I have it on the dark theme. You can also create an account for free, but you don't have to. You can just use this without an account. I'm going to create a collection, and I'm also going to add this collection to the repo so you can use it if you want. If you don't want to create it from scratch, you can just click here on Import and you can import the file, the folder, the link, the raw text, code repository, and so on. I have my collection, which is basically a folder. Then I will add a new request. We have first "Articles GET". For now, I'm going to leave `localhost`. We were using `localhost:8082/articles`, and this is a GET, so we're going to leave it like that. Right now I'm not running it, but if you click on Send, you will be able to send a request and receive a response here. I am using this two-side panel because I work better with that. If you click on View, you will be able to change this — toggle to pane view — like this or like this, if you want to see the request on the top and the response on the bottom. Whatever works best for you. I prefer to use the request on the left and the response on the right. Let's name this "GET articles". You can also name this more descriptive if you prefer, like "Get all articles". ### Environments in Postman The other thing I wanted to show you is that we are going to be changing this URL, because right now we are running in localhost:8082, but eventually we will deploy this whole application into CloudHub and we will have to change this URL to the CloudHub environment. For that, you can go here in Environments and you can create a new environment. One is going to be "local". On variable, let's name this `host`. This is going to be `localhost:8082`, which is what we are using right now. The other one is "dev". This `host` will eventually change to be something about CloudHub. We don't have this URL yet because we haven't deployed to CloudHub, but we will have it at some point and we can just put it here in the current value. Now to reference that, we go here and where we had `localhost:8082`, we're going to remove that and add double curly brackets, and inside here we're going to write `host`. Now we have `{{host}}/articles`. Here on the top, we can change the environment. Right now it says "No environment", but if we click on it, since we already saved the dev and local environments, we can just select which one we are in. If I select local, now this is saying that I am running in `localhost:8082`. I can keep all of the requests for articles right here, and all of the other resources are going to be in their own dedicated folder. This way, I can just come here and run all of these folders and collections. I don't have to do this every time that I need to run it. You can change the environment here from dev to local, and it's the same request and response. Let's create all of the articles requests. Duplicate. Now this is going to be "POST articles". We change this to POST, and then this is our body. To send this body, we can select here Body, Raw, JSON. This is the body that I'm going to send. Duplicate. This is "GET articles article ID". We change this to `articles/{articleId}`. Let's say 1, for example. We don't have a body here, so this is done. Let's duplicate. This is going to be "PUT articles article ID". The body is this JSON. Change the name to "PUT articles article ID". Save. Let me duplicate. This is going to be "DELETE articles article ID". There's no body, so that's all. In the future, I'm going to show you how to create tests if you want to create regression testing. It's super fun, I love to do that. But for now, we are good. For the rest of the resources that you're going to do on homework, if you click on Params, you will be able to add query params. I remember that we had some requests that had query parameters. Here you can add them — just add the key, the query param, and the value. You're going to be able to send that. You also have authentication. In our case, we don't have any authentication. There are headers. Prerequest scripts and tests are to run JavaScript scripts. Prerequest scripts are going to run before you run the request, and the tests are going to run after you run the request to make sure that everything is correct. We're not going to do that now. You're just going to need to use the Params tab for the query parameters, or the Body tab for the body, as we saw in the PUT or the POST. ### Debugging with breakpoints Let's say that I want to check what happens when I read one article. I'm going to right-click on this part and click on "Add breakpoint". This will add this tiny red spot. Ignore all of those problems — the application runs fine. Sometimes DataWeave sends this stuff. It's annoying, but it goes away sometimes. I have this red spot right here, and I'm going to run this. When it gets here, it's going to stop so I can check this out one by one. Right-click on the canvas and click on "Debug project". Eventually, it's going to ask me if I want to change perspectives — basically to show the debugging window or to see the console. The Mule Debug perspective is designed to support application debugging. Do you want to open this perspective now? You can also click here and remember my decision if you wanted to do it automatically. In my case, I don't like when it does it automatically, so I just keep clicking yes or no every time. In this case, I'm going to select yes, and the perspective has changed. You can see here the console still, so it's still running. It's deployed. Here you will be able to see the canvas. Here you have some things like Evaluate Expression, the Mule Breakpoints, the Mule Palette if you want to continue developing. You can also change the perspectives from up here. Right here, we are in the debug perspective. If you want to go back to the other perspective, we have Mule Design, and here you have API Design as well. We wanted to read one article. I have this breakpoint here. If I go to Postman, I will open the "GET articles article ID", and I'm going to search for article 1. I'm missing `/api` — I forgot about that. I'm going to add it here to the environment because I don't want to modify it in every single one. So `localhost:8082/api`. I can just modify the current value. Save. Now this is going to work if I do articles. Okay, it worked. Right now, if I get all of the articles, I have one with ID 2. When I go with the ID 1, I have some testing IDs — ID 3. Yeah, I have some IDs that are repeated because we don't have yet the functionality to stop this from happening. Don't worry about that. I'm going to get just one article. If I do `articles/1` and send — you notice this is not going to stop running because it's stopped right here. Right now, it's on the first logger, and I can come here and see my Mule message. Remember, we had the Mule message with attributes, payload, and variables. Here you can see attributes, payload, and variables. You can open the attributes, and then you can see headers and you can see the URI parameters. If you open URI parameters, you will see that we have `articleId`, and the value is `1` because we are sending number 1. If we check the payload, we don't have anything. It's empty because this is a GET. In variables, we have one — album headers — that's fine, we don't need that. Here on the top right, you have the navigation keys to go through whatever you are developing. If I click here, this is going to move just one space. You saw it moved from the start of the logger to the flow reference article ID. If I click on it again, it's going to go inside the flow reference. Now it's here on the article ID, and it's going to create the article ID variable. If I click on it again, it's going to come out and go back to the flow. Now if I check the variables, I have `articleId` equals `1`. It went to retrieve articles, and now it's going to retrieve all of the articles, which is going to create another variable. If I click on next — it moved again. Now we have `articleId: 1`, and then `articles`, and this has all of the articles. If you want to see a bigger version of what I'm clicking, you can move this here. Here you have all of the payload that we are seeing, and this is better. You can also see here what is the payload, and this is exactly what I got back from the other call where we got all of the articles. After that, what I did was use the filter. Remember, I showed you this — it's filtering from all of the articles. It's filtering the ones that match the ID only. Now if I check the payload again, this only contains one article which matches the ID 1. We can click Continue, and this is now done, and now I have my response. ### Using Evaluate Expression and the DataWeave Playground If I do the same but change this to 3 — remember that I had several articles with the number 3 — I can send this again, and it's going to come here and do the whole thing. Let's stop there. Now we have variables `articleId: 3`, and the `articles` — we have again all of the list of articles that we had before. Then it's gonna do the filtering thing right here. You can see the square — it stopped there. Here's the payload. You can see it's empty. Once I click on it, it's gonna do the filtering and it assigns whatever got back there. Now you might be wondering, why is there just one if there were three different articles? Because for my use case, I'm assuming that there is just going to be one article with the same ID. I haven't created the logic to prevent that from happening, so I ended up with three. But remember that I told you in the code I added this `[0]` at the end. This means that it's getting just the first one from what it got. Let me do a test here. If I get all of this code, here we have the Evaluate Expression — Evaluate DataWeave expression. I can put this code here and click on Evaluate. Now it's super small. Here you can see that it's returning the whole thing. Wait, sorry, I forgot to take out the zero. I have the zero here, and I'm just going to remove it. Now we have `filter ID equals vars.articleId`. Let's click on Evaluate again. Now we can see that we have an array of objects. We have the first article here with ID 3, and we scroll down and we have the second article with ID 3, and now the third article with ID 3. I'm going to show you DataWeave in future content, but I think you need to understand this for the rest of this stuff. Let me go to dataweave.mulesoft.com, the playground. I'm just going to show you this real quick so you can see what is happening. Here we are assuming that this is the payload that has the three articles — one, two, three. This is the payload. I can copy this whole thing to show you what is happening. Just the filter: `payload filter ID equals` — we don't have a variable here, so I'm just going to put `3`. This is super slow. Why is this super slow? Let me try that again. I don't know why this is not working right now. Sometimes when I'm streaming, I have a lot of issues because of all the bandwidth. Anyway, here you can evaluate different expressions so you can see what is happening in your code. If you want to see step by step, just click on Next, and we end up just with one because we are extracting the index 0 from the array that we had. I can also run 2, and it's also going to work because we also had IDs with 2. I'm just going to send it, and then here you can just click on this Resume button and it will just go ahead and continue without you having to go step by step. That's it. We got the ID number 2. What happens if I send an ID that does not exist? Let me click on Next. I receive a `null`, and that is because once I do the filter, I am setting up first the default — `default []` — and then square brackets. That means if there are no articles, then make this default to an empty array. In this case, there are articles, but then after we do the filter, it ends up being just an empty array because there is no article that matches the ID number 4. First we have the empty array, but then when I do the `[0]`, then I will receive the `null`. We can actually fix that if I put here on the code, after the zero — because I'm getting a `null` — I can use another default here and just put an empty object, for example. Save this, and this is going to start running again. Now, instead of receiving the `null`, I'm going to receive the curly brackets. This is just some examples so you can kind of understand how DataWeave works and what you are doing in DataWeave. I am now seeing that I'm not going to have time to do the rest of the implementation, so I hope this works for you. We can just continue the implementation on the next session as well. It started, so we can run this again. Send, and Continue. Now we receive the empty brackets. ### Wrap-up It's just taking me so much longer than I was expecting because it's just so much content that I want to cover. We're going to do that, but this is a great experiment for me to see what content you all need out there so I can create this content clean and upload it and show it to you step by step in a better version — like three minutes, five minutes, stuff like that. Thank you so much for joining the session. I will see you next week, July 26th, for session seven, to show you the finished implementation and to show you how to deploy the API to CloudHub. I also think that I should have done a way easier API, like a to-do or something like that, but oh well. It will be for the next content I create. Thank you all for experimenting this thing with me, and please feel free to show me whatever feedback you have for me. I am happy to help you all. I'll see you next week. Create those Postman collections. Bye! --- ## Session 7: Deploy the Mule App to CloudHub Runtime Manager | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=SUwnqoq8ZbI | Page: https://prostdev.com/video/mulesoft-from-start-deploy-cloudhub-runtime-manager | Series: MuleSoft from Start: A Beginner's Guide ### Introduction and Local Demo Welcome back! Hopefully we'll have just two more sessions to go after this one, and we'll be done with the MuleSoft from Start collection. We have articles, comments, writers, and categories—all of the GET, POST, PUT, and DELETE methods we've been building. I spent time working on improvements to the code since last week. I said it wasn't fair to ask you to finish the implementation because there was a lot to do. I finished the categories resource and changed some of the articles resource, so you can check out what I did in the latest commit. Let me show you what's working now. If I get all of the articles, I currently have id6—that's all, just one article. If I try to get that article (id6), it works and returns a 200 status. But if I try to search for id1, which doesn't exist, it returns a 404 Not Found with the error message: "The ID you're using to retrieve information does not exist. Try a different ID." That's how I decided to set it up. The same logic applies to PUT—if I try to update id1, it says the ID doesn't exist. For DELETE, if I try to delete an ID that doesn't exist like id1, it just returns no content, which is fine since DELETE doesn't return anything anyway. ### How IDs and 404s Work Let me delete id6 to show you how the auto-increment works. Now if I retrieve all articles, I have an empty array. I decided not to return a 404 here—I just want to show that there are no articles. You can change this behavior if you want. Now let's create a new article. Notice that even though I put ID3 in my request, it's not going to use ID3. I want the IDs to be unique and auto-generated, so the flow creates its own ID and ignores whatever you send. Let me check the categories first. If I get all categories, I already have "exercise" from before. Let me delete it and start fresh. Now if I retrieve categories, it's empty. Now let's create an article: "The importance of regular exercise" with category "exercise". Notice the ID—I sent ID3, but it created id7 because it's keeping the count going. Now if I retrieve that article, id7 is there. And if I check categories, we now have "exercise". Let me create another article: "The importance of DataWeave" with category "DataWeave". Now when I get all articles, we have id7 (regular exercise) and id8 (DataWeave). And if I get all categories, we have both "exercise" and "DataWeave". ### Deploying to CloudHub from Studio Now I want to send this to CloudHub because I see this is a working first draft. I've tested it locally and it makes sense, so now I want to deploy it to dev for other people to test. The manual way to deploy, which is the easiest when you're just getting started: make sure everything is saved, then right-click on your project and select Anypoint Platform → Deploy to CloudHub. ### Re-authenticating Anypoint Accounts I'm having an authentication issue. Let me go to Anypoint Studio settings and check my Anypoint Platform authentication. I'm going to remove all my accounts and re-add them. Let me sign in with my credentials and close the settings. Now let me try again: right-click, Anypoint Platform, Deploy to CloudHub. That helped—now it's working. ### Deployment Configuration and Environment Property The deployment dialog asks where I want to deploy this application called "maxines-blog". I'm going to select CloudHub 2.0 (you could also select CloudHub, but I'm choosing 2.0). I can only select the shared space in US East Ohio, so that's fine. The application file will be uploaded from this project. I'm leaving the runtime as is and making sure to use Object Store v2 because I am using Object Store. The deployment model can stay as default, and Ingress is empty—that's fine. Now for properties, there's something important we need to add. I'm going to create the environment property, and the value is going to be "dev" instead of "local". This property is going to override the property we had that said "local". I don't want to protect it because I want to be able to see which environment I'm in, and I don't have any passwords or anything that needs protection at this point. Let's deploy the application. It's deploying now—we can close the window and see it deploying in the background. ### Getting the Public CloudHub URL We can already get the public endpoint. You can see it here in Settings, or if you go to the Dashboard you'll also be able to see it in the URL. Either way works—just copy this URL. ### Updating the Postman Dev Environment Let's go to our Postman environments and update the Dev environment. The current value is that CloudHub URL, then slash API. That's it—that's our new URL. We can save this, and now we'll be able to test it. Let me change to the Dev environment here. Now this will be connecting to CloudHub instead of localhost. Let me try it—it started, finally! Now we can test from here. ### Testing the Live API We should have nothing there because we're on dev now. Let me try to get all articles—perfect, empty array. It should appear in the logs. Yes, I can see "starting" and "ending" in the logs. If we get categories, we have nothing. So let's post an article: "The importance of regular exercise" with category "exercise". Let's send this. It was created with ID1 because this is our very first article in the dev environment. Now if we try to get the categories, we have "exercise"—perfect! ### Managing Object Store in Runtime Manager The sweet thing about CloudHub is that we can check the Object Store from Runtime Manager. If we look at Object Store now, we have keys. We have "articles", we have "categories", and we have "next-article-id". We can see the keys but not the actual values stored there. The really cool thing is that we can delete stuff from here to try it again. Right now we have one article, right? If I check, we have one article with ID1. But if I want to just restart everything, I can remove all of the keys. Delete three keys—yes, I understand, delete. Now we go back to square one. We have no articles and no categories. If I post an article again, I will have ID1 again because I restarted everything. For example, if I restart just articles and categories (delete those keys) but don't remove the next-article-id key, then the next ID that gets created will be ID2. If I get articles, I have nothing. If I get categories, I have nothing. But if I post a new article now, the ID is number two because I didn't restart that counter. Now I have ID2 and the category "exercise". That's the sweet thing about having this on the cloud—I can control the Object Store easily. I'm sure there are ways to control it locally, but they're not as neat as this one. ### Wrap-Up and Next Session I'm going to remove everything again. That's all for today. I'm sorry I wasn't able to show you API Manager working—I just don't know why it's not running. But for the next session, I will show you what went wrong and how I fixed it. So I will cover API Manager for the next session, and then we'll do CI/CD with GitHub Actions. Every time you publish something to GitHub, it will automatically create the new deployment. I hope this was helpful. We did get to do a lot of troubleshooting, and I'll create more posts from everything I learned today. I will see you on the next session on August 2nd for session number eight. Thank you so much for joining, and I'll see you then. Bye! --- ## Session 8: Set up CI/CD & API Autodiscovery (API Manager) | MuleSoft from Start: A Beginner's Guide Watch: https://www.youtube.com/watch?v=OWfqLMPQMpE | Page: https://prostdev.com/video/mulesoft-from-start-cicd-api-autodiscovery | Series: MuleSoft from Start: A Beginner's Guide ### Setting up connected app credentials I grabbed the client ID and secret from my environment in Access Management. To make things clearer, I renamed these to "connected app client ID" and "connected app client secret" instead of just client ID and secret. Then in the configuration, I updated the property names to "anypoint platform client ID" and "anypoint platform client secret" so it's obvious what these credentials are for. ### Passing credentials via the terminal Now that we have those properties defined, we can pass them in via the terminal when we deploy. This way we don't have to put them in manually every time. I ran the deployment and it was successful—the app started up. ### API autodiscovery and API Manager The app was initially showing as unregistered in API Manager. I just needed to send a request to trigger the autodiscovery. I went into Postman and sent a request. The Object Store was empty at first since we didn't have any data yet. I created an article with ID 1, then retrieved the articles and saw article 81 in the response. I checked the Object Store in the Anypoint Platform and saw the data: articles, categories, and next article ID. Everything was working correctly. The only difference from before is that I created my CI/CD pipeline to deploy to CloudHub 1 instead of CloudHub 2. After the deployment, the app was now registered and active in API Manager. ### Testing the basic authentication policy Now we can see the policies in API Manager. We're using the basic authentication policy that we added at the top of the configuration. We have the authorization header in our request. If I were to change or remove that header, it wouldn't work anymore—sure enough, I got "authentication attempt failed." I added the correct credentials back and tried again. Perfect, it worked. ### CI/CD with GitHub Actions We finished setting up API autodiscovery and also got CI/CD working. I showed you how to do it with GitHub Actions, but I didn't actually implement it in this repo because this repository isn't just a Mule application—it has a lot of things and I think it would be messy. If you have any issues setting up GitHub Actions, you can let me know. You can also check out the MuleSoft MFA CI/CD repo that I showed you. If you go to `.github/workflows`, you'll find the `build.yaml` file. As I mentioned, if you just want to implement the changes that I did, you only have to change the workflow file and the `pom.xml`—those are the only things we changed. ### What's next: MUnit and Nexus We are so close. In the next session we will do some MUnit tests manually, and I will also show you how to add the MUnit stuff to your CI/CD pipeline. Please note that to add it to CI/CD you will need to have a Nexus Enterprise account. If you don't have it, don't worry—you can still check your MUnit tests manually. We will also take a look at the Postman tests that you can do if you want to do some of that, and I will just mention the bat CLI, but I will not go through it. ### Wrap-up That's it. Our next session is the last one—August 9. We are so close. I hope that everything has been working correctly for you. If not, please let me know. I am happy to help you. If you have any questions at all, please send them to me. I will put everything on the repo so you can go and check out all the changes that I did and also all the links that I showed you today. I will see you next week at the same time. Bye. --- ## Part 1: How to set up a CI/CD pipeline to deploy your MuleSoft apps to CloudHub using GitHub Actions Watch: https://www.youtube.com/watch?v=fcFJ3Jhgaw4 | Page: https://prostdev.com/video/cicd-mulesoft-github-actions-cloudhub-deploy | Series: MuleSoft CI/CD with GitHub Actions ### Intro Hi, I'm Alex Martinez, and today I'm going to show you how to create your CI/CD pipelines with GitHub Actions for your Mule applications. I have a simple Mule application here that is simply setting up a payload that is just a string saying "it worked", and a logger that will output that payload into the console. That's all that this is doing. So let's run it to make sure that it works before we start with the CI/CD pipeline. This is now deployed and ready to be called. I'm going to use cURL to make this easier. I'm calling `localhost:8081/test`, so once I call this I receive the "it worked" string that I had set up. So I know this application is working. ### Creating the workflow file Now inside of the Mule application, let's create a new folder called `.github`, just like that, and then inside this folder let's create another folder called `workflows`. There you go. Now inside this `workflows` folder we're going to create a new file called `build.yaml`. Once you have that file, you'll have to go into the article from the link in the description, copy the script that I put there in the article, and paste it right here. ### Walking through the build job Let's take a look at this file to see what this is doing. The name of this workflow is "build and deploy to sandbox". We're going to trigger this workflow every time that there is a push to the main branch, or that a pull request is merged into the main branch. In this case we have two jobs: build and deploy. The build job is going to be running on Ubuntu and the steps are listed like this: - The first step is going to check out this repo into the Ubuntu machine. - Second step is to cache all of the Maven dependencies. - Third step is to set up JDK 1.8 — this is the Java version that I chose; you can also use Java 11. - Then we're going to build this with Maven using the pom.xml from this Mule project. - After that the artifact will be stamped with the commit hash. This is just so you can go to CloudHub, and if you want to download the JAR file or make sure that this is the JAR file that you're using, you can check that it has the commit hash that you need. - Finally, once the JAR file is created, we're going to upload this artifact into the workflow so it can be used by other jobs. ### Walking through the deploy job Now if we scroll down to the deploy job, first of all we see here that it says `needs: build`. What this means is that GitHub actually runs each job asynchronously, but in this case we don't want it to run asynchronously — we want them to run synchronously to each other, because first we need to build the application and then we need to deploy the application. We cannot do that separately. So in this second job we're stating that we actually need the build job to run first. Then you'll see the same first steps that we had: it's running on Ubuntu, it's checking out the repo and caching the dependencies. After that it's going to download the artifact, or the JAR file, that was previously uploaded from the previous job, and then it'll deploy everything to sandbox. In this case our Anypoint Platform username and password are going to be used from here. I'll show you how to set this up from GitHub in a moment, but we'll be using Anypoint Platform username and Anypoint Platform password. Once these are extracted from our GitHub account, they'll be used as username and password environment variables. So now here we can see the Mule deploy that will be running using the Mule artifact, then the same name of the JAR, the username and password from Anypoint Platform. ### Adding the GitHub secrets Now if we go to our repo in GitHub, you can click on the Settings tab — make sure that you're logged in — and then scroll down and go into "Secrets and variables", and then "Actions". This will bring you to this page where we currently don't have any environment or repository secrets. Let's click on "New repository secret". Here, select the Anypoint Platform password name for the variable, and then the secret is your actual password. After that just click on "Add secret". Once you've added that secret, now let's go ahead and create a new one for the username. In this case this is going to be Anypoint Platform username, then add your own username and click on "Add secret". Now that we have our credentials here in GitHub, there's no need for us to hard-code the credentials directly in the workflow. Please don't do that — that is a huge security issue. ### Running the workflow So now all that's left is to run the workflow. If you remember, our triggers are either a push or a pull request merge into the main branch. So let's save this file and push it into the repo. Now once you push it, you'll be able to see here that you created this new commit, and here you'll be able to see the state of the action. You can also click here on the Actions tab and this will take you to all of the workflows that had run before. Now if you click on the latest workflow — that's the one that you just triggered — you'll be able to see the details of each step. Here you have the build job that already finished running, and then we have the deploy job that just started running. If you click on it you'll be able to see the details of what is running in each step and the outputs from the command line. As you can see here, it's checking whether the application is started or not. So if we go into Anypoint Platform and Runtime Manager, we'll be able to see that this is currently deploying. Once this finishes deploying, we'll be able to see a green state on our Actions. After our application finishes deploying correctly, we'll now be able to see that we have a green status in both of them. ### Wrap-up And that's it. I hope this video was useful. I'll continue to do more videos about the same topic — continue to the next video to see how to use encryption and decryption of your Mule properties in your CI/CD pipeline. See ya! --- ## Part 2: CI/CD pipeline with MuleSoft and GitHub Actions - secured/encrypted properties Watch: https://www.youtube.com/watch?v=qao50tJn_zs | Page: https://prostdev.com/video/cicd-mulesoft-github-actions-encrypted-properties | Series: MuleSoft CI/CD with GitHub Actions ### Intro Hi, welcome back to the series of CI/CD pipelines with GitHub Actions using MuleSoft. In this video we're going to learn how to decrypt properties using the CI/CD pipelines that we created in the last video. Let's do it. ### The encrypted property setup First of all, I already have my application. I did some changes — here I added the secure property using `${secure::encrypted.property}`. This is the name of the property that I'm going to be using. I created a new file under `src/main/resources` called `secure.properties`, and if I go here I'll see my `encrypted.property`, and as you can see this value is already encrypted. Also, something to keep in mind: in your `mule-artifact.json` file it's important that you set up here the `secure.key`, or the name of your key, into the secure properties field. This is so when you deploy to CloudHub this property will be hidden and no one can see it. Otherwise anyone that accesses CloudHub will be able to see your key. ### Running it locally Now let's run it to see that it's working. If I go to Run and then Run Configurations, I'll be able to set up my key in the Arguments tab, and I set it up here in the VM arguments part. So my secure key in this case is `myMuleSoftKey`. After that you can just click on Run. So my application is deployed. Let's now test that it works, same as before: `curl localhost:8081/test`. Let's run this to make sure that it works, and I received "it worked secure property" — yay. ### Creating the GitHub secret So now the first thing that we're going to do is to create the secret in GitHub. In Anypoint Studio we set up the `secure.key` (`myMuleSoftKey`) from the VM arguments in the Run Configurations. Remember from the previous video that if we go to GitHub in our repo and then click on the Settings tab, we'll be able to scroll down, select "Secrets and variables", and then "Actions". This will bring up our previous repository secrets, which we already set up as Anypoint Platform password and Anypoint Platform username. Let's now create a new one that's going to be called `decryption.key`, and as we had set up in Studio, this key in my case is `myMuleSoftKey`. So we're going to add this secret, and that's all for this step. ### Updating build.yaml Now if we go back to our Mule project and open the `build.yaml` that we had created in the previous video, let's scroll down until we see the deploy job. In this case we have to add the new decryption key here in the environment variables. Remember that we set up our `decryption.key`, so this name has to match with what we put in the command. So here, right after password, we're going to add a new key, and this is going to be referencing the secret that's called `decryption.key`. Now we're going to add a new parameter here in the Maven command, so right after this one let's add a new `decryption.key` that's going to be referencing the key variable that we just created right here. Make sure you add a backslash right here, otherwise this is not going to work. You might be wondering why this is called `decryption.key` and not `secure.key`. This is because we'll be changing this in the pom.xml, as I'll show you in a moment. We can now save this file. ### Updating the pom.xml Now for the next step let's open the pom.xml and search for the CloudHub deployment. This is going to be inside `build` and then `plugins`, then `plugin`, and we'll see here the configuration for the CloudHub deployment. So here we're going to add a new property before the CloudHub deployment tag ends. Let's go ahead and create here this new property. We have to create first a field called `properties`, and inside it we're going to be referencing the `secure.key` that we'll be using inside the Mule application, and the other value that we're going to be referencing is `decryption.key`. So the value that we send here from `build.yaml`, which is the encryption key, will be taken and will be put inside here. So the `secure.key` property will contain the value that we have in our GitHub secrets. So if you have a different key instead of `secure.key`, you have to change it here in the pom. Let me save this file, and also the `mule-artifact.json` here, so you can make sure that this key is not visible from CloudHub. ### Running the pipeline And same as before, as soon as we push this new commit, we'll be able to see here in the Actions tab that this workflow is starting to run. If you click on it you'll be able to see the details of each job separately. So the build was a success after 30 seconds, and now this is trying to deploy the application. We can already see in Runtime Manager that the application is being deployed — it's updating right now. As soon as this is updated in Runtime Manager, our deploy job will be successful. And now that the application has been successfully deployed in Runtime Manager, we can go back to our GitHub Actions and we'll see that everything is green. Let's try to call this application just to make sure that it actually works. Let's send the cURL request to our CloudHub URL and then `/test` — and "it worked secure property", yay. ### Wrap-up And that's all for this video. I hope you liked it. In the next video we're going to see how to incorporate MUnit into the CI/CD pipeline, so keep posted when this video is uploaded. Remember to subscribe to watch all of the other videos, and I'll see you in the next video. Bye! --- ## Part 3: CI/CD pipeline with MuleSoft and GitHub Actions - MUnit testing Watch: https://www.youtube.com/watch?v=ksAOahtpd6g | Page: https://prostdev.com/video/cicd-mulesoft-github-actions-munit-testing | Series: MuleSoft CI/CD with GitHub Actions ### Intro Hi, I'm Alex Martinez, and in this third video from this series I'm going to show you how to add MUnit to your CI/CD pipeline. There's something important here: you'll need Nexus credentials, so if you don't have them you'll have to get them before starting this. For more information about the Nexus credentials, you can go to the article that's linked in the description of the video to see the links to the documentation to learn what exactly that is, if you don't know. In the two previous videos we learned how to set up the basic configuration for the CI/CD pipelines, and how to add a configuration for any decryption that you need to do in your properties. Let's do it then. ### Confirming the MUnit tests work So before starting, we have to make sure that our MUnit tests are actually working and they're successful, otherwise we might encounter some issues in the pipeline because we did not run this earlier. So here you can see that my two tests are working — everything is in green, so I'm ready to proceed. And as I mentioned, you'll have to have Nexus Enterprise credentials for this. So if you're not sure if they work, you can come to this link — I'm going to put it in the description of the video, or it's also in the article that I wrote — and here you'll be able to put your username and password to make sure that your credentials actually work. So once you add your credentials, simply click on Login, and you'll see this page because you're able to log in. As you can see here, I'm logged in to my user. ### Adding the Nexus secrets to GitHub The next step is to go into your GitHub repo, and same as before, go into the Settings tab, scroll down, and select "Secrets and variables", "Actions". We already had three different repository secrets that we set up in the previous videos, so now we're going to create a new one. First we're going to add the Nexus username that you just used in the previous screen — in my case `amartinez-mule` — and click on "Add secret". Now let's create the second one: in this case we're going to set up the Nexus password, and you put here your own credential and click on "Add secret". Now we have set up five different repository secrets. ### Modifying build.yaml — the test job The next step is to modify our `build.yaml` file, which is the one that's running the workflows for the GitHub Actions. Please go into the description of the video and click on the article; you can copy and paste the script that I'm about to show you here. As you can see, here by these green lines, these are the new changes that we're adding. We're adding the new test job, and we're adding the new `needs: test` into the build job because we need to run this synchronously, as I explained in a previous video — we need to run the test job before the build job. There are two other additional changes in the build and the deploy jobs. One is here in the build job: if you go to the "build with Maven" command, I added the `-DskipMunitTests` so we can send this parameter to not run the MUnit tests on these two other jobs, because we'll already be running the tests on the first job. If you scroll down to the deploy job, you'll also see that I added the same thing here in the Maven deploy command. So now if we take a look at the test job, the first steps are the same as the other ones — Ubuntu latest, and then we check out this repo, cache dependencies, set up JDK 8 (again, you can set up another JDK if you want) — and then here in the "test with Maven" command, this will change a little bit from what we've been doing before. Here are our two secrets that we had set up, Nexus username and Nexus password, but in this case the variables that we'll be creating here will not be used in our Maven command. I'll show you in a bit where we're using this. And then if you have a decryption key, you have to send it in this command as well. Also, in this case we're sending the decryption key directly here as the `secure.key` property, as opposed to what we were doing before, which was setting up the decryption key. Now we have to set up the property directly, so if you have a different property name that's not `secure.key`, please make sure to modify it in this parameter. The other thing that we can see here in this command is that we're setting up a `settings.xml` file directly. So in this case we're saying that it's under the `.maven` folder and then the `settings.xml` file will be found there. ### Creating the settings.xml So we have to create it in a root folder. We're going to create a new folder called `.maven`, and then inside this folder we're going to create a new file called `settings.xml`. Now, again, you'll have to reference the article that's posted in the description of the video so you can copy and paste the contents of this file. In this case we're adding a new server that's called `mule-repository`, and we'll be linking the Nexus username and Nexus password from our secrets. We're also creating a new profile that's called `mule` and it contains this repository for the Mule repository that matches the server. So now this Nexus username and Nexus password will be taken from what we're sending here in the `build.yaml` — these two environment variables that are not being used anywhere in our Maven command will be passed down and will reach the `settings.xml` here. ### Running the pipeline And that's it. Those are all the changes that we need to do in order to run these tests. So now let's save both files and push all of these changes into our GitHub repo to run the GitHub Actions. And same as before, as soon as we push all of the changes we'll be able to see that the action is running. Now if you click on it you'll be able to see the details of each different job — in this case now we have first the test job, then the build, and then the deploy. Let's open the test job to see how it's doing. And after roughly two minutes we're able to see our MUnit coverage summary, as well as the number of tests and if there were any errors, failures, or skipped. In this case there were two tests and both of them were successful. Notice how I only have 33% of the application coverage with my MUnit tests — in the next video we'll see how to make sure that this coverage is a higher percentage so we don't let this happen in a real-life scenario. The build already finished, and now it's up to the deploy now, and we can see in Runtime Manager that this application is being currently redeployed. And once again, as soon as this application is deployed, you can go back to see that all of the three steps have been successful. ### Wrap-up And that's all for now. In the next video we'll incorporate the MUnit coverage into the CI/CD pipeline. I hope you liked the video. Remember to subscribe, and I'll see you in the next video. Bye! --- ## Part 4: CI/CD pipeline with MuleSoft and GitHub Actions - MUnit minimum coverage percentage Watch: https://www.youtube.com/watch?v=DKQA5gRXvn0 | Page: https://prostdev.com/video/cicd-mulesoft-github-actions-munit-coverage | Series: MuleSoft CI/CD with GitHub Actions ### Intro Hi everyone, Alex here. In the last video we learned how to include our Nexus credentials into our `build.yaml` file — just like the GitHub credentials — in order to be able to deploy our application using the MUnit tests in our new CI/CD pipeline. Now, in the last video we didn't learn how to actually do MUnit coverage; we only learned how to run the MUnit tests from the CI/CD pipeline in GitHub Actions. So in today's video we're going to learn exactly that: how to add the MUnit coverage into our CI/CD pipeline. ### Configuring coverage in the pom.xml This part is going to be actually way simpler than the last one, since we already have all the Nexus credentials and all the configuration to run the MUnit tests. The only thing that we're going to add here in the pom.xml is the configuration for the necessary minimum percentage required to pass the build. So inside the pom.xml we can scroll down until we see `build` > `plugins`, and then we'll search for the MUnit plugin right here, `munit-tools`. The first thing we're going to add is going to be here in the configuration part. Inside the configuration we're going to add these environment variables, which is `secure.key` — this is the decryption key that we've been using from the previous articles — and here we're going to set up the `decryption.key`, which is what we have set from our GitHub Actions. If you want to just confirm that all of this configuration is set up properly, you can just go to the previous articles or the previous videos to make sure that your `secure.key` or your `decryption.key` are set up properly. After setting that up, you can just go here inside `coverage` and you'll notice that you're already running the coverage. So now under that we're going to add `failBuild` set to true, so if this configuration is not completed properly then the build is going to fail — that's exactly what that means. And then we're going to set up the required application coverage to 90. In this case this is my own configuration; I want it to have a 90% coverage minimum in order to pass the build, but you can set up however you want. Other options that you could also have are `requiredResourceCoverage`, so for every resource that you have you'll have, for example, a 50% minimum coverage, and another setting would be `requiredFlowCoverage`, also let's say for example 50%. So you can set all of this up, or you can set just one, or maybe two, or however you want this to be configured. In my case I just want the whole application coverage to be 90% minimum, so that's what I'm setting right now. Now you can leave this configuration like that and it will create an HTML coverage report, or you can also set up more formats like console, sonar, JSON. But honestly, I feel like only the console and the HTML are really useful. The JSON is useful if you're going to be using this information for another pipeline or for another thing, and the sonar, well, if you're running SonarQube. ### Uploading the reports as artifacts Now we can simply save what we have here in the pom.xml and that's it. Or, if you want to see all of these reports from the GitHub pipeline, then you have to add something into the `build.yaml` file. So if we go here to `.github/workflows/build.yaml` and we scroll down in the test job all the way to the end, we'll be able to add here another step, which is to upload the MUnit reports. Basically, using the `actions/upload-artifact`, we're saying name this artifact "munit-test-reports" and take whatever is in `target/site/munit/coverage` and put it there. For more information about this configuration you can just go to the docs — I'll add this link in the description of the video, or if you go to the article in ProstDev you'll be able to see this link. ### Running the pipeline So now that we have all of these changes in pom and build, let's simply push all of these into our branch and see how it's running. And now if we go to our GitHub repo in the Actions tab, we'll see that this is already running. We can open it and see that it's running the testing, and at the end of the test we'll be able to see the console report. Here I have all of my coverage at 100% — so I already covered every single thing from my flows, so I have 100% coverage, everything passed, and this build is a success, at least from the test. Right now it's trying to deploy and we'll wait until that's done. And once this is finally deployed, we can see the status is "started", and we can go back to the GitHub Actions — this is all green so everything was done properly. Now if we scroll down here, we'll be able to see some artifacts right here. So we have the artifacts, which is where we have our JAR file, and the MUnit test reports, which is where we can find our sonar, HTML, JSON, and I believe the console is already done, so you'll be able to find those there. If you click on it, it'll download a zip file and you'll be able to unzip it and see the results there. ### Wrap-up And that's all for this video. I hope you liked it, I hope it works for you, and as you can see it was very simple — you just have to set up the pom.xml, and if you want to also get the artifacts from the CI/CD pipeline, you also have to set up the `build.yaml` file, but it's not that hard. I think I'm pretty much done with everything you all have been asking me, but on the next video I'll go ahead and explain how to sign in to Anypoint Platform using GitHub Actions if you're using MFA, or multi-factor authentication, in your Anypoint Platform account. So far, since I've been using free accounts, I don't have to have MFA activated, but a lot of the Enterprise users — I believe all of them — have to use MFA in their organizations. Instead of using the Anypoint Platform password and username, you'll have to use a Connected App, and I'll show you how to do that in the next video. Let me know if you have more suggestions of what else you want me to cover in this content of CI/CD pipelines, and I'll try to make it happen. All right, that's all for this video. Bye-bye! --- ## Part 5: CI/CD pipeline with MuleSoft and GitHub Actions - Enabling MFA through a Connected App Watch: https://www.youtube.com/watch?v=jvtw3fcSyKY | Page: https://prostdev.com/video/cicd-mulesoft-github-actions-connected-app-mfa | Series: MuleSoft CI/CD with GitHub Actions ### Intro Hey everyone, Alex here. In the last series of videos — the last four videos we've seen — we've been signing in to Anypoint Platform using username and password. And that's cool and all, because we've been using free accounts (or I have been using free accounts so far). But if you're using an Enterprise account, it's most likely that you'll have multi-factor authentication, or MFA, enabled in your organization, which means that you have to sign in using an SMS or another application so you can sign in from your phone, get a code, stuff like that. So there's a different process altogether to do the authorization with CI/CD pipelines using multi-factor authentication, which is using a Connected App instead of your Anypoint Platform username and password. In today's video we'll learn exactly how to do that. So let's get started. ### Creating the Connected App The first thing we're going to do is to sign in to Anypoint Platform, go to the menu here, and go to Access Management. Once you're in Access Management, please go to Connected Apps right here and select "Create app". You can name this whatever you want — in this case we're going to name it "GitHub Actions" just like that. Now select "App acts on its own behalf (client credentials)". After that click on "Add Scopes" and we'll add the following scopes: the first is going to be Design Center Developer, then we're going to select View Environment, View Organization, Profile, CloudHub Organization Admin, Create Applications, Delete Applications, Download Applications, Read Applications, and finally Read Servers. After you select all of those scopes, click on "Next" right here, select your business groups (in my case it's only one), select "Next", and we'll be using the Sandbox environment for this demo. Next, click on "Add Scopes", and after we do all that we just have to review that everything looks good and click on "Save". Now here you'll be able to copy the ID and the secret that we're going to use in the next step to add to GitHub. ### Adding the secrets to GitHub Now if you go to your GitHub repository, click on Settings, then scroll down and select "Secrets and variables", "Actions", and this will get you to where we've already selected all of our previous repository secrets. In this case we're going to create two more for the ID and secret that we just saw. Simply go here and click on "New repository secret", and the first one is going to be Connected App client ID. So now if we go back to Anypoint Platform we can select "Copy ID", we'll go back to our GitHub repo and paste here the secret. Now just click on "Add secret", and now repeat the process for the second secret, which is going to be Connected App client secret. Now we go back to Anypoint Platform, copy secret, go back to GitHub repo and paste this, select "Add secret". ### Modifying the pom.xml Now that we have the setup, we can go into our actual repo to modify the code. The first thing we're going to do is to modify our pom.xml file. You can scroll down until you find the CloudHub deployment, and in here you can notice that we have username and password. We're going to replace those two with our Connected App credentials, and these are the three properties that we're going to add now: Connected App client ID, Connected App client secret, and Connected App grant type. The grant type is going to be `client_credentials`, and the ID and the secret are going to be parameters that we're going to send from our `build.yaml`. So that's all for the pom.xml. ### Modifying build.yaml Now if we go to our `build.yaml` file that we have under `.github/workflows`, we now have to go into our deploy job. So we have the test, build, and deploy. If we scroll down we'll be able to find here that we have username and password in our environment variables, and we're connecting to Anypoint Platform username and Anypoint Platform password — these are the secrets that we have in our GitHub settings. So let's replace that with ID and secret like this. Now we have the ID environment variable, which is connecting to the Connected App client ID secret that we just created, and the secret variable connected to the Connected App client secret secret that we just created in our GitHub settings. And the second thing we need to change is right in the Maven command that we're sending. We have to modify this `anypoint.username` and `anypoint.password` with the two variables that we just created, like this. So now this is going to be `client.id` connected to the ID variable, and `client.secret` connected to the secret variable. These are the two properties that we're going to receive here in the pom.xml. ### Running the pipeline Now we just have to push all of these changes into our GitHub repo to see the pipeline running. So now if we come to the repo and click on the Actions tab, we'll be able to see that the workflow is now running. You can click on it to see the details of what's happening. Right now it's running all of the MUnit tests that we had previously set up, and once that's done it's going to continue to the build and the deploy steps. Once it reaches the deploy job, we can see that this is already deploying in Runtime Manager, and as soon as this is done we'll have a successful pipeline. Once the application appears as "started", we can go back to the pipeline and we'll see everything was successful. ### Wrap-up And that's it. That is how you configure a Connected App from Anypoint Platform into your CI/CD pipelines when you're using an Enterprise account, or just using multi-factor authentication in your account. I hope this has been useful for you. Please let me know in the comments if you would like me to create any more content about CI/CD pipelines or any other type of MuleSoft content. All right, that is all. Remember to subscribe to the channel so you never miss the new videos that we create, and also subscribe to ProstDev.com so you receive notifications as soon as new articles are created. Bye-bye! --- ## Part 6: CI/CD pipeline with MuleSoft and GitHub Actions - Deploying to CloudHub 2.0 Watch: https://www.youtube.com/watch?v=hfQaYcjUB9c | Page: https://prostdev.com/video/cicd-mulesoft-github-actions-cloudhub-2 | Series: MuleSoft CI/CD with GitHub Actions ### Intro Hello hello everyone, my name is Alex Martinez, I am ProstDev's founder, and today we are going to learn how to deploy your Mule application using GitHub Actions to CloudHub 2.0. So let's get started. ### Prerequisites First of all, you'll need an Anypoint Platform account, of course. You can create a free trial account if you go to `anypoint.mulesoft.com` and just sign up for a 30-day trial account. You can keep creating free trial accounts as many times as you want, just make sure that you change your username — you can even use the same email. The second thing you're going to need is a Mule application that's already on your GitHub repo. In my case I'm just creating a test, so here in "alexandramartinez-test" I have my private repo. It can be private or public, whichever one you prefer, but make sure that you already have a base project created. If you're not sure how to do this, please take a look at other videos that I have on that. Once you have those two things, then we can get started. ### Creating the Connected App The first thing that we're going to do is to create a Connected App in Anypoint Platform. So if you go here to Access Management, then select Connected Apps from here, and click on "Create app". You can put any name that you want — in my case this is going to be "GitHub Actions". Then select "App acts on its own behalf", so you can create scopes and you can use client credentials. Then click on "Add Scopes" and you'll add the following scopes: the first one is going to come from Exchange, and you're going to select the Exchange Contributor one. Then if you go to General, which is right underneath, and you scroll down, you'll select View Environment and View Organization. Then if you scroll down, select Runtime Manager and select Create Applications and Delete Applications. Then from there also you're going to select Manage Application Data and Manage Runtime Fabrics. You scroll down a little bit more, you can select Manage Settings and Read Applications. Those are the minimum scopes. If you have any questions about this, I have put the link in the description of the video, or also in the article in ProstDev.com. So once you have those nine scopes, select "Next", select your business group (in my case it's SF), "Next", select your environment — I am going to be deploying to Sandbox, so that's the one that I'm going to select — and finally you can just review that all the scopes are correct and click on "Add Scopes". So once you have all that you can click on "Save" and you'll have created your first Connected App. So now you're going to click here to copy the client ID, and click here to copy the client secret. You're going to be needing these two for your GitHub Actions. ### Adding the secrets to GitHub Now you can add your GitHub secrets directly here in Settings, and then if you scroll down you'll see "Secrets and variables", "Actions", and here you'll be able to add your repository secret. So the first one that we're going to add is the Connected App client ID. This is the client ID, or the ID, that you copied from Anypoint Platform, so let's paste that right there and click on "Add secret". Now you can continue to do the same but for the Connected App client secret. Or, if you go to your VS Code and you open your GitHub Actions extension — this is one of the ones that I really like to use because of the secrets — if you click here on "Secrets" > "Repository secrets" you'll see that we already have the Connected App client ID, and if you click on the plus you'll be able to add the new secret and then add the actual secret value. In my case it looks like this, so you can press enter and you'll have both secrets right here. You can also double check that it was created on GitHub — you just refresh the page and you'll see both of them. ### Modifying the pom.xml — group ID and version Once you have those, we can get started on the changes to your Mule application. As you can see, I have a very basic application — it's just an HTTP listener that's setting up a payload and then is outputting the logger, and that's pretty much it. So I can close this. The first thing that we're going to modify is the pom.xml. So you can open that up. The first thing we need to do is to modify the group ID to match our organization ID. So if we go back here to Access Management, you can click on Business Groups, select your business group, and here you'll be able to see your business group ID, which is your organization ID. So copy that and put it here in the group ID — make sure that this matches your organization ID. The next thing we're going to modify is the version. So if you have here the `-SNAPSHOT`, make sure to remove that, so it only says `1.0.0` or whichever version you may have, but just remove the snapshot. ### Modifying the pom.xml — Mule Maven plugin Next, if you take a look here at the properties, we have the Mule Maven plugin version — in my case it's 3.3.5, so I want to use 4.1.1. Actually, the minimum version that you should use is 4.1.0. The next thing we're going to modify is the configuration for the Mule Maven plugin, so go ahead and go to the article in the description of the video so you can copy and paste the thing that I'm going to put here. So I recopied it, and then if I paste it I'll be able to see this. We have the artifact ID, which is `mule-maven-plugin`, the same as we had before, and it's going to be using the property that we had at the top, which is 4.1.1 in my case (if there's a more recent version I would advise you to use that, just make sure that it works). And then the configuration: we have the classifier `mule-application`, the URI is `anypoint.mulesoft.com`, the provider always has to be MC if you're using a CloudHub 2.0 environment, in my case it's going to be Sandbox (you can change this to match whatever environment you're using). The target in this case is going to be `cloudhub-us-east-2` because that is the one that I have in my free trial account. So how do I know that? If I go here to Runtime Manager and I click on "Deploy application", you'll see here that if I select CloudHub 2.0, there's only one shared space that I have available, which is "US East (Ohio)". So then how would I know that this is US East 1 or US East 2? If you go to the link in the description of the video or in the article, you'll find this link to the regions and DNS records for CloudHub 2.0. So for example in my case I had US East (Ohio), which is right here, and if I take a look at the target ID I can see that it's `cloudhub-us-2`, but I don't need the target ID, I need the target name. So if you take a look at this you'll see that there are some capital letters that are different, so you really have to pay attention to that. In my case this is my target name, so it's `Cloudhub-US-East-2` with capital C, US, and E right after that. ### Modifying the pom.xml — server, app name, and replicas Then I have the server, which is in my case "Repository". You can put any name that you want here, any name at all, just make sure that it matches some other configurations that I'm going to show you in a moment. This server is going to be able to do the actual authentication to Anypoint Platform for us. In the previous videos that you've seen from this series, we have done Anypoint Platform username and password, and we have also done Connected Apps directly here in the pom — instead of having the server here, we used to have the client ID and client secret directly here in the pom. In this case we are going to be using the server, so we don't have to add the credentials directly to the pom — we're going to add them in the `settings.xml` in the Maven settings. Next we have the application name. In my case I am just going to be extracting the artifact ID from this project, so if I scroll up it's `project` > `artifactId`. So this is the application name that I'll be using. Then we have the release channel — in my case I'm selecting "none" because I want this to be deployed to Mule runtime 4.4.0. If you want this to be on 4.6.0 or up, then you can select here instead of "none" you can put "LTS" or "Edge", whichever one you prefer. Finally I have just one replica with 0.1 vCore, and this is going to generate the default public URL for me. That is all the settings that I have. If you have any questions, please go to the docs of MuleSoft so you can see all of the different options that you can set up here. ### Modifying the pom.xml — repositories Now let's scroll down a little bit more until you reach here the repository. This has to be version 3 of Exchange, so make sure you change this to version 3 and version 3, and that is all that you have to do. If you already have Anypoint Exchange version 3 here, then you can just leave it like that. Now in this same space, inside `repositories`, let's add a new repository. Again, go to the description of the video for the article and you'll be able to see this. This ID is "Repository" — this is the same ID that we had at the top in the configurations, make sure they match — whatever ID you want to put, make sure that it's the same one. And then the name is "Private Exchange repository", and this is the URL. This URL is extracting your group ID from the top, so if I go and scroll up we have `project` and then we have the group ID right here, so this matches your organization ID. You can also just copy and paste it and put it here instead of that, but I thought it was just easier to reference it. And finally, keep scrolling down, you'll pass the plugin repositories, the dependencies (maybe you have more dependencies here depending on your project), and then just after that, still inside `project`, make sure to add the distribution management with a repository with the same ID as before — in this case "Repository" — and then the URL again will have your organization ID. In my case I'm again just referencing the group ID that I have at the top. All right, that is all for this file. Make sure you save this so you have all of the latest changes. ### Creating the settings.xml And then let's create the `settings.xml`. What we've done in the previous videos or articles is that in the root folder we create a new folder called `.maven`, and inside that we create a new file called `settings.xml`, and this is the one that we're going to be referencing from our GitHub Actions. So once again, copy and paste the file that we have in the article, and you'll end up with something like this. We have the server ID, which matches the server IDs that we've had three times referenced in the pom.xml, and then the username has to be exactly this, which is `~~~Client~~~` so MuleSoft knows that you're using client ID and client secret from a Connected App. And finally the password is going to be this. So if you want to actually paste your credentials here, it would look something like this, but I really encourage you not to put your credentials in the repo, because then anyone is going to have access to them. So in our case we are just setting up the property reference of client ID and client secret, and then we're going to be passing this down from our GitHub Actions. All right, so that is all for this. Make sure you save this. If you have MUnits or something else, just make sure that you have your Nexus credentials here as well, with a different repository ID and so on. In this case, because it's just a basic case, you can just set this up like this. ### Creating the GitHub Actions workflow Now after that, let's go ahead and create the actual GitHub Actions workflow. So you have to create a new folder called `.github`, and then inside that create a new folder called `workflows`, and then inside it let's create a new file that's called `build.yaml`. And here again, we're going to copy and paste what we have in the article, and this is what we pasted. So this workflow is named "publish to Exchange and deploy to CloudHub 2.0". This is going to run every time that there's a push on the main branch, and there's only just one job. If you've seen some of the previous videos/articles, I normally have two or three jobs, but in this case I thought I'd keep it simple — it's just one job that's going to build, so check out this repo, cache dependencies, set up JDK (these are the same steps that we have in the past), and these two are the new steps. So the first one is to publish everything to Exchange, and the second one is to actually deploy the Mule application to CloudHub 2.0. And you'll notice that there's something different between these two: everything is exactly the same except for this flag right here that says `-DmuleDeploy` — this means that this is actually going to deploy and not just publish to Exchange. So those are the two different things. As you can see here, in this case I am just skipping the MUnit tests because I have nothing about that, and I also would have to set up my Nexus credentials, which I am not doing right now. And as you can see from here, we are setting up the client ID and client secret that will be read from this `settings.xml` file, and these are coming from the secrets that we set up earlier — the Connected App client ID and client secret. We can also check them out again if we use the GitHub Actions extension, so we can just make this bigger, and as you can see these match — these are the repository secrets. We can refresh; we have the Connected App client ID and the Connected App client secret. You cannot see these anymore, you can only modify them, so this is very secure. ### Recap of all the changes All right, so that's pretty much it. You can save this file and now you have all of the different changes. So let's just review all of the changes that we did. In the pom.xml we have a new group ID that matches the organization ID from your Anypoint Platform account, we modified the version to not have the snapshot, we modified the Mule Maven plugin to be 4.1.1 at least, we added more configuration to have the CloudHub 2.0 deployment with all of the necessary information like environment, the server, the target, and so on. After that we changed the Anypoint Exchange to version 3, we added the new repository with the same ID as the server, and the distribution management with the same ID. Then in the `build.yaml` we created the whole workflow from scratch, and in the `settings.xml` we added the server to connect the client ID and client secret to whatever we need to authenticate to Anypoint Platform. That was a lot of "twos". ### Running the pipeline All right, so let's go ahead and add all of these changes, commit them — "testing my first workflow run" — so let's commit this and sync changes. This is going to push my branch to the main branch. If we go back to the GitHub repo and we click here on Actions, you'll be able to see that this workflow is already running. You can click on "build" and you'll be able to see what's happening. The first thing is that it's trying to publish everything to Exchange, and after a while you'll be able to see "publication status completed", so that means that this was deployed to Exchange correctly. And now if we scroll down, we'll be able to see "deploy to CloudHub 2.0", so this will start actually deploying everything to Runtime Manager, and if we scroll down we'll be able to see here "deploying artifacts", "starting deployment", "deploying", and "checking if the application has started". So if we go here to Runtime Manager we're able to see the application is being deployed correctly. If you open it you'll be able to see here the target name, which is `Cloudhub-US-East-2`, the version which is one, the application file, and the runtime version which is 4.4.0 — this is the release channel, so you can also change this to Edge or LTS (long-term support). Another thing that I want to mention is that if you have a change that you want to send to the application, you always have to change the version in the pom.xml, otherwise it's going to tell you that the version is already published in Exchange and it won't be able to deploy it. So you have to continue — if you're doing just minor adjustments you can do `1.0.1`, `1.0.2`, `1.0.3` and so on, or you can also change the version as `1.1.0`, `1.2.0` and so on, or you can also change it as `2.0.0`, `3.0.0` and so on, depending on the changes that you're doing for each new version. ### Wrap-up And that is all. Congratulations, you have deployed your Mule application to CloudHub 2.0 correctly using GitHub Actions. So that is all for this video. Let me know if you want me to do anything else. Remember that if you want to change something in MUnits, if you want to do secure properties or something similar, you can just go ahead and watch the previous videos. Let me know if you're having issues with the CloudHub 2.0 deployment — I'll be happy to create more articles and videos for you all if I'm missing something that I didn't explain before. So thank you so much for watching. Please subscribe to all of my channels and everything, and I'll see you in the next video. Bye! --- ## Data Cloud/MuleSoft Integration Part 1: Connected App, Ingestion API, & Data Stream (Salesforce) Watch: https://www.youtube.com/watch?v=WVkf2ni-S6s | Page: https://prostdev.com/video/datacloud-mulesoft-connected-app-ingestion-api-data-stream | Series: Data Cloud + MuleSoft Integration ### Intro Hello hello everyone, welcome to the Data Cloud and MuleSoft integration series. My name is Alex Martinez, and today we are going to see how to do all of the prerequisites — or the settings — that you'll need to have ready for Salesforce Data Cloud in order for this integration to work. If you're new to MuleSoft, or you don't know MuleSoft at all, that is completely fine. This series is intended for Data Cloud users, and I am going to guide you through everything that you'll need in order to have this integration ready and running. ### Creating the Connected App All right, so the first thing we're going to do is to go to `login.salesforce.com`, write down your username and your password that you use to log in to Salesforce. So the first thing we're going to do is to go to the Setup right here on the top right corner, click on that, then right here in this screen go to the Quick Find and search for "App Manager", click on that so you can open it. Once it's open, click on "New Connected App" right here in the corner, and we're going to be filling out the following details. First, for the Connected App name, we're going to select "MuleSoft Integration Connected App" — this can be any name that you want, just make sure that you know what the name is. The API name is going to appear there immediately. Then select your contact email — just make sure it's an actual email, because you'll receive some link there. After that, go here to where it says "Enable OAuth Settings" and check this. For the Callback URL we're just going to add `login.salesforce.com/services/oauth2/callback` — you can also find this link here if you hover over the Callback URL. In this case we will not be using this, so we're just setting it up because it's a required field. Next, for the Selected OAuth Scopes, we're going to be selecting the CDP scopes. So first of all we have "Access all Data Cloud API resources" — click on Add. Next we have "Manage Data Cloud calculated insights data", "Manage Data Cloud identity resolution", "Manage Data Cloud Ingestion API data" (I for sure know that we're going to be needing that one), "Manage Data Cloud profile data", "Manage user data via web browsers", "Perform ANSI SQL queries on Data Cloud data" (for sure we're going to need that one), and "Perform segmentation on Data Cloud data". As a disclaimer, I am not 100% sure if we need all of these scopes because I am not a Data Cloud expert, but I am pretty sure that we might need — at least we need the Ingestion API one and the query one. So you can figure out if these are all really needed or not; this is just for an example. And then finally, if you scroll down here, "Enable Client Credentials Flow" — make sure you have that. So "anyone with a consumer key and consumer secret can access your org on behalf of the selected user" — yes, press OK, we are aware of that. So when you run this, the credentials thing, you just have to make sure that no one else has your credentials. That is all for this part. So just scroll down and click on "Save". Changes can take up to 10 minutes; click on "Continue". ### Generating the consumer key and secret Once you're in the details page, scroll down where it says "Enable OAuth Settings" and click on "Manage Consumer Details". It will send a verification code to your email, and then you have to input it here and click on "Verify". Now here at the top of the screen you'll see some consumer details — I am not showing them on purpose because I don't want you to see them. But here where you have the staged consumer details, you can click on "Generate", and this will generate a new consumer secret and consumer key. Once you have them set up right here, please copy those two, keep them somewhere safe because you're going to use them, and then click on "Apply". Once you click on Apply it will take them from here and put them up on the screen where I am not showing it. So if at any point you feel that your credentials have been compromised — you need to change them for whatever reason — you can just come here and click on "Generate" and regenerate some new ones, and click on "Apply" so they go to the top of the screen. So that is all for the Connected App. ### OAuth and OpenID Connect settings Now let's move on to the OAuth settings. Once more, inside the Setup part, in the Quick Find let's look for "OAuth and OpenID Connect Settings", so click on that. The only thing you have to check here is that you have the "Allow OAuth Username-Password Flows" set to ON. If you don't have it, please set it to ON; if you do, then perfect. ### Configuring the Ingestion API Now let's move on to the Ingestion API settings. So if you go here and look for "Ingestion API", you'll be able to see here "Data Cloud Configuration", select "Ingestion API" and click on "New" right here to the right side of the screen. In the connector name we're going to be using "MuleSoft Ingestion API". I already have this set up, so once you have this you'll be able to click on "Save" and move on to the next screen. Now once you're here, you have to take a note of the source API name that you see here — in my case it's `MuleSoft_core_Ingestion_API` — because we're going to be using this later in our MuleSoft application. Now I already uploaded my schema, so that's why you can see objects here, but if you don't have any schema, please go to the description of the video where I have the article, and you'll be able to see a YAML schema that you can use. You can also go to the GitHub repo. Click on "Update Schema" or "Upload Schema" and put here your YAML schema — you can use the one that I put in the example or you can use a new one. Once you have that, you'll be able to see a schema similar to this. If you use the same example that I did, then you'll have the object name "exercises" and the object name "runner profiles". Take a note of these two object names because you'll also be using them in the Mule application. ### Creating the Data Stream So that is all for the settings. Now let's go back to the Data Cloud home page. From the homepage, make sure you select here on the apps icon, search for "Data Cloud", and click on the Data Cloud with the bunny — you should have just one, but in case you have more, this is the one that we'll be using. Next, select the "Data Streams" tab. If you cannot see it for whatever reason, you can click on "More" and you'd be able to see it here. I already have this set up, but you will not, so you'll have to click on "New", then look for "Ingestion API" and click on "Next". Select here your MuleSoft Ingestion API. In my case, because I already have all of the objects selected, I don't have any available options here, but if this is the first time that you do this, you'll be able to see the two objects, "exercises" and "runner profiles", and you'll have to select them both. Then inside that configuration, for both objects — for the exercises and the runner profiles — you'll select the category "Profile", and in the primary key `MA_ID`. Click on "Next", select the default data space, and click on "Deploy". Again, I am not showing these steps because I already have it set up, but once you do those steps you'll be able to see what I'm seeing right now. So for example, if I open my MuleSoft Ingestion API runner profiles, I'll be able to see my object API name (which you'll be using in your query if you're doing a SELECT), and here are the fields of all of the object's fields that you've uploaded in your YAML schema. As you can see here, I have the `MA_ID` key as a primary key, so this is the one that I'll be using to delete objects or records. ### Recap of the credentials you need And we're done. That is all the setup that we have to do in Salesforce Data Cloud before getting started with the MuleSoft integration. Just to recap, you should have a text file similar to what I have right now, and we're going to be setting up the following fields: 1. The **Salesforce username** — this value will be whatever you use to log into Salesforce.com. 2. The **Salesforce password** — this is the same password that you use to log into Salesforce.com. 3. The **CDP consumer key** — this is the consumer key that you generated after you did the whole thing with the email (which I wasn't able to show you because I didn't want you to see my consumer key and secret). 4. The **CDP consumer secret** — those two are the ones that you generated from the Connected App. 5. The **source API name** that you generated from the Ingestion API — if you're using the same example that I did, then your name would be the MuleSoft integration API. 6. The **object name**, which is the object name that came with the YAML schema — if you're using the example that I did, this should either be "exercises" or "runner profiles". And then, just as a note, I wanted you to remember what the object ID (aka the primary key) is that you set up for your YAML schema or your objects in Data Cloud — in my case it was `MA_ID`. ### Wrap-up All right, that is all for this video. Please subscribe so you can receive notifications as soon as we publish new content. I'll be continuing with this Data Cloud series, and if you have any more suggestions on what else I should be creating content about, please feel free to comment or send me a message directly, whatever you prefer. Remember to follow us on all of our socials, to subscribe at ProstDev.com, and to subscribe on youtube.com/prostdev please. And I'll see you in the next Data Cloud integration video. I hope this was helpful, and I'll see you then. Bye! --- ## Data Cloud/MuleSoft Integration Part 2: Deploy your own Mule app on Anypoint Platform (CloudHub) Watch: https://www.youtube.com/watch?v=A48011haXRw | Page: https://prostdev.com/video/datacloud-mulesoft-deploy-mule-app-cloudhub | Series: Data Cloud + MuleSoft Integration ### Intro Hi everyone, Alex Martinez here, welcome to the second video of the Data Cloud and MuleSoft integration series. In this second video we're going to go through the steps for you to deploy the Mule application into Anypoint Platform, and then we'll get to call this application in the next video. So for now, it's okay if you come from a Salesforce or a Data Cloud background — you do not need to know MuleSoft. We will not be developing this Mule application here today; you'll just get to see how to download the application, put it in Anypoint Platform, and deploy it so you can get the public URL. You do not need to do anything else about this. So it's okay if you don't know MuleSoft, I'll guide you through all of the necessary steps. ### Prerequisites Okay, so first of all, you'll need to have an Anypoint Platform account. So if you go to `anypoint.mulesoft.com`, you'll be able to sign up and create a 30-day trial account. It's okay if your trial account expires, because you can create a new one just using a different username — you can even use the same email, just change your username, and you'll be able to create as many free trial accounts as you want. Next, if you go to the description of the video, there will be a link to the article of this video, and there you'll be able to download this JAR, or to get to the releases page. If you want to find it, you can just go to `github.com/alexandramartinez/datacloud-mulesoft-integration/releases`, and here you'll be able to find the Data Cloud integration Mule JAR. When this video was created and when the article was written, this was on version 1.0, but if there are any new versions from this moment on, I would advise you to use those. I'll make sure to document here in the release notes everything that you'll need, or everything that will change from the other articles or videos in the new version. Anyway, for now, just come here in the assets and download this JAR file because we'll need it later. Also make sure you go through the whole video — the last video, or the last article, Part 1 — because we'll need four credentials that we created from the last video. ### Deploying the app in Runtime Manager Inside your Anypoint Platform account, let's go ahead and go to Runtime Manager right here. If this is the first time that you open Runtime Manager, it will ask you to select an environment, either Design or Sandbox — please select Sandbox. Then once you're here, click on "Deploy application" right here in the middle. Add your application name — in my case it's going to be "Data Cloud integration". Then in the deployment target, just make sure that you have selected CloudHub 2.0. You can also select CloudHub to select CloudHub 1, but that is going to take some more things, so it's okay, just select CloudHub 2.0 (there are more details in the article if you want to change to the other one). Then on the application file, go here and choose file and click on "Upload file". Here you'll select the JAR file that you downloaded from the releases section on my GitHub repo. Then under the Runtime tab, make sure you have selected the runtime version — in my case I'm going to select "none" so I can select the runtime version 4.4.0, but it's okay if you want to experiment with others (just, at the time that this video was created, I was just using 4.4.0). You can leave all of the rest of the defaults. ### Setting the properties And then click on "Properties". From the Properties tab, select the Text view, and you can copy and paste this snippet from the article in the description of the video. You'll have pretty much something like this, where you have the Salesforce username, Salesforce password, CDP consumer key, CDP consumer secret. These four credentials are what we got from the previous video or article: the Salesforce username is the username that you use to log into Salesforce, the Salesforce password is the password that you use to log into Salesforce, and the CDP consumer key and secret are the credentials that you generated when you created the Connected App in Salesforce. Now here I added just some random examples so you can see how it should look. After you add those, go here to the Table view and click on "Protect" next to all of the values. So the first one is going to ask you if you're sure — just click on "Protect value" — and then you can click on the rest so you can protect all of them. Now after all of that is done, you can click "Deploy application". It will start uploading the application file, as you can see from here, and once this whole bar has finished, then the application will start deploying. ### Waiting for the deployment Once the JAR file has been uploaded, you'll now see the screen right here, which is the logs. It's okay if you see here a red circle in the Data Cloud integration part, because it's just starting to deploy. If you go to the Settings tab here on the left, you'll be able to see here that this is applying the new configuration, so that's okay, don't worry about it, just give it a few more minutes. It can take between 2 and 10 minutes, I would say — it's really random, I don't know why. Sometimes it can take like 2 minutes, which is great, but sometimes it has happened that it can take 8 minutes, so just give it a few minutes. You can also view the status here, but it's not going to tell you a lot of information, or you can also check out the logs so you can see that it's actually running things. All right, so this time it actually took 2 minutes, so that is good, it was not that long, and now we can see the green circle here, so that means that this was deployed properly. Also here in the Settings tab you'll see the application status is "running". ### Getting (and protecting) the public URL Now what we want to get from here is this public endpoint, so make sure you copy this value and paste it somewhere safe because we'll be using this to call our application in the next video. But very important: do not share this URL publicly like I am doing right now. By the time you see this video this URL will not be available for you, so even if you try to call it, it will not work. But this might not be the case for you, so do not share this URL, because anyone that has this URL will be able to access all of your Data Cloud information — in the sense that you added your credentials into the configuration of this app, so with this URL you can access that directly. If you want to create a new URL because it was compromised, you can simply delete this application by going here, you can just click on "Delete", and then check this and click on "Delete application". Then you'll have to go through all of the steps that we just did and create a new application with a new name that no one else knows. In this way you'll have a new URL. Another thing that you can do is that you can stop your application when you're not using it, and then start it again when you want to use it. So for example, now if I would try to get to this URL, I would not be able to use it because this is right now not running, as you can see here. So if I click on "Start", it will start doing everything again. It already has the property saved so I don't have to add any of that information again — I just have to start this application, it will do its thing to deploy, and then it will appear with a green success circle again. ### Wrap-up And that's all. You deployed your first Mule application — if this was your first or not. As I said, you did not have to develop this application, I already did that for you, you just had to download the JAR and put it here so you can deploy the application directly into CloudHub. And that is all for this video. We now have our Salesforce credentials that we already put here — we'll be using more of the things that we got from the previous video for Salesforce in the next video, so make sure you go through that one as well. We now have our public endpoint, or our public URL, that we'll be using to call our application in the next video. And keep posted for the next videos, because I'll be adding so much more stuff to this thing, like for example you can add basic authentication so you don't have to worry about someone having your public endpoint, because you'll have actual credentials to access this endpoint, which is awesome — but that will come later in time. The best part is that all of these are no-code tools, because I already did the work for you, so you don't have to create the Mule application from scratch, you just have to download the JARs. So remember to follow us on all of our socials at ProstDev.com, subscribe please, and if you go to youtube.com/prostdev, please subscribe there as well. Make sure to put the notifications so you receive everything that we post. You can also send me messages or comment in the video if you have any suggestions, feedback, or just general comments for me. All right, that is all for this video. I'll see you on the next video where we actually call this Mule application. Bye! --- ## Data Cloud/MuleSoft Integration Part 3: Call your integration with Postman Watch: https://www.youtube.com/watch?v=CNAnWwUxycg | Page: https://prostdev.com/video/datacloud-mulesoft-call-integration-postman | Series: Data Cloud + MuleSoft Integration ### Intro Hi everyone, welcome to Part 3 of our Data Cloud and MuleSoft integration series. My name is Alex Martinez, as you know by this point I hope. In this third part we're going to learn how to use our integration — how to actually call it, how to insert, query, and delete stuff from Data Cloud. We'll use Postman for this video, but you can use any other REST client like Thunder Client, cURL, or Advanced REST Client. So let's get to it. ### Prerequisites Just a few prerequisites. First of all, you're going to need the Postman collection that's in the GitHub repo, so head to the description of the video or to the article and you'll be able to find the link to download the Postman collection. Second of all, make sure that you have downloaded Postman on your local computer, or you can also use the web version of Postman just by creating an account. And the third thing is to make sure that you have the Mule application URL that we got from the previous video. If you don't have it yet, just go back and make sure you get it from CloudHub — you'll be able to see the endpoint or the URL. So make sure you have that. And that is all for the prerequisites. ### Importing the collection and setting up the environment So once you have that, go into your Postman, make sure you select here the Collections tab and click on "Import". Now you can put here the collection JSON that we got from the prerequisites, and just like that you'll have imported the collection in Postman. Now let's go ahead and create the environment that we're going to need. Click here on the Environments tab on the left. You could create a global variable like this — just put your value here — but just to follow best practices, let's create a new environment. Go here and click on "Create environment". Let's name this "CloudHub" because we want to remember what this is. The variable name is going to be `host`, and you can paste it in the initial value, but the real one that we'll need is a current value, so make sure you paste your URL right there in current value. And that's it. Now you can save this by clicking here on the Save button, or just selecting Command-S or whatever is in your own operating system. Now once this is saved, you have to make sure to select the environment from here like this, because if you don't select it you won't be able to see it in your request. So you see here it is pointing to the right URL, but if I move it back to "No environment", then we go back to "unresolved variable", so make sure you select the new environment here. Also notice that all of the paths here already have a slash, so make sure that your value in `host` doesn't end with a slash and just ends with `cloudhub.io`, otherwise you might have some issues with that. ### The schema request All right, so that is all. Let me minimize this a little bit so we can see the schema one. So the first request that we have is the schema. Here in the Params tab we have the OpenAPI version — right now it's set to 3.0.3, but you can modify this if you want to have a different version. What this does is, if you go here to Body, we'll be able to see that we have this JSON payload — we have a customer object, and then we have an address object, and inside the address we have a geo object. So we have three levels of objects. And if you're familiar with Data Cloud, you'll know that this is not like... if you use the YAML schema, you cannot do this, you have to have just one object level. So with this request we'll be able to send this JSON schema and it will be transformed into the YAML schema that we'll need for the Ingestion API. So if I were to call this right now, I would correctly receive the YAML schema that I would use. You can just not use this first part and just use from here on, and this would be your YAML schema to use in Data Cloud in the Ingestion API. So if I go back here to Params and I were to change this — let's just make a random number, 25 — click on "Send", then we would correctly receive here `d25`. So just make sure you modify that to whatever it is that you need. ### The query request All right, now the second request is the query one. So if I open that, we still have the `host` and then `/api/query`. So here we don't have any parameters, but if I go to the Body part, here is where we'll be sending the query, the SOQL query for your Data Cloud instance. So for example, if I have this, I can just click on "Send" and this will query the results. In my case right now I don't have anything, so this is why I have an empty array. ### The insert request But we can do some insertions. So if you go to the insert request now, we have some parameters here. So if you check this out, we have the source API name, which is "MuleSoft Ingestion API" in my case — this might be different for you depending on what you created from Part 1 of this series. My object name in this case is going to be "runner profiles", again depending on what you created on Part 1 of the series. Then if we go here to Body, this is the array of objects of what I'll be inputting into Data Cloud. So these are all random values, we have five objects in total. So let's send that, and we receive a "200 OK" response. So that means that this is working correctly. So now if we go back to the query, it's going to take a little bit before we actually see the results here — it might take between 2 and 5 minutes perhaps. This is just because we're waiting for Data Cloud. So let's come back whenever this starts returning some results. And there we have it — all of our items were correctly inserted here. So yay, the insert worked. Now let's get to see... let me just run this again to verify — yes, everything is here, all of the five items are here. ### The delete request So now let's delete them. If you go here to the delete part: in the request, first we have the same parameters as the other requests, so if we open this a little bit we'll see that we have the same source API and the object name, and we have the "MuleSoft Ingestion API" and "runner profiles", same as the insert. Now in the Body this is going to be different. We're just going to send pretty much what is your primary key for each of the objects — again this is what you set up in Part 1, so if you're not sure you can go back and check that out. In my case these are my five IDs that I created in the insert. You can also go here to the query — my primary key is `MA_ID`. So for example this is number three. So from the delete, let's say that I want to delete all of them except number five. So let me just remove number five from here, and let's run this. So once again we receive a "200 OK", so this is working properly, and now we just have to wait one more time and continue sending the query until we start seeing a change here. And there we have it — now we only have one object here in the results, and this is the ID number five. Let me run this again, and yes, this is what we have. So our delete worked perfectly. ### Wrap-up And that is all. That is how you use this integration for your Data Cloud stuff. Someone had mentioned to me that the delete was kind of hard, so now you can just come and use this integration whenever you need it. Again, the Anypoint Platform free trial account lasts 30 days, but you can keep creating new accounts anytime that you need. You have all of the articles, you can just follow all of the steps in case you need to redo the whole thing, or if you have your same Data Cloud settings you just have to do it once and then you just have to download the JAR from my GitHub repo and upload it to CloudHub and set up all of the properties, and that's all. I'll continue improving this application though. For example, I mentioned in the last video and the article that you have to make sure that no one can access the URL that you have, because it contains all of your Salesforce credentials, so that would not be good. So there is another way for you to make sure that that URL is secure, and that is adding basic authentication — for example a username and password that only you have access to. In that way you can just add the security policy from MuleSoft directly and just set up your username and password, so you're the only one that can access the URL. Even if someone else has the same URL as you, they won't be able to access it because they don't have your username and password. So keep posted for the next videos and articles, make sure to subscribe at ProstDev.com, and make sure you subscribe to the YouTube channel at /prostdev, because we'll continue doing so much more fun stuff, so you don't want to miss it. And also let me know if you want me to add a little bit more features to this — I'll be happy to take a look at that. Oh, and also another quick quick quick note: because we're using the insert and delete here from the streaming API in Data Cloud, you are bounded to a maximum of 200 records per time. But there is a workaround — you can either modify the code to use the bulk API instead of the streaming API, or keep posted and I'll let you know how to use the aggregator module to make sure that this doesn't have a limit. All right, that is all for this video. I hope you liked it, and I'll see you on the next video. Bye! --- ## Data Cloud/MuleSoft Integration Part 4: Secure your API with basic authentication in API Manager Watch: https://www.youtube.com/watch?v=r_AM3P8Y-_Q | Page: https://prostdev.com/video/datacloud-mulesoft-secure-api-basic-auth-api-manager | Series: Data Cloud + MuleSoft Integration ### Intro Hi everyone, welcome to Part 4 of the MuleSoft and Data Cloud integration series. My name is Alex Martinez, and today we are going to learn how to secure your API with basic authentication in API Manager. So in Part 2 of this series we learned how to retrieve the URL from CloudHub, but I did warn you a lot of times that you should not be sharing that URL with anyone else because they would have access to all of your credentials. So the solution to add security to that URL would be to add some security policies to the API. Now this is perfect to do in MuleSoft because you do not need to know how to code these security policies — you can simply apply them using Anypoint Platform, just in clicks, not code. So let's learn how to do that. ### Prerequisites Now, some prerequisites before we get started. Of course you'll need to have an Anypoint Platform account. I am expecting that you already went through Parts 1, 2, and 3 of the series because you should have everything prepared for this. Then you'll also have to download version 2.0.0 of the JAR. So if you go to `github.com/alexandramartinez/datacloud-mulesoft-integration/releases` and you get to the release 2.0.0, you can scroll down here and you'll see the assets part, and here you have the 2.0.0 application JAR — so download that because we'll use it in a moment. Next, you'll need your Anypoint Platform environment credentials. So in your Anypoint Platform, go to Access Management right here, go to Business Groups, select your own business group, then click on Environments here and select your environment — in my case this is going to be Sandbox — and here you can extract the client ID and the client secret. Finally, if you followed Part 3 or the previous part, you should already have your Postman collection ready to run, and please double-check that this works before getting started. ### Creating the API in API Manager All right, so let's get started. First of all, go into your Anypoint Platform and then head to API Manager. In API Manager, click on "Add API", "Add new API". Here in the runtime section, select "Mule Gateway", leave all of the defaults and click on "Next". Now in the API part, select "Create new API". In our case we're going to name this "Data Cloud API" and select an HTTP API, click on "Next". You can leave this Downstream empty, click on "Next", and also leave the Upstream empty and click on "Next". Now just review that everything looks good and click on "Save". Once you save it, you'll be able to get the API instance ID from this part, so copy this number because we'll use it later. ### Upgrading the deployed Mule app Now let's upgrade the deployed Mule application. So if we go to Runtime Manager, open our app, and in the Settings tab you'll be able to see that you have your application file, so click here on "Choose file", upload file, and once you select the new JAR — the 2.0 — you'll be able to see it here. You'll also see the "Apply changes" button that just appeared. But before we continue, let's go into the Properties tab and scroll down. We already had our Salesforce and CDP credentials, so let's go into the Text view, and if you go into the description of the video you'll see the link to the article where you'll be able to copy and paste this text right here, because this contains all of the properties that you'll need to upload here. So let me change into the Table view to see them one by one. We have the `anypoint.platform.gatekeeper`, this will be set to "flexible". We have the `api.id`, which is the API instance ID that we just copied — let me put that there. Then we have the Anypoint Platform client ID and Anypoint Platform client secret, which are the two credentials that we got from the beginning. Your client ID should look something like this, and I'll put the client secret in a moment. After you add both of the client ID and client secret, remember to click on "Protect" so that no one can actually get to this value. Perfect, now these are all of the properties that you should have. You can pause this video if you need to take a look at them. After you're done, you can click on "Apply changes". Now once this has been deployed, you'll see this as "running" and there will be no processes here. If for some reason it's taking more than expected, like more than 5 minutes and nothing is happening, just make sure and double-check that all of the properties you set up are correct. ### Confirming the API is active Now after this, let's go back to API Manager. So if we open the menu and select API Manager, we'll be able to see our API, select that, and now you'll see the API status here set as "Active". If for some reason you're not seeing it, you can also refresh the page and make sure that this appears as "Active". If it doesn't, please make sure that you're using the correct API instance ID and your environment credentials. And if you come back to Postman and run the query again, you'll receive a "200 OK" because everything is good. ### Adding the basic authentication policy Now let's go back to API Manager and now we can start adding the policies. So select the Policies tab, click on "Add policy", search for the "Basic authentication - Simple" one, click on "Next", and then just add whatever username and password you want to use for your own credentials. In my case, as an example, I am going to be setting this up to "foo" and "bar" — to not use this, please create your own credentials. And once you have added them, click on "Apply". Once this policy has been created, you'll see this green message, and then you can go back to Postman and send this again. It will take a few seconds, but eventually you'll be able to see the error being shown. And here we have it — we have a "401 Unauthorized", and we have the error saying that you're supposed to be sending basic authentication but you're not sending it. ### Adding the credentials in Postman So to fix that, we'll add the credentials to our environment and then we'll reference them from our collection so it applies to all of the requests. So if we go to the Environments and select the CloudHub environment we had previously created, let's add a new variable called `username`, and in the current value we'll write "foo". Now let's add the other variable, which is `password`, and then in the current value we'll set this up as "bar". If you don't want to show this, you can also change this type as "secret" and it will appear like that, so if you're sharing your screen or something, people won't see your credentials. In my case I'm just going to let it be as default so you can see my credentials for now. And then click on "Save", or select Ctrl-S or Command-S to save the environment. Now let's go back to the Collections tab and open our collection by clicking it. Here in Authorization, let's select the type as "Basic Auth", and then in the username we can create the variable — so if we use the double curly brackets we'll be able to select the `username`, and the same thing for the password, let's add the double curly brackets and select `password`. So now if you hover over it you'll be able to see that it says "foo" and this one says "bar". So make sure to save this, and now you can go back to your query and send it. And now we receive a "200 OK" because now we are sending our basic authentication with Postman. ### Recap And that's all. Congratulations, you applied a basic authentication security policy to your API without having to code anything into it. Just as a summary: you had to create an API in API Manager like this one, and then you had to copy the API instance ID in order to send it into the new JAR file that we have created — so here we had to upgrade this to the version 2.0, which is the change to the code that I created in order for you to activate autodiscovery. Then we also added the properties — the API ID, the gatekeeper, the Anypoint Platform client ID and client secret — in order to be able to apply those policies. And finally we added the two variables to the environment, and then we also modified the Postman collection to contain the username and the password. ### Wrap-up And that's all for this video. Remember to follow us on all of our socials at ProstDev.com, subscribe so you can receive notifications as soon as we post any new content, also on our YouTube channel /prostdev so you receive notifications as soon as new videos are uploaded, and feel free to contact me for any questions or suggestions that you may have for me. I am happy to keep creating content for you all. That's all then, I'll see you in the next video. Bye! --- ## How to manually authenticate to Salesforce Data Cloud through a MuleSoft integration Watch: https://www.youtube.com/watch?v=iGlPOy_EMlE | Page: https://prostdev.com/video/manually-authenticate-salesforce-data-cloud-mulesoft | Series: Data Cloud + MuleSoft Integration ### Intro Hi everyone, my name is Alex Martinez, and today I'm going to show you how to authenticate to Data Cloud if you have to do it manually. There are a lot of connectors that you can use where you don't have to manually do all the authentication — specifically for Data Cloud, there are some connectors for Data Cloud in MuleSoft that you can use so you don't have to manually do the authentication yourself. But I needed to do an operation that was not available in the Data Cloud connector, so I had to manually do the authentication in order to be able to use that. It took me a while to figure out, because I did not know that you first have to do the authentication to Salesforce and then you can do the authentication to Data Cloud. As you can see, I'm not a Salesforce expert, so I was really struggling with this one because I wanted to do it from MuleSoft. ### Creating the project So first of all, just click on "Develop integration". Yes, I am in Anypoint Code Builder — I just prefer it over Studio, but you can go ahead and do it in Studio as well. My project name is going to be "Data Cloud auth", and I am selecting the latest Mule runtime and the latest Java version. Click on "Create project". All right, so once it's created, I'm just going to go ahead and build a new flow. Now inside this flow, I'm just going to show you really quickly: we have the Salesforce connectors — these can be either Salesforce CDP or Salesforce Data Cloud — and this is where you would use directly all of the operations that you need. But in my case I wanted to use the "Get all jobs" operation that is not currently available here, so I had to create it from scratch. ### Building the first request (authenticate to Salesforce) All right, so the first thing is that you need to create a request that will be sent into Salesforce. So for this I am just going to create a new Transform Message. I kind of already had this preset, but you can see what you need for this Transform Message. In this case, because ACB doesn't let you use the UI yet for the Transform Message, that is okay, so you can use it directly here as a Transform Message and you can put the code here, or you can also put it in an external file, but for now let's just leave it simple. Zooming in a little bit so you don't get lost. So first of all this output will have to be `application/x-www-form-urlencoded`, so this is going to be in that format. And then down here we can get started with our body. So this is going to be an object, and inside the object we're going to have first the client ID. Also note, there are different ways to authenticate to the Salesforce API, or to retrieve the Salesforce authentication. In my case I am going to select the password type, but you are free to use whichever type that you want. So here's a client ID, I'm going to leave it empty for now, then we have the client secret, I'm also going to leave it empty for now, then we have the username and the password. Finally the grant type is going to be `password`. So once again, you can go to the Salesforce documentation to make sure what is the request that you want to use. I tried using other ones but it didn't work for some reason for me — maybe I did something wrong — but this is the one that did work for me. So you'll have to do a Connected App so you can extract the client ID and client secret, and the Connected App has to be configured for the authentication. So I have a few videos and articles on how to do that configuration. And then just use your username and password that you used to log in to Salesforce. ### Setting the credentials as properties For this case, because I want to show you that it actually works, I am going to have to add some properties to this so you don't see my credentials. All right, and here we have it: we have the Connected App client ID, Connected App client secret, Salesforce username, and Salesforce password. So really quickly I'm going to set that up in the globals — so Properties, and we have here the Configuration Properties. So the file is going to be just `default.yaml` and that's it. You should have your global properties in another file, but just for the sake of this demo let's just do it like this. And then I'm going to create a new file here called `default.yaml`. In here we are going to set up our Connected App, and then we have client ID, secret, then we have Salesforce username and password. So now I just have to put my credentials here — and you can also do it in quotes just in case you're not sure if it's actually taking the correct value or not — and just put here your credentials, and I'll be right back. All right, my credentials have been set, and now this is going to take them from the properties and put them here and send this as the payload. ### Sending the first HTTP request So now we have to add the request, the HTTP request. First I'm going to go to the Advanced tab, and I want to save this as a variable, so this is going to be the Salesforce response variable, so I'm going to change the name to that as well. Now if I go back to General, this method is going to be a POST, and the URL is going to go to first the `login.salesforce.com` and after that to `/services/oauth2/token`. So remember this URL: if you're trying to log in to just the general Salesforce.com service, then that is the URL that you need to get the authentication token from Salesforce. Remember, first step is to get it from Salesforce. ### Building the second request (authenticate to Data Cloud) And after we receive the response, which is going to be saved in the SF response variable, then we need to create yet another request. So if we use another Transform Message, and now we need to do the second request to get the authentication for Data Cloud. So first of all, this format is going to be yet again the form URL encoded. Then we are going to do another object right here, and inside the object we are going to have the grant type once again, but now the value that we're going to send here is going to be hardcoded to this `urn:salesforce:grant-type:external:cdp`. So here CDP, this is what we are requesting, which is the Data Cloud access, or so I think. After that we have to set up the subject token, and this is going to come from the variable that we just created, which is the Salesforce response from the request, so if we do `vars.salesforceResponse` and then inside here we're going to have an access token in the response, so we simply have to do `.access_token`. We're going to retrieve that access token that we got from our Salesforce response, and this is going to be sent there. And finally the subject token type, and this is going to be hardcoded once again, so now this is `urn:ietf:params:oauth:token-type:access_token`. I don't really know what that means, but okay, that's how it works. I did tell you I'm not very good at Salesforce. And now we're going to send another HTTP request here, but now this is going to be interesting. But first let me go in Advanced and save this in a variable — so we named the other one Salesforce response, so let's call this Data Cloud response. I guess it's not really calling Data Cloud, it's still calling Salesforce, but I just want to make sure that I know the difference here. All right, so if I go to General, first the method is going to be POST, then the URL — we're going to make this a `++` (concatenation), because the first thing that we're going to get from here is the variable, so the Salesforce response that we created earlier, we're going to use that one, but instead of (for example, here we got the access token) instead of getting that, we are going to get the `instance_url`. And after that we are going to concatenate a string, and this string is going to be `/services/a360/token`. So this is where we are going to be calling our second request, but we needed to get this instance URL and this access token from the first request in order to do this one. And that is the whole authentication. After you receive this second response, you will once again retrieve the instance URL — but not from the second response, sorry, from the first one — and then you will be able to call the Data Cloud API to do whatever you wanted. ### Calling the Data Cloud API (get all jobs) In my case it was to get all of the jobs that were currently available in my organization. So how to do that: we do another request, and now if I take a look at this request and I go — this method is going to be a GET, but then the URL, let's do a formula right here, and what we're going to have here is first of all a string saying `https://` and then we're going to add whatever we got from the `dcResponse.instance_url`. So this is again from the second response — you got an object and one of those properties was the instance URL, so you're going to use that to be the host. Plus, and then you can finally add the string of whatever you want the path to be — in my case I needed to use `/api/v1/ingest/jobs`. So this is the URL that I was trying to get to. I just needed the Data Cloud response instance URL, and we also need to send a bearer token in the headers. But in this version of Anypoint Code Builder I cannot add the headers here, so I'm going to have to add them manually instead of using the UI, but that is okay because we're not afraid of using code. So pretty much you open this thing and then you close it, and then here inside you just have to look for HTTP headers, and here you have the CDATA. So inside the CDATA you can pretty much create an object, and inside the object we're going to add the authorization, and this is going to be `Bearer ` space (very important to leave the space right there) and then plus, and we're going to do `vars.dcResponse.access_token`. So from the Data Cloud response we are using the access token and the instance URL. And that is all. We should be able to call this — wait, I'm missing the HTTP, so we need an HTTP listener. Connectors, HTTP Listener, and we need to add the global property for the listener, so let's do that quickly: HTTP Listener config, this one, and we're calling `localhost:8081`. Perfect. So now here we just have to select that HTTP listener. Let's do test right here, and that is all. ### Running and debugging So that is all. Let me save this and run it. All right, this has been deployed, and let's send this and — ta-da! — we correctly get all the data from the response. I'm not going to show a lot of the data because I don't know if I can show it, but we can see that we are authenticating correctly. Now let's see if we can debug. So first of all, I'm going to set up a breakpoint in the Transform Message — here's a listener, here's the transform, I have a breakpoint there — then let's do the HTTP request, and then the second transform, the request, and the final request to the actual thing that we need. So I set up breakpoints everywhere. Now let's run this again to see if we can stop and watch what's happening. All right, this has been deployed, let's send the request, and now we stop here at the beginning. There's nothing in the Mule message, so we continue, and now we are at the request and we already have a Mule message here, and in our payload we correctly have all of the things that we have to set up. I'm not going to show all of them because of security issues, but we have the client ID and everything that we needed set up. So after we continue, we can now see in the variables that we have the Salesforce response, and we have an access token that I'm not going to show, and we have more stuff like the instance URL and so on. So now if I continue once more, now we get to the HTTP request, we have the variable SF response, the Mule message now contains our new stuff which is the grant type and so on, everything that we set up right here. So now if we continue and send that request, now we have two variables, the Salesforce response and the DC response, both of them have access tokens and more stuff that I'm not going to show you. And finally if we finish, now we send the last request and we receive correctly the data from Data Cloud. ### Wrap-up All right, that is all for this video. We learned how to manually do the authentication to Data Cloud using MuleSoft. Remember that you don't have to do this if the operation is already there in one of the MuleSoft connectors. So another thing that you could do would be to create your own connector, so you don't have to do this authentication manually — but I'm just waiting for the connector team to create that last operation that I'm missing from the Data Cloud connector, so this is just a temporary setback that I had to set up. But then I'm sure it's going to come out and I'm going to just update it to the new version of the connector. But for now, at least you'll learn how to actually do the authentication and everything in case you need to do your own API that is not in MuleSoft specifically — at least you can see all of the steps. All right, that is all for this video. I'll see you later in another Data Cloud and MuleSoft, or just MuleSoft, video. Bye! --- ## Data Cloud/MuleSoft Integration Part 5: Insert Data with the BULK operations Watch: https://www.youtube.com/watch?v=hbrxfIfP4fI | Page: https://prostdev.com/video/datacloud-mulesoft-insert-data-bulk-operations | Series: Data Cloud + MuleSoft Integration ### What's new in this update Hello hello everyone, my name is Alex Martinez, and today I'm going to show you a few of the updates I made to my Data Cloud and MuleSoft integration. By this point I fully expect you to have followed Parts 1 to 3 of the previous videos or articles — we're now at Part 5. I don't require you to follow Part 4 (that's just for security reasons, if you want to secure your API or your app), but Parts 1 to 3 you do need, because there's some specific configuration to set up for MuleSoft, Data Cloud, and Postman. Once you've done that, go to my **REST client request** and into Postman, where you'll be able to download the latest Postman collection. Here's how the new collection looks: we have the four previous operations — the insert and delete, which I just renamed to **streaming** — and the rest are for **bulk**. So you should now have these five operations. ### Updating the deployed app The second thing you should do is, also in my Data Cloud / MuleSoft integration repo, go to **Releases** — there's now a **release 2.1.0** that handles the bulk operations. Scroll down and you'll see the JAR; download that JAR and upload it into Runtime Manager. As I said before, you should already have your application running in Runtime Manager, so just go there, click **Choose file**, upload the new JAR, and click **Apply changes**. Now you're on the latest version of the code. Similarly, I have my API Manager set up with the basic authentication policies applied to my application — but you don't have to do this if you don't want to. ### The four new bulk operations So what changed? I have these five new operations — really four, because the bulk ones are: **get all jobs**, **upsert** (you can choose CSV or JSON), **get job info**, and **delete job**. Before I get into what's new, let me show you a little bit of the bulk process in Data Cloud. ### How the bulk process works If we take a look at this diagram, this is the flow of Data Cloud. First we have **create job**, and when we create a job the state is **open** (the blue circles are the operations you use in MuleSoft). After you create a job you have to **upload the job data** — this still remains in the open state, because you can keep uploading as much data as you want, and you have to close it manually. Once you're done uploading, there are two things you can do from the open state: - **Continue** the rest of the process by closing the job with the **upload complete** state, or - **Abort** everything — change the state to *aborted*, and then there's nothing more you can do with that job. If you choose to continue and close the job with the upload complete state, Data Cloud takes the job information and all your data and queues it with the state **in progress**. Once Data Cloud is done processing, it depends on whether it was successful: if it was, your job is updated to **job complete**; if it wasn't, it's updated to **failed**. Now, if you want to **delete** a job, you can only delete it in one of these four states: **upload complete**, **job complete**, **aborted**, or **failed**. As you can see from the arrows, those are the four states where you can remove or delete the job. ### Get all jobs Now that we have a bit more context on how the bulk works, first we have the **get all jobs** operation. This returns all of the jobs you have — or have access to — in the Cloud, whether they're open, closed, job complete, and so on; you'll see the full list. I'm not going to run this one, because the jobs have critical information inside that I don't want to show you. But you don't have to set up any query parameters or a body — you can just send it, once you've set up your Postman collection. ### Upsert (CSV or JSON) The second operation is the **upsert with a CSV**. In this request you do have to send some query parameters — the same ones we've been using for the insert operation: the **source API name** and the **object name**. Then in the body you send everything as CSV, and the Mule application takes it and sends it to Data Cloud. If you don't want to use CSV — say your data is in JSON — that's completely okay; you can use the **upsert JSON** operation. It's the same thing, just a different body: it sends a JSON payload, and inside the Mule application it transforms that JSON into CSV before sending it to Data Cloud. Now, remember from our diagram that we have to first create the job, then upload all the data, then close the job. This **upsert operation does those three steps for you**. At the end, if everything went well, you'll receive a response with an **upload complete** state — which, going back to the diagram, means everything went well, we continued, and we closed the job with upload complete so Data Cloud can start processing. ### Get job info In the response of that upsert you'll receive all the details of the job, including the **state** and the **ID**. If you take that job ID and put it into the **get job info** operation, you'll be able to retrieve all of the information for that job — including whether it's still upload complete, in progress, failed, or job complete. So after you close the job, you only need to wait for Data Cloud to process it: your job could be upload complete, in progress, or already processed and changed to either job complete or failed. ### Delete job Finally, another thing you can do with the job ID is **delete the job**. Going back to our diagram, remember you can only delete a job that's in one of the four valid states. If you're sure you can delete it, run this; if the state isn't one of those, you'll receive an error — and that's okay. If your job is in the **open** state — meaning you only created it, or maybe uploaded data but never closed it — that's also okay, because the Mule application is smart enough to know you're trying to delete the job. Once it's in the open state, it will **abort the job** first and then delete it for you. ### Wrapping up So those are the four new operations I've added to my application. If you have any questions, please don't hesitate to ask — I'm happy to help. And of course, if you want to look at the code, just go into my GitHub repo: the whole Mule application is inside the Data Cloud integration implementation, so you can check out how I generated all of this. That's all for this video. I hope it's helpful, and I'll see you in more Data Cloud and MuleSoft stuff later. Bye! --- ## DataWeave programming challenge #1: Add numbers separated by paragraphs and get the max number Watch: https://www.youtube.com/watch?v=JviC_4lSQEk | Page: https://prostdev.com/video/dataweave-challenge-1-add-numbers-paragraphs-max | Series: DataWeave Challenges ### Intro Hi everyone, my name is Alex Martinez, I am the founder of ProstDev, and today I bring to you the very first DataWeave challenge. You may have already seen the blog post that went out, but if not, this is the video representation of the same challenge in case you want me to explain it to you in video. So let's do this. ### The challenge We have this input payload, which is a text — we have a series of numbers separated by paragraphs. This is one paragraph, this is the second paragraph, this is a third paragraph, and so on. So what we're going to do is that we are going to count each paragraph separately. The first paragraph would be 2 + 1 = 3, the second paragraph would be 3 + 4 + 5 = 12, and the third paragraph would be 15, and so on with the rest of them. After you have those numbers, you are going to check which paragraph is the highest number. In this case it's 29, but you would have to figure that out in the script, not manually. So that's it for this challenge — you would have to generate the code to get to this 29 somehow. ### Clues and solutions If you need clues, you can go to the comments from the video, or you can go to the link where the whole challenge is written in a blog post, and there you'll be able to see some clues if you don't want to see the answer right away because you want to keep trying — the clues are going to help you get there. Or, if you're done with it and you're just ready to see the different answers that I provided for you, you can just go ahead and scroll down to the end and you'll be able to see the different solutions that I created for you. That's it. This is challenge number one — it's posted for the several challenges that I'm going to be releasing over the next few weeks. Good luck! --- ## DataWeave programming challenge #2: Rock Paper Scissors game score system Watch: https://www.youtube.com/watch?v=cZVwl55yTWo | Page: https://prostdev.com/video/dataweave-challenge-2-rock-paper-scissors-score | Series: DataWeave Challenges ### Intro Hello hello, I'm Alex Martinez, I am the founder of ProstDev, and today I bring to you challenge number two of DataWeave programming. ### The challenge In this case we have this input payload, which is just a series of characters, but each character represents a different move from the Rock Paper Scissors game: R is for rock, P is for paper, and S is for scissors. The rules of the game are simple — rock defeats scissors, paper defeats rock, and scissors defeats paper. Now, the first column of this series of characters is what your opponent chose, and the second column is what you chose. So for example, in the first round you both chose rock, so this is a draw. Now the way that we are keeping the score is: you will keep a 0 if you lost a round, you will earn a 3 if it was a draw, and you will earn 6 if you won the round. So in this case, since it was a draw, you will get just 3 points. The second round was rock and paper, so paper defeats rock, which means you won — you won 6 points. And on the third round, rock and scissors, and rock defeats scissors, so you lose this round, which means you get 0. So after you count the points from each one of the rounds, you will get a result out of 30. And that's all. ### Clues and solutions So that's the explanation for challenge number two of DataWeave in ProstDev. If you want to see the clues or the solutions that I provided for you, you'll have to go to ProstDev.com, or go into the comment section and click on the link in the description. Remember to subscribe so you get notifications as soon as new challenges are released. Good luck, thank you! --- ## DataWeave programming challenge #3: Count palindrome phrases using the Strings module Watch: https://www.youtube.com/watch?v=8owpvAhYfEs | Page: https://prostdev.com/video/dataweave-challenge-3-count-palindrome-phrases-strings-module | Series: DataWeave Challenges ### Intro Hi, I'm Alex Martinez, I am ProstDev's founder, and today I bring to you challenge number three of DataWeave. ### The challenge In this challenge we have a list of different phrases, and we will have to figure out whether these phrases are palindromes or not. If you're not familiar with what a palindrome is, it's basically a phrase, a word, a date — something that can be read in the same way from left to right and from right to left. Note that this does not include punctuation, special characters, or spaces. For example, this phrase, "Mr. Owl ate my metal worm" — we can still read the phrase if we read it backwards. The same applies for here, this is a question mark that should not be taken into account, but this phrase is also a palindrome. The same applies for one word like "Anna" — even if the first A is uppercase, it should not matter. And the same should apply for dates as well: 2020-02-02, if you read it backwards it still says 2020-02-02. After you can correctly identify which of all of these phrases are palindromes, you will now have to count all the characters in each of the phrases — this time including punctuation signs, spaces, or any other kind of character. After you get the character count of each one of these phrases, then you will sum them all to retrieve one single number. In this case the total sum of all of the palindrome phrases is 148. ### Clues and solutions That's it for this challenge, I hope you like it. Remember to go to ProstDev.com/blog to read all of the other challenges. If you're watching this from YouTube, go into the description of the video and you'll be able to find the article link to find all of the clues and the correct answer — or one of the correct answers, there are a ton of them that you can create. You can also comment your own solutions so others can learn from you. Remember to subscribe so you get notifications as soon as new challenges are uploaded. Good luck! --- ## DataWeave programming challenge #4: Solve the Tower of Hanoi mathematical puzzle Watch: https://www.youtube.com/watch?v=_3GJtx-NK8M | Page: https://prostdev.com/video/dataweave-challenge-4-tower-of-hanoi-puzzle | Series: DataWeave Challenges ### Intro Hi, I'm Alex Martinez, and today I bring to you challenge number four of DataWeave. In this case we are going to see the Tower of Hanoi mathematical puzzle. If you are not familiar with it, don't worry, I will explain the rules of the game for you, and then you can just go ahead and try it out on your own. ### The input payload We will have three different scenarios that you can see in the article — so if you're watching this from YouTube, go into the description of the video and click on the article in ProstDev so you can see the three different payloads that we are going to check. For the first payload we will have this one right here. Since this is kind of a game, it will contain the number of moves, so you will be responsible for updating this field every time that you move one of the disks. Here you can see the total number of disks for this game — in this case there are three. Then the target tower that we are going to move this to is tower C (or the stick, or the stack, or whatever you want to call it — I'm just going to call it tower because it's just easier for me). And here you can see the three different towers, A, B, and C. ### The rules of the game So now let me show you a visual representation of what all of this means in case you're not familiar with the game. This is a Tower of Hanoi game. Basically, here we have three disks, which is the one that we just saw in the payload. We start at zero moves, and the thing is that there are some rules: - One rule is you can only move one of the disks per turn or per move. Here you can see it says "minimum moves: 7", so all of this puzzle can be solved with at least seven turns or seven moves. So you can only move one of these disks at a time from one of the stacks to the other one. - The other thing is that you cannot put a bigger disk on top of a smaller disk. In this case this is number one, number two, and number three. So for example you cannot put disk number three on top of disk number one or disk number two, but you can put disk number one on top of disk number two or disk number three. Let's solve this quickly. Done — so we did it in seven moves, as it was the minimum moves to do. ### Solving it in DataWeave Now all you have to do is to do the same thing but in DataWeave. So once you finish doing all of the changes in DataWeave, you will end up with a payload similar to this one, where we have seven moves, we still have three disks, the target tower is still the same (it's C), the towers A and B are empty now (as we also had in our previous game that we solved), and the three stacks are going to be here — so here in C we have one, two, three, and they are also in order, they are not three, two, one, they are one, two, three. So you will have to solve and get to this answer. ### The other two scenarios There are two other scenarios that I can show you. This is the second scenario: here we only have four disks, we have all of the four disks in tower A, and we are going to move them to tower C. So that part is still the same, but the rules of the game might change a little bit — well, not the rules, but the way that you code this might change a little bit depending on whether this is an odd or an even number, so pay attention to that. And finally the third scenario. Here I am going to change things a little bit to make sure that you all are not cheating. Here we have seven disks, the target tower now is Y — now we're not using A, B, and C, we are using X, Y, and Z. So it's very important that you don't hard-code the name of the towers in your code, you have to make them dynamic. So now that the target tower is Y, we start with all of the disks on X, we're going to move them to the Y, and this is how this is going to end. This was done in 127 moves. ### Wrap-up And that's it. I really hope you like this challenge. Remember to subscribe to the YouTube channel or at ProstDev.com so you can keep receiving notifications as soon as new content is available for you. I will keep doing these challenges for you, so please subscribe so you can get up to date all the time. Good luck, bye! --- ## DataWeave programming challenge #5: Reverse a phrase's words, but keep the punctuation Watch: https://www.youtube.com/watch?v=Dk1MZQ7T-GM | Page: https://prostdev.com/video/dataweave-challenge-5-reverse-words-keep-punctuation | Series: DataWeave Challenges ### Intro Hi everyone, Alex here. I come here with DataWeave challenge number five for you. So in this case we are going to take a series of phrases, take each one of the words, and reverse them. So let me show you how we do that. ### The challenge In this case we have these five sentences or phrases — "hello world", "May the force be with you", and so on, in different presentations, so you can kind of test if they are all working perfectly. So each one of the lines is a phrase: "hello world", "May the force be with you", and then this one has an exclamation point, these two have commas, and then this one has an exclamation point at the end. So these are different tests that you can take to make sure that the reversal of the order is working. But all of the punctuation signs have to stay in the same index, or in the same position. For example here, if we turn this around, this will be "world hello", but the exclamation point will remain in this part. Same with this: if we reverse the order of the words we will have "force", the comma, and then the rest of the string. And then here on this one, at the end, we will have the exclamation point. Here you can see this is how all of the five phrases should look: "world hello" — notice that upper or lowercase on each of the words doesn't really matter, we don't have to do anything about it. "you with be force the May", "world hello!", and then both of these have "force," (with the comma) and then "May", "you with" and then exclamation point at the end on this one. ### Wrap-up So that's all, that's what you have to do for this challenge. It looks easy, but I really did spend a lot of time doing it, so I hope it's way better for you. And please let me know what your solution was so I can take a look at it, because mine was long and I feel like there's an easier way to do it — I just didn't find out how exactly. So that's all. Remember to go to ProstDev.com/blog to find the article, or if you're watching this from YouTube, go into the description of the video and you'll see the link to this article so you can post your solution and see others' solutions as well. Remember to subscribe to the YouTube channel ProstDev, or at ProstDev.com subscribe to receive notifications as soon as new articles come out. That's it, good luck! --- ## DataWeave programming challenge #6: Using tail-recursion to get the factorial of a number Watch: https://www.youtube.com/watch?v=fgzImvWfgHY | Page: https://prostdev.com/video/dataweave-challenge-6-tail-recursion-factorial | Series: DataWeave Challenges ### Intro Hi everyone, Alex here. Today we are going to check out the DataWeave programming challenge number six. ### The challenge In this challenge we have a list of four different numbers — three of them are positive and there's one negative. The first thing we are going to do is to get the factorial of each positive number from this list, so we would end up with something like this. In this case I am asking you to create a tail-recursive function to do this procedure. You can also use `reduce` if you want, but the challenge is to create a tail-recursive function so you can get more comfortable creating tail-recursive functions. And also notice that this doesn't work with a regular recursive function — that's why it has to be tail-recursive. Why? Because this 300 is going to reach the limit of the regular recursion, which is 255. The second step is to sum all of the results, so all of the numbers from here we're going to sum up, so we end up with a number similar to this. The third step is to retrieve the digits, or the characters, located at the positions from 20 to 25, like this. Now you have to make sure that you are returning a number and not a string — in this case you can see that there are no double quotes here, so this is indeed a number and not a string. ### Wrap-up So that's all for this challenge. I wish you good luck trying to solve it, and remember, try to use tail-recursive functions instead of `reduce` or a regular recursive function, which is not going to work. If you're watching this from YouTube, go into the description of the video to find the link to the article so you can see all of the details, and you can also post your own solution in the comments there. Remember to subscribe to ProstDev.com so you receive notifications as soon as new articles come out, and also subscribe to this channel on YouTube so you receive notifications as soon as new videos come out. All right, good luck! --- ## DataWeave programming challenge #7: Modify certain values from a JSON structure Watch: https://www.youtube.com/watch?v=qBiFkditGbk | Page: https://prostdev.com/video/dataweave-challenge-7-modify-values-json-structure | Series: DataWeave Challenges ### Intro Hi everyone, Alex here, back from vacations. This is the DataWeave challenge number seven. So in this challenge I am going to explain what we are going to do, but basically we are going to transform a JSON structure — or the values from a JSON structure — into something else. ### The challenge First of all, this is our input JSON structure. We have different objects, different arrays, and so on — some of them are nested, some of them are not, and some of them have numbers and some of them don't. And that's all right. What we are going to transform: first of all, we are going to transform every single value into uppercase, so "a" will be uppercase, "b" will be uppercase, and so on. But the second condition is that the values with the field name "this" are not going to be transformed. There are two fields, or two values, with this name "this": here we have "def" and here we have "h". Both of those values are not going to be transformed into uppercase. So we end up with something like this: "a", "b", and "c" have been transformed to uppercase, but then "def" has not, it's still in lowercase. Then the same applies for the rest of them — all of them are in uppercase with the exception of "h", because of the "this" name field. ### Wrap-up And that's all for this challenge. It sounds like it's super easy to do, but if you don't have the correct function you will have a hard time doing this. So I recommend you read the documentation first, because there is one function that is going to make your life way easier — you can just use this function and this problem will be solved with that. But you have to really read the documentation if you are not familiar with this function. If you are watching this video from YouTube, you can go into the description of the video to find the link to this article, where you will find more clues if you still don't know what function I'm talking about. So that's it for this challenge. Good luck, and I will see you in other videos. Bye-bye! --- ## DataWeave programming challenge #8: Sum all digits to get a 1-digit number Watch: https://www.youtube.com/watch?v=4BThMEAJ02s | Page: https://prostdev.com/video/dataweave-challenge-8-sum-digits-one-digit | Series: DataWeave Challenges ### Intro Hi everyone, Alex here. In this DataWeave challenge we are going to take all of the digits from any numbers, and we are going to keep summing up these digits until we get to just a one-digit number. Let me demonstrate. ### The challenge Here is a problem that we have: we have several numbers here, we are going to sum them all, and then keep summing each digit until we end up with just one digit. In this case it's going to be 2. Let me give you another example. In this case we have 1 and 2, so if we sum 1 and 2 we end up with 3. Or in this case we have first 1, 2, 3 and then 4, 5, 6, and we end up with 3 — we can either do 123 plus 456 like this, 123 + 456, we end up with 579, so now we do 5 + 7 + 9, we end up with 21, and now 2 + 1, we end up with 3. Or another way to see it is 1 + 2 + 3 + 4 + 5 + 6, we end up with 21, so 2 + 1, we end up with 3. ### Wrap-up That's all. I hope you like this challenge, it was very fun to do, and I will see you then in a different video. Bye! --- ## How to Enable MuleSoft Vibes in VS Code (Agentforce, 100% Free) Watch: https://www.youtube.com/watch?v=s0k7z6VoDuI | Page: https://prostdev.com/video/enable-mulesoft-vibes-in-vs-code | Series: Other MuleSoft videos ### Install the Anypoint Extension Pack All right, let's learn how to enable MuleSoft Vibes. First of all, if you haven't installed the Anypoint Extension Pack in your VS Code, go ahead and do that. It's as simple as finding it in the Extensions marketplace and clicking Install — just make sure it comes from Salesforce.com. Once it's installed, you'll see the MuleSoft logo, and this will be Anypoint Code Builder. But that isn't quite what we want. You'll also have this second logo — that one is MuleSoft Vibes. Let's go ahead and log in to Anypoint Platform. ### Log in to Anypoint Platform Now it's going to do the usual thing: you open the login in your browser, log in there, and then it sends you back to VS Code. As you can see here, I don't have the AI features yet — so let's learn how to turn those on. Let's head back to Anypoint Platform. I'm going to log in with my username and password. Once we're in, we can go to Anypoint Code Builder, and right from the start you can select "Go to Access Management." ### Enable Agentforce in Anypoint Platform Here, first, let's enable Agentforce. You'll accept the terms and conditions for Agentforce in Anypoint Platform. Once we have that, we're going to need to add the Salesforce org so we can enable some of the other settings. ### Create a Salesforce Developer Edition org Let's head to developer.salesforce.com and click on free trials. In here you'll be able to see the Developer Edition — there are other free trials you can try out, but for now let's just get our Developer Edition free trial. Once you add all of your details, click "Sign me up," and you'll receive your details via email. After you set up your password and everything, you'll be able to log in to your new organization. You'll receive a verification code in your email, and after that you're inside your Developer Edition Salesforce org. ### Turn on Agentforce and Einstein in Salesforce Let's click on Setup. In Quick Find, search for "agentforce" and head to Agentforce Agents. Just switch this on and that's pretty much it. You can also make sure you've turned on Einstein, just in case — search for "einstein," open Einstein Setup, and turn it on. Just like that. ### Connect Anypoint Platform to your Salesforce org Now let's search for "mulesoft" and select the Anypoint Platform Setup. Click on "Complete the Connection" and copy your Salesforce org tenant key. Once you go back into Anypoint Platform, you'll be able to paste that right there. By the way — if you get an error message ("Organization is not yet onboarded"), just wait a few minutes, then refresh and try again. That's it. Add a name so you can recognize the org later — I'm just going to call this "sfdevaccount." Click Add, then copy that Anypoint Platform org key, paste it back in Salesforce, and click Connect. Ta-da — that's it. ### Enable Agentforce in Anypoint Platform (final toggle) Now just make sure to enable Agentforce in Anypoint Platform. And that is all you need to be able to use MuleSoft Vibes from Anypoint Code Builder — or from VS Code. ### Try it out Now you can just try again, and you can see that you have MuleSoft Vibes enabled in your VS Code. That's all for this video — I hope you liked it. Remember to subscribe and stay posted for more guides and MuleSoft tutorials. --- ## Why is my Array of Strings not filtering correctly? It may be an Array of Strings and Keys instead! Watch: https://www.youtube.com/watch?v=0VgIRB3JfZM | Page: https://prostdev.com/video/array-of-strings-and-keys-filtering-dataweave | Series: Other MuleSoft videos ### Intro Hello, hello everyone! My name is Alex Martinez, and today I come with a very short video, because I just wanted to explain to you — in a video, not just in a blog post — what my problem was and how I solved it, in case this happens to you too. ### The problem Here's what was happening to me, and I didn't know why. I had several sets of arrays — arrays of what I thought were strings, but that was not really what was happening. For example, as you can see here, we only have one array, and in this array we have array one and array two, just for demonstration purposes. We can see that in array one, the `item` is actually a Key — you can see from here it's a type Key — whereas these others are Strings. But when I was trying to do this, as you can see from the output, they all look like Strings: this Key doesn't look any different from the Strings. So I thought they were all Strings. I was wrong. My problem started when I was trying to do `distinctBy` on the array. Here's the thing: if this had not been a Key, this would correctly filter out the arrays that are repeating, like these. But since one of them was a Key, this is what was happening to me — and because you can't really see the difference between what is a Key and what is a String in the output, I wasn't aware that it wasn't a String. I was forming this array somehow such that I was actually storing Keys instead of Strings. So I ended up with multiple arrays of different types — not all of them were Keys (because if I were to make them all Keys it would also filter correctly, but that was not the case). It would end up with something like this, and this would not filter correctly. ### Solution 1 There are two solutions I can think of for fixing this. One is to create a `map`. Here we don't need the index, so just like this. And then we do another `map`, because first we're iterating over each of these arrays — we have an array of arrays of Strings and Keys — so we have to map again. In the second `map` we do `as String`, so now this Key is going to become a String, but it will also check all of the different ones to make them a String too. Then we can do the `distinctBy`, and now — even though this was a Key — it will still be filtered correctly. So that is one solution, but it may be a little too expensive if you have a lot of different items, because you're doing two `map`s and then the `distinctBy`, so you're kind of going through the whole array. ### Solution 2 If you don't want to do that, or there's another way, or you cannot change the types of the items, you can do this instead. Rather than doing two `map`s, we can keep the `distinctBy`, but inside it this becomes `joinBy`. The separator can be whatever you want — it can be empty, a comma, a dash, or whatever you want to make them different. In this case I'm just going to put a comma. So this means that when we do the `joinBy` on array one — let me just do it here really quickly — array one, with the comma, this is what it ends up being, and its type is a String. If we do a `typeOf`, this will be a String. And if we do the same with array two, this will also be a String. If we take a look at this, it would be Key and String — so by doing this, we're transforming the values inside into one big String instead of having the array. This essentially becomes an array of Strings, instead of an array of arrays of Strings and Keys. That's another way. This way you're outputting whatever is the same in the values, but you're not actually changing the item — in case, for whatever reason, you cannot change it. Just to make sure we can check on this: if I do zero, we get the first array, and if I do zero again, we get the Key. So if I do `typeOf` on this whole thing, we end up seeing it as Key, because that's the first thing that's coming. If I were to remove this, it would be String. So that way we're keeping the same types of the values, if that's what you want. And if not, then you can also do the `map` and change the actual item to be a String. In my case, this is the solution that worked best for the use case I was doing, so this is what I ended up doing — because going through all of the Keys was really expensive performance-wise. ### Ending All right, that is all for this video. I hope you liked it, I hope this helps, and I will see you in the next videos. Remember to subscribe and to follow me on all of my socials — everything: LinkedIn, YouTube, and prostdev.com. All righty, see you then. Bye! --- ## How to upsert fields from an object in an array with the update operator in DataWeave 2.0 | Mule 4 Watch: https://www.youtube.com/watch?v=TqrmeLhGtQc | Page: https://prostdev.com/video/upsert-fields-object-array-update-operator-dataweave | Series: Other MuleSoft videos ### Intro Hello hello everyone, my name is Alex Martinez, and today we are going to learn how to upsert fields from an object in an array with the `update` operator in DataWeave. ### Use case This is our use case. We have here an array of objects, and each object — let's say it's a user — has an `id`, a `name`, and some `notes`. We want to update one of those users with this information right here from this variable, `updateUser`. In this case we have `id`, `notes`, and `newField`. Note that we don't have a name, but we want to match this `id` to the `id` from the array, and then update that information from there. In this case the name is not in the `updateUser` variable, so we want to keep the name from the original object. But for the notes, since we do have some notes, we want to update them here. And this is the important part of the video: we want to add this new field into the original object even though it's not originally there. That is why we need to upsert some fields. ### map function — iterate through the array and match the user to update The first thing we need to do is match this `updateUser` `id` to one of the IDs from these users. To do that, we have the array `users` and we are going to do a `map`. We do the user here, remove that, and then inside we are going to add a conditional. So if we select here `if`/`else`, then we can add some conditions: if nothing matches, let's just leave `user`, but if something matches we want to match the `user.id` to what we have in `updateUser.id`. I'm going to just put three question marks here so I can see if the rest of my code is working, and as I can see, it is working — it's just not outputting anything yet because I haven't implemented this specific path. Note that I'm using this operator (`~=`) instead of `==`. This is because the `==` operator will not match if I have something like this, but if I use the other operator it will match, because it's only making sure that the coerced values match and not the actual type of the value. ### update operator — update each field with the new values Once we have this conditional, now we can get started with the `update` operator. We are going to update the user that does match the `id`, and then add the `update` operator like this. As you can see, it is using cases — each case will be one different field. The only field that we do not want to change is the `id` field, because we already know that they match, so why bother. So `case n` — this is just the name of the variable, you can put whatever you want here, I'm just going to put `name`. Sorry, `case n at @name`: if there is something at `@name`, that would be inside the object, inside the `user.name`. In this case it would be "Alex 2" because I'm using the other `id`. Then what I want to do is update that with `updateUser.name`, and as you can see, we are getting a `null`. I'll get back to that, but for now let me continue with the rest of the cases. So `case n at @notes`, then I want `updateUser.notes`, and as you can see it has been updated correctly. Let's continue now with `case n at @newField`, `updateUser.newField`. In this case, as you can see, nothing is being added, because we are getting the values from the user, and because this user doesn't have the new field, it's not putting it right here. This is where our upsert operator will come in handy — but before we do that, let's address something. ### default — keep original values when needed We did notice that the name now is appearing as `null`, because I am just setting `updateUser.name`, which is `null` since I don't have any name in `updateUser`. If I did have something here — let's put `name` "1 2 3" — then this would be updated with "1 2 3". But because that's not the case, nothing is being output. So what we can do here is simply add the `default` keyword and then say `user.name`. So if `updateUser.name` doesn't have anything or is `null`, then please use whatever is inside `user.name` — in this case "Alex 2" — so nothing is being changed. Another way I can do that: because I already have this variable being created here, I can take that `n` and put it right here, and it is the same value. I can just repeat the same thing for all of the cases, just in case I need to use it later. Once we have this covered, now even if I remove the notes, this will still have the value it had before. So this is exactly what I want. ### upsert operator — insert the new field into the original object Now, to add this new field, I need to use the upsert operator from the `update` operator. That is super simple to do: I just have to add an exclamation mark (`!`) right here, and now it is saying, "if this new field does not exist, then please create it" — and it is being created properly. ### Conditionals inside the update operator — avoid null values per field Now the other tricky part here: once I have this code, if I'm going to update some user but I don't have this new field — let's remove it — then this field will still be created, but with a `null` value. So it depends: if that's what you're looking for, then cool, but if you don't want that, there are two different ways to fix it. One way is to add a conditional right here. It doesn't look as fancy, but you can do it. You're pretty much saying: take this from here, `if` `newField` is not empty, then do this thing, but if it is empty, don't do anything. So now if I don't have the `newField`, it's not being created, but if I do have it, then it is going to be created. That is exactly what we want in this case. ### do — create a local scope if needed A super quick note: if you don't want to put this same line twice, you can also use a `do` operator. For example, here inside the `if` we can open a `do`, and then this whole thing will be inside the `do`, the `else` will be outside, and we can close the `do` right here. So now with this we're creating a local scope, so we can create a variable and say `newF`, for example, and we can simply take all of these from here, `newF`, and we can also update this with `newF`. Now `newF` will be this, and this will be `user.newField`, and now it also works. So you can also do something like that if you don't want to keep repeating stuff — it depends on however you want to do it. In this case I'm just going to leave it without the `do` so we can continue with the other example. So if you don't want to do that whole thing, because it was a lot, we should just remove this `if` and leave it like it was before. Perfect. ### skipNullOn — avoid null values in the whole output Now if we remove the new field one more time, we have the new field `null`. So the other thing you can do: if you go to the output directive right here, if you're using JSON, then you'll be able to do `skipNullOn` and you can select anything from an object or everywhere. In this case I'm just going to select `everywhere`, and now everything that is `null` will be erased from here. Let me demonstrate with another field really quickly. If I add a test field right here — oops, sorry, if I put it as `null` — and I put this back, then this will also be removed. As you can see, both of these fields are removed. So it also depends if that's what you're looking for, or if that's not what you wanted. ### Ending But this is how you use the upsert operator, and how you can also add some conditional values to each field if you want to — it goes right here, remember that. That is all for this video. I hope this was helpful, and I will see you in the next video. Bye! --- ## Interactive MuleSoft tutorial: Anypoint Platform API Catalog by Rolando Carrasco | UAPIM | Exchange Watch: https://www.youtube.com/watch?v=wBllgVhIrg8 | Page: https://prostdev.com/video/interactive-tutorial-anypoint-platform-api-catalog-killercoda | Series: Other MuleSoft videos ### Intro Hello everyone, my name is Alex Martinez and today I am super excited to talk to you about this amazing dynamic tutorial where you can actually go and interact with it, do stuff, and learn about it. This awesome tutorial was created by Rolando Carrasco, who is one of the MuleSoft mentors — I believe he might be an Ambassador, I'm not sure at this point — but it's amazing because you will be able to learn about what you're doing and do it in an actual new environment. You don't have to do it on your local computer; you can just use this remote computer, access everything there, and learn to do it right there. Right now I just accessed the link that you will be able to find in the description of the video or in the article. Once you enter, it will just take a moment — as you were seeing, a lot of things were running and everything was being installed. Once it's installed, here it says you just have to select "keep the local version currently installed", as you can see here. So click on that, or just leave it like that and press enter so you can select it. Now it will continue installing some stuff, and once that is done you will be able to get started with the tutorial. On the right side of the screen you will be able to see everything that is running, and on the left side of the screen you will be able to see the actual tutorial that Rolando created for us. So after you do that, just wait a moment for this to be installed. There are also some prerequisites: an Anypoint Platform account, API Catalog CLI permissions, acquiring the organization ID, and having Node.js and npm. All of this is going to be installed here automatically. The code editor will also be used here, and the project files will come from a GitHub repo that was generated by one of the technical product marketing managers at Salesforce. In this scenario, here is what you will learn: how to create a Connected App, how to install the API Catalog CLI, how to authenticate the API Catalog CLI, how to introspect the set of API specs, and how to use different MuleSoft Anypoint Platform API Catalog commands. ### Getting started All right, this has finished, so we can just click on start. Now we are going to start. This is our whole system — this is our terminal. If you click here on "editor" you will be able to see what is actually happening behind the scenes. This is Visual Studio Code, as you can see here. You have another terminal, and here you have the Explorer of all the files that you have in case you want to take a look at them, but you can just stay here on the terminal tab. ### Installing the prerequisites The first thing that you have to do is to install curl, jq, node, and npm. You can do this on your own; I am just going to guide you through it and do it myself so you can see how this is done. In case you have any issues or get stuck on anything, you can come back to this video and follow through. You can simply click on this and it will automatically execute everything for you. As you saw, I just clicked on it and it started automatically installing everything (I zoomed in a little bit). That installs curl, jq, node, and npm. Now you need yq for some expressions, so let's click here. This is running, and that was installed. Now validate node and npm again — you can just click on the command and it will run it for you. As we can see from here, it's on version 20.12.1. ### Installing the API Catalog CLI Then we're going to install the MuleSoft API Catalog CLI, so click on that and it will install. It's going to take a little bit of time to install. While that's being installed, I just wanted to quickly show you here: if you take a look at this part, it says that I have 53 minutes to finish this. To be honest with you, I did it once already and it took me about 20 minutes. So you do not need to have the Plus membership — just with the regular membership it's okay, and you will be able to finish this in less than 60 minutes. All right, this has been installed. Now let's click on "API catalog" to see that everything was installed correctly, and we are able to see this output right here, which is what we need. ### Cloning the GitHub repo Let's click on next. Now this is telling us to clone the GitHub repo, so let's do that. Once we have that, we will change the directory to that place. You can also go back to the editor here and see that the repo was actually cloned. Now let's go back to tab one and click on next. ### Creating the Connected App in Anypoint Platform Now we are going to start creating the Connected App from Anypoint Platform. First of all, you have to open Anypoint Platform, so let's open it by clicking it here. It's asking us to sign in. I'm just going to go ahead and sign up and create a new account. All right, I'm pretty much ready — let's create this account. My account was created. Now let's see what we need to do. We have to go to Access Management and click on Connected Apps, so let's do that: here, Access Management, and then Connected Apps. Then we need to create a new app, put the name, and click on "App acts on its own behalf". So, create app, "killercoda" — whatever name you want to put — just click on this and then add scopes. Which scopes are we going to be adding here? We have an "API Catalog Contributor" scope — so this one. Then in the general section we have "View Environment", this one right here. And that is all, just two scopes. Click on next. This is my business group; click on next. In my case it's sandbox, so click on next, and that's it. Perfect. Click on "Add Scopes". Now you have the two scopes, and click on Save. Perfect, now you have it here. So now we just have to copy the ID and secret that we will need in a moment. Let's click on next step. ### Setting up credentials and authenticating Now we're going to be setting up our credentials. I can just click on this, and this is going to ask me to add the client ID. If I go back here, I can click on "copy ID", then go back and paste it here and press enter. That's it. Now let's do the same for the secret: let's go back, copy the secret, then go back and paste it. There we go, click enter. Then we are going to create the auth string. We are going to be setting up the client ID, then a colon, and then the client secret. So let's click on that. Let's validate that it was created by echoing the auth string, and we do — awesome, there's that. Now let's click on next. Now we need to authenticate to Anypoint Platform, so we need to run this, which is doing a curl POST request to `anypoint.mulesoft.com` setting up the access token. Let's run that, and we can see that we received an access token. ### Getting the organization ID Now we need to get our organization ID. Rolando here is telling us to go to API Manager and get it from the URL. If we go to API Manager, you can indeed get it from the URL — in this case it's like "4500..." and it ends on "EDB". Another way is to click here on "Environment Information" and you will also see it right here: business group ID, and we have the same one, which is "4500...EDB". Just another way, because I want to show you all the ways: if you go to Access Management, click on Business Groups, and then this is your business group, you can get it from here as well — business group ID "4500...EDB". So whichever way you want to retrieve this org ID, just retrieve it. Now run this, paste it here, and press enter. There's that. ### Generating the descriptor file Now let's click on next. Now we need to generate a descriptor file. If we run `api-catalog create descriptor`, we can run this, and that created the descriptor. Awesome, there's that. We can also check that it was created. If we go to "follow tutorial", we have here the `catalog.yaml`, and this is what was created. So you can also go check it out there in the files if you want. Let's go back here. When you're following the whole tutorial, you can of course take a look at everything that's happening here; I will just skip that for a moment. So we were here — we want to copy the following text. You can simply click on it and it will copy, and then we want to paste it here in the `catalog.yaml`. Paste it, and it's saved (it's automatically saved, you don't have to save it). ### Publishing assets to Exchange Now get back to tab one. You can execute the following command to publish the asset: `api-catalog publish assets`. Before I do that, I am going to go to my Exchange to show you that I have nothing else here. So here we go, this is my root and I have nothing here. Let's go ahead and run this. Okay, so that was a dry run, so you can see the contents from the dry run here, and it actually says that it's processing — so that's good. Now let's actually run it. If we click on it, it will actually start. Now it's doing the whole thing; you can see that it's publishing to Exchange. We're seeing the output, and after that we will be able to see the asset actually on Exchange. All right, so the API was successfully published. Now if we go back to Exchange, let's just refresh this, and now we can see that we have the Shipping API, the Invoice Payment System API, and we actually have a connector of the Invoice Payment System API. So there's that — we have that new thing. Click on next, and that is all. If you check out the Shipping API, this is what we published, which is awesome. You have everything here — you even have the snippets. Well, this one doesn't work well, but this one works, awesome. You can just do everything from here, and this is a great way to learn because you can see all of the commands and execute them by just clicking on them. You don't have to set up anything else. ### Ending It took me, again, less than 60 minutes. If we take a look at my screen and see how much time I have left, I now have 40 minutes left, so as you can see it did take me less than 60 minutes. You can just go back and restart, or do more stuff about this. This is an awesome way of learning. Thank you, Rolando, for creating this — I really appreciate it. I don't know how long it took you to do this, but it's awesome. You can get all of the information in the description of the video or in the article. If I already created the article, it's going to be in the description of the video; if not, you can just go to prostdev.com or check out everything in the description of the video on YouTube. And that's all. This is awesome — thank you, Rolando, again, and I hope I get to see more people creating interactive tutorials on this platform called Killercoda. That is how you solve it, in case for any reason you weren't able to do it all on your own — that is totally fine. You can also send some feedback here for Rolando, or you can contact him on LinkedIn or on Slack if you're on the MuleSoft community Slack workspace. So from my side, remember to follow us on youtube.com/ProstDev, or remember to subscribe at prostdev.com, because you will receive notifications as soon as we publish new stuff. All right, that's all for this video. I hope you liked it, I hope this was useful, and I hope there are more tutorials from this platform. See you later, bye! --- ## Exposing DataWeave: Map+Filter vs. Reduce - which is faster? Watch: https://www.youtube.com/watch?v=xCPb678hDAo | Page: https://prostdev.com/video/exposing-dataweave-map-filter-vs-reduce-which-is-faster | Series: Other MuleSoft videos ### Intro Hey everyone, Alex here. I don't know if you have seen, but I did a blog post about exposing DataWeave: what is the difference between `map` and `filter` versus using `reduce`? I really, really loved this one, so I thought I would make a video for you to see it in action and not just read through the whole post. So let's do this. ### The example criteria Pretty much this is the example criteria. I have an array of objects — I have two objects in this case, just for an example — so we have `id`, `criteria`, and `yearOfBirth`. Pretty much what's going to happen is that we need this output: depending on the criteria, for example here, if the criteria is less than three, then this will continue to the output. So in this case we have criteria zero and criteria five, so only this one should pass to the output. Now the other thing is that we are also adding fields. In this example we are adding the `isValid` field, because we need it for the `filter` that we are doing here after the `map`, and we are also adding one additional field, whatever, and for example the field `years` that is counting how many years this person has. ### Map then filter (the original) So yes, I hear you — a lot of people told me, "Why are you doing the `filter` after the `map`? You should do the `filter` before the `map`." And yes, I hear you. So this is one way, this is the original way that the script was doing it. At any point you can pause this if you need to take a look at the code; you can also see the code in the article, and you'll be able to find the article in the description of the video. In this first scenario we were doing the `map` first, so a whole iteration — let's say two objects — and then we were doing the `filter`, so there's another iteration to the whole thing. So we did a total of four processings of the objects. ### Filter then map This is the second way: we have the `filter` first and then we do the `map`. So here in the `filter` we do the `filter` first, so we are doing the whole iteration first, and then we are discarding one object and keeping one object. So we did two, and then in the `map`, since we only have one object now, we are doing three total iterations — or three objects instead of four. So this is one object less if you do the `filter` before the `map`. ### The reduce approach But then I thought, "Okay, is there a way that we can do just one iteration instead of more than one?" And I came up with this `reduce`. For example, in this case, we do the `reduce`, we are immediately checking what the criteria is. If the criteria is met, then we are adding the object to the accumulator with the new fields, and if the criteria is not met, then we are just continuing with the same accumulator that we had before. So in theory `reduce` is only doing one iteration in total, so just two objects, right? We had four here, three here, and one here. So in theory the `reduce` option should be the most performant one, because we are only going through the whole thing once. ### Benchmarking all three (10,000 items) You would think that, but I did this whole script that you can also find in the article. Pretty much it's creating the items from this variable, because I'm creating 10,000 items which have the `id`, the `criteria`, and the `yearOfBirth`, which are random integers. Then we have the three different functions: we have map and filter, only reduce, or filter and map. As we saw from the previous scripts, here we have the `map` and then we have the `filter`, and in the only-reduce one we have the same thing. Now the criteria is going to be 50 instead of five or three, because now we have more objects. And finally we have the filter and map, where we are doing the `filter` before the `map`. So we have the three different options. Then I created here some scripts to run this. It's timing each of the approaches and then ordering them by the fastest one first, and the one that took longer is going to be at the end. As you can see from here, filter and map took like five — I don't know how much that is. Now I have map and filter, and we have only reduce. So only reduce took way longer than the other two. Filter and map, and map and filter, are close, but filter and map ends up being way less time. So I can run this more times if I just do a small change to the script. As you can see, this changed, and I can keep doing it — just adding spaces or something — and the times keep changing. So you will have different times every time, but as you can see, the order stays the same: filter and map, and map and filter, are always pretty much very close to each other, but `reduce` is way, way longer. It takes way longer to run `reduce`. That was a shock for me. ### Verifying the functions actually run And then the other thing, just in case you are not sure if this is actually running and working for all of the functions — it is. If I come here, I have the same script as I did before, but now it's outputting here the total time and the start and the end. So for example, for this one I am timing just the filter and map, and I am actually outputting the results. If I scroll down, you can see these are almost 40,000 lines of output. We have here the timestamp at the end, and we are counting the total time by doing the same thing that we did in the other script, which is the end minus the start, so we get the actual number. As we can see from here, this number is super similar to this other number, so it is running. Again, I can just keep making changes here to the script so I can rerun this, and this keeps changing — the number will keep changing — but it stays pretty much the same. So if I were to do, for example, instead of filter and map, I can take the only reduce and put it here, and the time will change again. As with the other one that we saw, that number is pretty similar to this number, right? And we can still see that we have almost 40,000 lines of output, so this is actually running everything. Again, I can run it several times just by modifying something in the script, like adding a space or something like that, and this number keeps changing, but it never gets to how the other number was. We can do the same for map and filter, and it will be very similar to filter and map, but again just a tiny bit slower than the other one — but not as slow as the `reduce`. ### Ending And that's all. This is a short video; I just wanted you to see this in action in case you don't want to run everything on your own. That's fine, I got you. You have the article where you can see all of the scripts. You have this example where you saw how the different functions are working, then we saw this one which is actually comparing the three functions at the same time, and then we also have this that is showing the actual results that are thousands and thousands, because we are still running 10,000 here. If you want to try with more items, you can of course change this, or if you want to try with less items — let's say five — you can also change that and see the differences in the output. So there's that. I hope you like this video. Please remember to follow me and subscribe to the channel, because I will continue creating great content for you. If you have any suggestion, any question, any comment that you have for me, make sure to comment — you can comment here in the video, or you can comment on prostdev.com/blog articles, and I will make sure to follow up there. You can also contact me however you like; just let me know what you think and I will make sure to make adjustments for you. All right, that's all then. Bye! --- ## How to add JVM/Command-line arguments to the Mule 4 Runtime in Anypoint Code Builder (ACB) Watch: https://www.youtube.com/watch?v=-u9jZ45Ut1A | Page: https://prostdev.com/video/add-jvm-command-line-arguments-mule-4-runtime-anypoint-code-builder | Series: Other MuleSoft videos ### Intro Hi everyone, welcome to another ProstDev video! Yay. My name is Alex Martinez, I am ProstDev's founder, and today we are going to learn how to add JVM or command-line arguments to Anypoint Code Builder, or to the Mule Runtime in Anypoint Code Builder. So let's get started. I will assume that you already have all the prerequisites — you already have Anypoint Code Builder installed and you kind of know how to use it, more or less — so I will just get right to the point. ### Create the base Mule project First, let's click on "Develop an Integration" and select any name. I'm just going to use "test". Select your project location and click on "Create project". It's going to take a little bit for this to finish running, but once it is, you will be able to see your code right here in the canvas. All right, here we go. I'm just going to add this super quickly — you can go to the GitHub tutorial repo in the description of the video — and I will just paste this right here and format the document. There we go. It will just take a few seconds for this to load. So pretty much we are just setting up an HTTP listener, and then we are setting up the payload. But here is the thing that we are going to have to pay attention to: in the HTTP listener we are setting up the HTTP listener host and port using properties. In the configuration properties we are using an `env` property that we will have to pass in runtime. You can also use this same tutorial in case you are using encrypted properties or secure properties — then you will be able to pass your secure key as an argument, just as we will do here today with the `env` property. Also, you will notice that I'm not using best practices: I am putting my global elements right here in this same file. I would advise you to create a `global.xml` file in case you are using a real project. Don't do this — this is just for demonstration. Now, if we go to the Explorer on `src/main/resources`, let's create a new file, and this file is going to be `dev.properties`. You can also create a YAML file in case you are more familiar with that — that is totally okay, just make sure that it says `.yaml` instead of `.properties`. In my case I'm just going to use `dev.properties`. All right, that is all my setup. ### Add arguments to the Runtime Now for the fun part. Let's go to the main screen of Anypoint Code Builder to learn how to do the settings. If you see here, there is this button that says "Open ACB settings", so you will click on that. It will open the Mule settings, or all of the settings that start with "Mule". If you see here, we have the "Mule Runtime: Default Arguments" setting. If you go right to the right of this huge string, you can add a new argument by saying `-M -D`, then the name of your argument or your property, like `env`, and then this will equal the value, so `-M -Denv=dev`. If you have an encrypted property or something like that you can also do this and then `secure.key` equals your secure key. This is how you will send this argument to the Mule Runtime once you run this application. ### Run your app locally So let's see if this works. If we go here to "Run and Debug", just make sure that you have selected "Debug Mule application" and click on this green thing. All right, so this is deployed. If I open my Thunder Client that I have right here, I can create a new request, and this will be to `localhost:8081/hello`, I believe. So let's send this request, and we receive a 200 OK with the Hello World. So yay, this application worked correctly because we are sending the argument. Now, just in case you want to make sure that this is working, I'm going to stop this application, go back to Anypoint Code Builder settings, and now I'm going to remove the argument that I just added there. Save, close, and I'm going to run this again. Now we can see that our application has failed — it has not deployed properly — and if you go here it will say that the value for key `env` is not found. So you will have to set that up in the settings. ### Conclusion All right, that is all for this video. I hope you liked it. Please go to the GitHub tutorial repo in case you have any issues, or if you want to copy and paste stuff. ### Ending screen Remember to subscribe to the YouTube channel, follow us on socials, and go to prostdev.com for more articles and videos. All right, see you next time. Bye! --- ## Anypoint Studio 7.10.0 | Notification center for new versions of connectors/modules Watch: https://www.youtube.com/watch?v=kpOpW5IY5bY | Page: https://prostdev.com/video/anypoint-studio-7-10-notification-center-connector-versions | Series: Other MuleSoft videos ### Intro Hey guys, this is Leonardo Gonzalez from Mexico City. In this video I'm going to do a very quick tutorial for our friends at ProstDev, one of my favorite MuleSoft development communities. So this video is about a new Anypoint Studio feature in the newest version, which is `7.10.0`, that has been available since last week. What I want to show you is the new notification center for new versions of connectors. This will help us get some recommendations when new connector versions are available. ### Release Notes As of this new version, it's no longer necessary to go to the Anypoint connector release notes right here to discover which are the new versions of the connectors that we are using in our Mule applications. For example, I'm using here the File connector and the Database connector, so we can do this with a few clicks in our applications. As you can see, the new version was released last week, and the release notes indicate that this new feature is already available. So we are going to try it. This is saying that Anypoint Studio `7.10.0` introduces a new notification mechanism to help us keep up to date with the newest version of each module in Anypoint Exchange. So let's try it. I already have downloaded the latest version of Anypoint Studio. If you don't, you can download it right here on the official Anypoint Studio website, so you can do that. This already installs my latest version of Anypoint Studio. ### Demo So all we have to do is right click on our Mule project and choose the Manage Dependencies option, and finally the Manage Modules option. That's it. The blue dots on the left of the version of each connector indicate that there is a new version of that connector available in Anypoint Exchange. So to update them to the latest version, it's very easy: we just have to click on the Update version button right here for each of these connectors, and that should be it. Just click on Apply and Close, and that's it. All versions of our connectors are now up to date. As you can see in the `pom.xml` file, this guy was `1.10.0`, now it's `1.10.3`. And also, where is the other one, which is the File connector? This guy also was `1.3.0`, now it's `1.3.4`. ### Ending So that's it. You are on the latest version now. Thank you guys, happy coding! --- ## What is an API? | Understanding APIs (Part 1) Watch: https://www.youtube.com/watch?v=1Gb3IW0UvuY | Page: https://prostdev.com/video/what-is-an-api-understanding-apis-part-1 | Series: Other MuleSoft videos ### Intro So you're here because you want to learn more about APIs. Well, in this series of videos I will explain what APIs are right from the beginning. My name is Alex Martinez — you can call me Alex. Get comfortable and let's get started. ### API Okay, so first of all, what does API mean? API means Application Programming Interface. And that doesn't tell you anything, so let's continue. ### Implementation What is the implementation? The implementation is the body of the API, and this implementation can contain different programming languages. It can be made out of Java, PHP, Python, or MuleSoft. ### Request/Response So this implementation — how do you call it, or how do you interact with it? Well, you have the implementation, and then you send something into the implementation, or the body of the API, and the API (or the implementation) will return something to you. You say, "Hey API, what's up?" and the API responds, "I'm good." These two little things are called the request and the response, because you request something to the API and you get a response back from the API. ### Aspects Now, an API can have four different parts or aspects to it. It has inputs, operations, outputs, and data types. So what are these things? ### Inputs First we have the inputs. The inputs basically have data or information that we're sending into the API. This is just the first handshake that we do to call the API. We also send the operations into the request. ### Operations The operations basically tell the API what we want the API to do with the data that we're sending in the input. ### Outputs Next we have the outputs, which are part of the response. In the response, the API will give us back some data, and this data is the outputs. So you send data with the inputs, and along with some operations I will tell the API exactly what to do with those inputs and what we need to receive from the outputs, or the response of the API. ### Data Types We also have data types. Data types can be numbers, words, files, formats — in this case I put here some examples like CSV, JSON, and XML. If you're familiar with these terms, cool; if not, don't worry about it. Let's just say that if you send numbers in the input, you may receive numbers in the output. But also, let's say that you send some numbers in the input and you want to receive words, or strings, in the output. Those are different data types. You can have the same data types in the input or in the output, or even in the implementation itself. Remember how we just said that the implementation can be made out of different programming languages? If all of the implementation was created in Java, then most likely all of the data types that will be used inside of the implementation will be Java objects, Java arrays, collections, strings, booleans — all of that fun stuff. ### Recap And that's it. So, just a quick recap on the beautiful diagram that we just created: we have the implementation, we have a request, and we have a response. We send the request into the implementation (or the body of the API), and we receive a response back from it. In the request that we send into the API, we put inputs and operations — this is basically to tell the API (or the implementation) what to do with the data that we're sending. After the API processes all of this information, it will give us back an output, and remember that the outputs also have data types. So this is just Part 1 of the Understanding APIs series. For now, it's just important that you understand the different aspects or parts that an API can have: request, response, inputs, outputs, operations, and data types. That's all you need to understand for now. ### Ending So if you want the written version, remember to go to prostdev.com and check out the Understanding APIs post series. Again, my name is Alex Martinez — you can call me Alex. I hope this was fun, and see you in the next video. --- ## 5 reasons why you need an IDE Watch: https://www.youtube.com/watch?v=wFSUFu6zGFI | Page: https://prostdev.com/video/5-reasons-why-you-need-an-ide | Series: Other MuleSoft videos ### Intro Hi there, I'm Alex Martinez — you can call me Alex. For this video I will talk about IDEs. You may already know what an IDE is and why using one is a great idea, but if you're new to the software development world you may want to take a look at this quick video of five reasons why you need an IDE and how it can save you so much time. ### What is an IDE? Okay, first of all, what is an IDE? An IDE is an acronym for integrated development environment. It's nothing more than an application that can be used to develop any kind of software. You can download it from the creator's website and install it like any other app. IDEs provide a graphical user interface that can be easily used to perform basic day-to-day functions. A metaphor that comes to my mind is a calculator. If you're a mathematician, you probably have bigger problems to solve than a basic average or division calculation. You can be more productive and solve more complex problems if you don't have to worry about the smallest operations that a tool can solve for you. This is basically what an IDE does for a developer. ### How can an IDE help me save time? So how can an IDE help you save time? ### 1. Syntax highlighting Number one: syntax highlighting. If you speak English you may say "cheers" at a toast at a wedding, but you will say "salud" in Spanish or "prost" in German. Just like a regular language, every programming language has their own syntax — their own way of writing or communicating the same thing. Here are some examples of how you could create two variables — in this case `x` and `y` — and add them to create a new variable in different programming languages. As you can see, they have some things in common, like the equal or plus symbols, but they also have some differences, like the end of the line or the variable declarations. When you're using an IDE that knows the programming language's syntax that you're using, the IDE is able to recognize what a symbol or a keyword means, and then it shows your code with certain colors or format in order to make it a bit more readable. ### 2. Text autocompletion Number two: text autocompletion. You know the thing Google does when you start writing some text for a new search and it wants to complete or give suggestions as to what you might be searching for? Well, the same thing happens with an IDE. Since it already knows the syntax of the language you're programming in, it can give you suggestions of what you want to write next. This helps you to be faster because you don't have to write everything yourself. You can just start writing something and the autocompletion will give you a list of possible choices. That's pretty cool, right? ### 3. Refactoring options Number three: refactoring options. Whoa, wait — what does refactoring mean? Don't worry, it means that you can restructure your existing code or project's resources without affecting any behavior. For example, renaming a file, changing files to different folders, or even renaming a variable. Okay, why would I want to do that if it already works as it is? Well, sometimes the code can get too complex to understand, even by the developer who wrote it, and there are some changes that can be done to improve the code's readability. For example, you can create a function that will add two numbers and return the result, but you don't want to think of descriptive names for the function or the two variables, so you just name them `x`, `a`, and `b`. That's not too readable, right? If I were to rename the function or the variables to something more descriptive, instead of manually renaming the `x` in the definition of the function and then once more in the function call, I can just select the `x`, right-click on it, and select Refactor → Rename. After writing the new name for my function, I can see that all the references were correctly updated. This can also be done for the `a` and `b` variables, that can have a more descriptive name like `firstNumber` and, let's say, `secondNumber`. Yeah, way better — now I can clearly see what the `addNumbers` function is doing. ### 4. Importing libraries Number four: importing libraries. There are some functions that someone already created that are available for you to reuse in your project — they're called libraries. There's absolutely no need for you to reinvent the wheel in your code if you can just import a library and take advantage of it. There are times when you remember a library's function but you don't remember the name of the library. So instead of taking time off your day to Google the library, you can just write the function that you wanted to use and let the IDE handle the import for you. Are you convinced now? Yes, no, maybe? Okay, let me give you the last reason why you need and want an IDE. ### 5. Build, compile or run Number five: build, compile, or run your project. Once you finish coding your project or feature, you'll want to do a quick test to verify that it's working as you expected. How are you going to run your code to check that? Well, if you have Java code, these are the commands that you have to run to start running your application. That's a lot, eh? No, scratch that — let's go to an IDE and run it. That's it, beautiful. Let's see it again. Amazing. Pretty sure you want an IDE now, huh? You just have to click on the green arrow and click on run, and that's it. ### Recap Okay, so to recap, an IDE saves you so much time and so many headaches because of these five reasons: 1. Syntax highlighting, that helps you to be able to read the code and not get lost in between hundreds of lines of code. The IDE uses different colors or formatting for the code to make it more readable. 2. Text autocompletion, just like the Google search bar does but for your code. If you forget a keyword, the IDE will give you the best suggestion of what you were looking for. 3. Refactoring options, like renaming a file, a function, or a variable, or moving files from one folder to another. This way you don't have to manually update every single reference in the whole project. 4. Importing libraries, in case you remember the code but not the name of the library. Most likely the IDE already knows what function you're referring to — it just has to import that library to your project and you're all set. 5. Build, compile, or run your project just by clicking one button. There's no need to memorize all those commands and run them one by one from the console or terminal. There are not only five benefits that an IDE can give you — there are a lot more that I love: debugging, testing, managing your dependencies or environments, using version control systems. If you're a new developer, though, these five ways to save time are the most important ones that you need to know. ### Ending This video's written version can be found under prostdev.com/blog, under the same name: "5 reasons why you need an IDE and how it can save you so much time." You can create a new account there and give a like to this blog post to be able to save it in your profile — that way you can have this information handy when you need it. So what's your favorite IDE? Why don't you leave me a comment to let me know which ones you love, or you can let me know if you need any suggestions. Subscribe to this channel for more technology content, and that's it for today. Thank you so much for watching. This is Alex — I hope you have a nice day today. See ya! --- ## 3 ways to concatenate objects in DataWeave 2.0 Watch: https://www.youtube.com/watch?v=4OGSdpmtR-s | Page: https://prostdev.com/video/3-ways-to-concatenate-objects-in-dataweave-2-0 | Series: Other MuleSoft videos ### Intro Hi there, I'm Alex Martinez — you can call me Alex. In today's video we'll see three different ways to concatenate objects in DataWeave 2.0. I'll also explain what I mean by object concatenation, so don't worry about that. You may be familiar with other languages' concatenation functions, especially when using two strings, for example. Well, this can also be achieved in DataWeave with the `++` (plus plus) function, like this. Now, how can we concatenate objects? Let's find out. ### What is object concatenation? When I say this, I mean the action of combining two different objects and creating one single object containing all the fields (or keys) and values of those objects. For example, we have these two JSON objects and we want to achieve this in the end: one single object that contains everything that the other two objects had. This is object concatenation. If we translate this example to DataWeave, it looks like this. Here we have our two objects and the output object that we want as a result. So let's talk about the three options that we have to achieve this. ### Option 1: Plus Plus The most commonly used way to achieve object concatenation in DataWeave is the `++` (plus plus) function. The code would look something like this. You can see why this is one of the favorites: it's easy to read and understand, and it's very intuitive. You just have to say object one plus object two. There is another syntax to do the same thing, still using the `++` function. In this syntax you are writing the function first and then positioning the first and second arguments inside parentheses, separated by a comma. Either way is correct — it just depends on what you're used to, or what the standard practice is in your project. Both the parentheses syntax and `++` do the same thing. ### Option 2: Object destructor The second way we can achieve object concatenation is using an object destructor, or the curly brackets and parentheses: `{( )}`. This syntax can be cleaner to see in code, but it can also be a bit confusing for new developers if they're not familiar with the use of the object destructor. I'm sure you're wondering how this works if you haven't seen this syntax before. Well, when there are parentheses inside curly brackets, the parentheses become object destructors. The object in question is separated into keys and values, and these are then added to the new object that's being created by the outer curly brackets. Let me do that explanation step by step. - **Step one:** the parentheses destroy object one and object two into their key-value pairs. This basically means: get rid of the outer curly brackets that are surrounding the two objects. Now we only have key-value pairs that don't belong to any object. Of course, we wouldn't be able to see that in code, because you can't have key-value pairs that don't belong to any object. - **Step two:** create a new object that contains all these key-value pairs. This is why you need to have those outer curly brackets — they will put together the key-value pairs that the parentheses broke down. I bet now that you understand how it works, you want to change all your old transformations from using `++` to using object destructors. At least, that was my first reaction, to be honest. ### Option 3: Object destructor with an array The third and final way we can achieve object concatenation is using an object destructor with an array. Wait, say that again. In case you're not familiar with DataWeave syntax, you can create an array of any type of data using square brackets. For example, to create an array of numbers you can write it like this. You can also create an array of strings — boom. An array of objects — boom. An array of arrays. I guess you follow me. Why would I use arrays if I can just use parentheses for my object destructors? Okay, just hear me out. Isn't it a bit annoying to have to add parentheses for each of the objects? Sometimes I forget to add them and end up with syntax errors, because you can't just do this — you can't just create outer parentheses for all of the objects. You have to surround each object in parentheses. So it turns out that the parenthesis destructors not only destroy an object into key-value pairs, they also destroy arrays containing objects. What?! You can do something like this. See, now you can list your objects inside an array using square brackets, and then use the object destructor — I mean the curly brackets and the parentheses — on that array: `{([ ])}`. Yes, this is the most complex syntax out of the three that we saw today. It can be quite confusing for new developers, and you may have to write this explanation out in your internal project's documentation if you don't want to keep repeating the same thing every time you get a new resource. Lucky for you, you can send them the link to this video. ### Recap Okay, to recap: there are three options to concatenate objects in DataWeave 2.0. - You can use the most popular choice by Mule developers, the `++` (plus plus) function. Remember that any of these syntaxes are correct. - You can opt for a cleaner but less known option, using the object destructor `{( )}`. - Or the most complex one — but without having to surround each object in parentheses — the object destructor with an array `{([ ])}`. Do you know of any more ways to achieve the same object concatenation output in DataWeave? Leave me a comment and let me know — I would love to hear more suggestions or questions about this video. ### Ending This video's written version can be found under prostdev.com/blog with the name "Combining objects: concatenation in DataWeave 2.0". You can create a new account there and give a like to this blog post to be able to save it in your profile, that way you can have this information handy when you need it. This is all for this video. Thank you so much for watching. Subscribe to this channel for more technology content. This is Alex — thank you and have a great day. See you on the next video! ---