+1 (726) 227-3027

Talend ESB Has No Cloud Successor: Exit Patterns for Data Services and Routes

Every Talend estate we inherit has a quiet corner nobody has looked at in three years: the ESB side. A handful of Data Services exposing tRESTRequest endpoints, two or three Camel Routes shuttling files and JMS messages, all deployed as OSGi bundles into a Talend Runtime container on a VM that someone patched manually in 2021.

It worked. That is exactly why it is now a problem. The data integration half of the platform has a forward path — Qlik Talend Cloud, remote engines, the migration patterns we have written about at length. The ESB half does not. There is no "lift my Routes into the cloud" button, and there is not going to be one. Like MDM before it, ESB is an exit project, not an upgrade project.

This is the inventory-and-exit playbook we run on those engagements.

Step 1: Inventory by workload type, not by project

Open the Studio workspace and sort every ESB artifact into four buckets. Do this before anyone proposes a target platform, because each bucket has a different, cheap answer — and mixing them is what turns a six-week job into a six-month one.

  1. Synchronous request/response services. tRESTRequest / tESBProviderRequest jobs that a caller invokes and waits on. Usually thin: validate input, hit a database or a SOAP backend, return JSON.
  2. Mediation Routes. Real Camel: cFile, cJMS, cMessagingEndpoint, content-based routing, splitters, dead-letter queues.
  3. Scheduled or triggered batch dressed up as a service. A job that happens to have an HTTP trigger because that was the easiest way to let another team kick it off. These are not services at all.
  4. Registry and gateway concerns. Service Locator, Service Activity Monitoring, the API gateway bits, WS-Security policies.

In our experience the split is roughly 60% bucket 3, 25% bucket 1, 15% bucket 2. That is good news: most of what looks like an ESB migration is really "give this batch job a better trigger".

Capture for each artifact: caller list, call volume per day, p95 latency expectation, authentication mechanism, and whether any caller is outside your organization. The last field decides how much ceremony you need.

Step 2: Re-trigger bucket 3 and delete the endpoint

Anything that is a batch job with an HTTP trigger should stop being an HTTP service immediately. You have two clean replacements:

  • Qlik Talend Cloud task triggers. The Management API exposes run endpoints, so a caller does a POST against the platform, not against your container. We covered the mechanics in automating Qlik Talend Cloud with the public API.
  • An orchestrator queue. If the caller is another internal system, a row in a request table or a message on a queue, polled by a scheduled job, removes the runtime dependency entirely and gives you a natural audit trail.

Either way the OSGi bundle goes away and the job becomes an ordinary Standard job. Do this bucket first: it is the fastest reduction in surface area, and it shrinks the scary number in the steering-committee slide from "40 ESB artifacts" to "16".

Step 3: Rehost bucket 1 as plain services

For genuine synchronous services, resist the urge to shop for a new integration suite. A tRESTRequest job that validates a payload, runs a query, and returns JSON is 150 lines of application code in any modern stack. The realistic options, in the order we usually recommend them:

Spring Boot behind your existing API gateway. Most shops already have a Java team, a build pipeline, and a gateway (Apigee, Kong, AWS API Gateway, Azure APIM). A controller plus a JdbcTemplate query replaces the job, and the gateway handles TLS, rate limiting, and OAuth 2.0 token validation — concerns Talend's own OSGi container was never a comfortable place for. If you are on the calling side of OAuth today, our notes on OAuth 2.0 from Talend explain the same token flows from the other direction.

Apache Camel standalone (Camel on Spring Boot or Camel K). This is the honest answer for bucket 2 as well, and it deserves emphasis: Talend's Routes are Camel. The cJMS component is a thin wrapper over camel-jms. Your routing logic is portable in a way your Studio project is not. Export the generated route, read the blueprint.xml or the generated Java DSL, and you will usually find a route you can move into a Spring Boot project with import changes and a dependency list. That is a genuinely rare gift in a migration and most teams do not realize they have it.

Hand it to the database or the warehouse. A surprising number of "services" are a parameterized SELECT returning fewer than a hundred rows. A read-only view plus a warehouse REST endpoint, or a PostgREST-style layer, retires the artifact outright.

What we would not do: rebuild these as ELT jobs in the data platform. Request/response latency and batch throughput are different engineering problems, and the ETL-vs-ELT boundary does not help you answer a 200 ms SLA.

Step 4: Extract the container configuration before it dies

The technical risk in this project is not the code. It is that the Talend Runtime container holds configuration nobody wrote down: etc/*.cfg property files, JMS broker credentials, keystore paths, Service Locator registrations, and OSGi feature installs applied by hand over years.

Before you decommission anything, snapshot it:

# from the Talend Runtime / Karaf install root
tar czf runtime-config-$(date +%F).tgz etc/ deploy/ lib/ext/

./bin/client -f - <<'EOF'
bundle:list -t 0
feature:list -i
config:list
EOF

Put that tarball and the command output in the migration repo. Then diff config:list against what your new deployment actually sets. The gaps in that diff are your production incidents — every one of them, every time. Keystore aliases and JMS redelivery settings are the usual suspects.

Step 5: Decide what happens to WS-Security and SOAP

If you have external callers using WS-Security with signed and encrypted SOAP envelopes, that is the one piece that does not port cheaply. Options, cheapest first:

  • Terminate the SOAP contract at the gateway. Many gateways can accept the legacy envelope, verify the signature, and forward plain JSON to your new service. The caller sees no change; you never write WSS code again.
  • Keep CXF, drop Talend. Talend's service stack is Apache CXF. A Spring Boot app with cxf-spring-boot-starter-jaxws, your existing WSDL, and your existing policy files is a supported, maintained path, and those .wsdl artifacts move unchanged.
  • Negotiate the contract away. Sometimes the external partner has offered a REST API for five years and nobody on your side had budget to switch. Ask before you engineer.

Step 6: Cut over one service at a time, with both paths live

Run the old and new endpoints in parallel behind the gateway and shift traffic per-caller, not all at once. For read-only services, mirror a percentage of live traffic to the new implementation and compare responses — payload-level diffing catches the things nobody specified, especially date formatting, numeric scale, and null-versus-absent JSON keys.

Keep the same structured-logging and alerting discipline you use on the data side, so an ESB cutover shows up in the same dashboards as everything else (error handling and observability patterns). And treat each migrated service as needing a regression suite before it is called done — the approach in testing Talend jobs transfers directly, with recorded request/response pairs as the fixtures.

The realistic timeline

For a typical estate — around 40 artifacts, a couple of JMS integrations, one external SOAP contract — we plan on:

  • 1 week inventory and classification,
  • 2 weeks retiring bucket 3 (the biggest win, the least risk),
  • 4 to 6 weeks rehosting the real services and routes,
  • 2 weeks parallel running and decommissioning the container.

The trap is starting with the hardest service because it is the most visible. Start with the batch jobs wearing service costumes. They shrink the problem by half and buy you the credibility to spend real time on the WS-Security one.

If you are looking at a Talend Runtime container that nobody wants to own and no cloud target to move it to, get in touch — the inventory in Step 1 takes about a week, and it is the difference between an exit plan and an open-ended rewrite.