Most Talend estates we walk into now sit next to a dbt project. Talend lands the data; dbt models it. That split is the right one — we argued for it in ETL vs ELT in 2026 — but the seam between the two tools is where the 3 a.m. pages come from. Nobody owns it, so it gets wired with a sleep 600 and a hope.
This tutorial covers the handoff itself: how a Talend job tells dbt that raw data is ready, how dbt refuses to build on stale or partial loads, and how to orchestrate both without inventing a second scheduler.
The failure mode you are designing against
The naive setup runs Talend at 02:00 and dbt at 03:00 on separate cron entries. It works until one of these happens:
- The Talend job runs long. dbt builds on yesterday's data and every downstream dashboard is quietly wrong. No error, no alert.
- The Talend job fails halfway through a multi-table load. dbt builds on three of five tables. Referential joins silently drop rows.
- Someone reruns Talend at 09:00 to fix a source glitch. dbt does not rerun, so the fix never reaches the marts.
None of these produce a red light. That is what makes them expensive. The fix is to make the handoff an explicit, recorded event rather than a coincidence of clock times.
Pattern 1: a load-audit table Talend writes and dbt reads
The cheapest reliable contract is a table in the warehouse that Talend updates as the last step of each load. Everything else in this tutorial builds on it.
create table if not exists meta.load_audit (
dataset_name varchar(200) not null,
load_started_at timestamp_ntz not null,
load_ended_at timestamp_ntz,
rows_loaded number,
status varchar(20) not null, -- RUNNING | SUCCESS | FAILED
job_name varchar(200),
job_pid varchar(100),
primary key (dataset_name, load_started_at)
);
In the Talend job, wrap the load with two tDBRow components on the OnSubjobOk / OnSubjobError links:
At the start of the subjob, tDBRow inserts the RUNNING row:
insert into meta.load_audit
(dataset_name, load_started_at, status, job_name, job_pid)
values
('"+context.dataset_name+"', current_timestamp(), 'RUNNING',
'"+jobName+"', '"+pid+"');
jobName and pid are Talend's built-in job variables, so the audit row points straight back at the run in the Talend Management Console log. On OnSubjobOk, a second tDBRow closes it out using the row count from the output component's NB_LINE_INSERTED global variable:
update meta.load_audit
set status = 'SUCCESS',
load_ended_at = current_timestamp(),
rows_loaded = "+((Integer)globalMap.get("tDBOutput_1_NB_LINE_INSERTED"))+"
where dataset_name = '"+context.dataset_name+"'
and job_pid = '"+pid+"'
and status = 'RUNNING';
On OnSubjobError, the same update sets status = 'FAILED'. Do not skip the failure branch — an audit table that only records successes cannot distinguish "failed" from "never started," and the whole point is that dbt can tell the difference.
Pattern 2: make dbt refuse to build on a stale load
Now give dbt eyes. Expose the audit table as a source and attach freshness plus a data test.
# models/staging/_sources.yml
version: 2
sources:
- name: raw
database: analytics
schema: raw
tables:
- name: orders
loaded_at_field: _loaded_at
freshness:
warn_after: {count: 6, period: hour}
error_after: {count: 26, period: hour}
- name: meta
schema: meta
tables:
- name: load_audit
Run dbt source freshness before dbt build in the same command sequence. It exits non-zero on error_after, which stops the run before a single model is rebuilt.
Freshness alone does not catch a half-finished load, so add a singular test that reads the audit table:
-- tests/assert_raw_loads_completed.sql
with expected as (
select column1 as dataset_name
from values ('orders'), ('order_lines'), ('customers'), ('inventory'), ('fx_rates')
),
latest as (
select dataset_name,
status,
row_number() over (
partition by dataset_name order by load_started_at desc
) as rn
from {{ source('meta', 'load_audit') }}
)
select e.dataset_name,
coalesce(l.status, 'MISSING') as status
from expected e
left join latest l
on l.dataset_name = e.dataset_name
and l.rn = 1
where coalesce(l.status, 'MISSING') <> 'SUCCESS'
Any dataset whose most recent load is RUNNING, FAILED, or absent returns a row, the test fails, and dbt build stops. This is the check that pays for itself: it turns "silently wrong dashboards" into "a red build with the offending dataset named."
For per-table gating rather than all-or-nothing, put the same logic in a macro and call it from a where clause or a pre-hook on the specific staging models that depend on that dataset.
Pattern 3: Talend triggers dbt, not the clock
With the contract in place, remove the second cron entry. Three ways to trigger, in order of how much infrastructure they need.
a) Talend calls dbt Cloud's API. A tRESTClient (POST, Bearer token from your secret store, never a context default committed to Git) against the run-job endpoint:
POST https://<region>.dbt.com/api/v2/accounts/{account_id}/jobs/{job_id}/run/
{ "cause": "Triggered by Talend job ORDERS_DAILY (pid <pid>)" }
Put the token in a tContextLoad read from a secrets file or environment variable, keep the job id in context per environment, and log the returned run id into load_audit so you can trace warehouse state back to a dbt run. Follow with a polling loop (tLoop + tRESTClient on runs/{id}/) if downstream Talend subjobs need to wait for the models; skip the poll if nothing downstream cares.
b) Talend shells out to dbt Core. A tSystem calling dbt build --select source:raw+ --target prod on a machine that has the project checked out and profiles configured. Simple, but you have now coupled dbt's Python environment to your Talend runtime host, and tSystem gives you an exit code and stdout rather than real observability. Acceptable for one job on one server; it does not scale past that.
c) An orchestrator owns both. Airflow, Dagster, or Prefect runs the Talend job (CLI, TMC API, or a Kubernetes job — see our containerizing Talend jobs tutorial) as one task and dbt build as the next, with a real dependency edge:
run_talend = KubernetesPodOperator(
task_id="talend_orders_daily",
image="registry.internal/talend/orders-daily:2026.1",
...
)
dbt_build = BashOperator(
task_id="dbt_build_marts",
bash_command="dbt source freshness && dbt build --select staging.orders+",
)
run_talend >> dbt_build
If you already run an orchestrator, use it. Option (a) is for shops where Talend Management Console is the only scheduler and adding Airflow is a six-month procurement conversation.
Whichever you pick, keep the audit-table test in place. The trigger says "Talend thinks it finished"; the test says "the warehouse agrees."
Where each transformation belongs
Once both tools are wired together, the arguments start about who owns what logic. Our working rule:
| Work | Owner | Why |
|---|---|---|
| Source connectivity, auth, pagination, SFTP, mainframe copybooks | Talend | dbt cannot reach outside the warehouse |
| Parsing fixed-width, multi-schema, XML, PDF, proprietary formats | Talend | Procedural and Java-library territory |
| Landing raw, append-only, typed as loosely as tolerable | Talend | Keep the load boring and rerunnable |
| Deduplication, SCD Type 2, star schemas, metrics | dbt | Set-based SQL, reviewable in a PR |
| Business rules an analyst should be able to read | dbt | Ownership follows readability |
| Row-by-row calls to external APIs mid-pipeline | Talend | Warehouses are bad at side effects |
| Reverse ETL back into Salesforce, NetSuite, an ERP | Talend | It is an integration problem again |
The anti-pattern is a tMap with forty expressions implementing business logic that finance needs to audit. Move that into a dbt model where it lives in version control and reads as SQL. The mirror anti-pattern is a dbt model with a Python UDF scraping an API because nobody wanted to open Talend Studio.
Migration order when you already have both
Teams that end up here usually have transformation logic scattered across both tools. Do not "rewrite everything in dbt." Sequence it:
- Instrument first. Add
load_auditand the freshness/completeness tests to the existing pipeline. You get the reliability win in a week without moving any logic. - Draw the current line. For each Talend job, record what it extracts and what it transforms. The jobs that only land data are already correct and need no work.
- Move set-based work only. Joins, aggregations, dedup, and dimension builds go to dbt. Leave connectivity and parsing alone.
- Reconcile before cutover. Run the Talend and dbt versions in parallel and diff the outputs — a full-outer join on the business key with a column-by-column comparison, run for a full period including month-end. Our testing Talend jobs tutorial covers the assertion patterns; the same approach works for cross-tool reconciliation.
- Retire deliberately. Delete the Talend transformation subjob only when the diff has been clean for a full cycle, and keep the job in Git so you can read the old logic when someone asks why a number changed.
Operational details people forget
One source of truth for environments. Talend contexts and dbt targets both encode dev/test/prod. If they can disagree, one day Talend will load prod while dbt builds dev. Derive both from the same environment variable in your deployment pipeline.
Timezones in the audit table. Use UTC everywhere (timestamp_ntz in UTC, or timestamp_tz). A load audit that mixes engine-local and warehouse-local time makes freshness thresholds meaningless twice a year.
Reruns. Talend reruns should insert a new audit row, not update the old one. dbt's test looks at the latest row per dataset, so a successful rerun automatically unblocks the build, and you keep the history of what failed.
Permissions. The Talend warehouse role needs write on raw and meta. The dbt role needs read on both and write on its own schemas. Do not share one superuser role between them; when something writes to the wrong place you will want to know which tool did it.
Alerting on the seam. Route the dbt test failure to the same channel as Talend job failures. If dbt's alerts go to the analytics team and Talend's go to integration, the handoff failure gets discussed twice and fixed never.
The short version
Make the handoff explicit: Talend writes an audit row, dbt tests it before building, and one dependency edge — API call or orchestrator — replaces the clock gap. That is roughly two days of work and it removes an entire class of silent data errors.
If you are wiring Talend into a dbt-centric stack, or unpicking transformation logic that grew across both, our Modern Data Stack & ELT services team does exactly this work. Get in touch and we will look at your actual job inventory.