Talend Tutorials

Job Flows And Connections

Job Flows And Connections

| Matt Irvin

This tutorial is all about how to make sure your information is taking the proper paths in your jobs. In it, we’ll cover a bit of how tMap outputs work, as well as some of the component triggers that can happen in a job. These and other component connections will go a long way in assuring your job performs exactly as expected, and allow you to see how to be sure you capture any information that may seep through the cracks.

Step 1: Create Filter Job

Create a new job and title it “JobFlow_1”. This will be the first of two jobs that show how information travels through rows.

Step 2: Set up Input

We’ll need two separate tRowGenerator components to feed into a tMap, the first should be set up to only feed in the value “41”, as follows.

image 1

Be sure that this only sends one value through, not looping over multiple times.

image 2

The second row generator will be your lookup. Two columns will be in its output, one for check, the other with some assorted values. Our tmap will filter out any combinations that don’t match input from row1.

image 3

image 4

For the next step, you’ll need to open the tMap component and set up an Inner Join. But we actually created our Main and Lookup incorrectly. The join should be on the row that is currently our Lookup. Right click the row connection on the canvas and set row2 to Main. Row1 will automatically switch to Lookup.

Step 3: Set up the tMap

Drag the “check” columns from row2 over the check column in row1, linking the two columns, so the join will compare their values. Then open the “tMap Settings” menu for row1 by clicking the wrench icon in its header.

Change the “Join Model” to “Inner Join”

image 5

Next we will set up the output. Create two output tables and then drag both columns from row2 into both sets of output data. Then click the wrench icon for the second output row and set “Catch lookup inner join reject” to true. Now, when there is data in row2 that doesnt match the data in row1, it will be filtered to this output.

image 6

Step 4: Watch the Flow

Add two tLogRows and create output connections for both of them.

Try running the program and see how the data moves.

image 7

You may want to change the output mode of both the logrow component to a Table.

image 8

Then, you may easily see which output comes from which output.

image 9

Step 5: More Control – Triggers

Now let’s create another job, and title it “JobFlow_2”

This job will show how different Triggers work.

Create four tJava components and place them as follows, connecting them with the listed items under the “Trigger” connections.

image 10

Step 6: Set up the Code:

Insert the following code into each component…

image 11

image 12

image 13

image 14

Then click on the “If” connection and insert the following:

image 15

Step 7: What Happens

Try running the code. The output should be as follows:

image 16

Both “Hello!” and “OK!” appeared, so we know that both tJava_1 and tJava_2 ran. If we change the string “check” in tJava_1 to meet the If condition, we can change the output.

image 17

When this runs now, the If condition is met, and so its output appears too.

image 18

We can assume that Error will be triggered if our component receives poor information.

Step 8: A Little Extra

This covers the basics of job flow design, but there is more to learn.

There are other connection types that we haven’t covered, so be sure to read up on them in the official Talend documentation.

Looking For More Talend Help?
Contact Us Today