Job Flows And Connections
This tutorial is all about how to make sure your information is taking the proper paths in your jobs. In it, we’ll cover a bit of how tMap outputs work, as well as some of the component triggers that can happen in a job. These and other component connections will go a long way in assuring your job performs exactly as expected, and allow you to see how to be sure you capture any information that may seep through the cracks.
Create a new job and title it “JobFlow_1”. This will be the first of two jobs that show how information travels through rows.
We’ll need two separate tRowGenerator components to feed into a tMap, the first should be set up to only feed in the value “41”, as follows.
Be sure that this only sends one value through, not looping over multiple times.
The second row generator will be your lookup. Two columns will be in its output, one for check, the other with some assorted values. Our tmap will filter out any combinations that don’t match input from row1.
For the next step, you’ll need to open the tMap component and set up an Inner Join. But we actually created our Main and Lookup incorrectly. The join should be on the row that is currently our Lookup. Right click the row connection on the canvas and set row2 to Main. Row1 will automatically switch to Lookup.
Drag the “check” columns from row2 over the check column in row1, linking the two columns, so the join will compare their values. Then open the “tMap Settings” menu for row1 by clicking the wrench icon in its header.
Change the “Join Model” to “Inner Join”
Next we will set up the output. Create two output tables and then drag both columns from row2 into both sets of output data. Then click the wrench icon for the second output row and set “Catch lookup inner join reject” to true. Now, when there is data in row2 that doesnt match the data in row1, it will be filtered to this output.
Add two tLogRows and create output connections for both of them.
Try running the program and see how the data moves.
You may want to change the output mode of both the logrow component to a Table.
Then, you may easily see which output comes from which output.
Now let’s create another job, and title it “JobFlow_2”
This job will show how different Triggers work.
Create four tJava components and place them as follows, connecting them with the listed items under the “Trigger” connections.
Insert the following code into each component…
Then click on the “If” connection and insert the following:
Try running the code. The output should be as follows:
Both “Hello!” and “OK!” appeared, so we know that both tJava_1 and tJava_2 ran. If we change the string “check” in tJava_1 to meet the If condition, we can change the output.
When this runs now, the If condition is met, and so its output appears too.
We can assume that Error will be triggered if our component receives poor information.
This covers the basics of job flow design, but there is more to learn.
There are other connection types that we haven’t covered, so be sure to read up on them in the official Talend documentation.