Talend Tutorials

Unix Deployment And Scheduling Of Jobs

Unix Deployment And Scheduling Of Jobs

In this tutorial, we’ll go outside of the studio environment to show you how to set up your unix machine to run tasks outside of Talend Open Studio. We’ll create a simple task and then run it from a command line, then set a recurring task to do it daily.

Step 1: Create Job In Talend

To start off, simply add a tJava component into the canvas. This will be the only component in our job.

image 1

Secondly set up a local context variable in the context tab named “input”. Switch to “values as a table” and make the Default setting “standard”.

image 2

Lastly insert the following code into the tJava component. With the display code to check the value of context parameter input.

image 3

Save when everything is done.

Step 2: Export to Folder

Right click the job name in your repository and select “Build Job”.

image 4

Choose the location you want a zip file to be sent to. If you don’t have to move it anywhere else, select the “Extract the zip file” option. This will save the item and allow it to be runnable in the location specified. The “All” option downloads both a batch file for Windows and Shell file for Unix.

image 5

Step 3: Run on Command Line

In this demo we’ll be working with unix. Remember .bat is for windows and .sh is for unix.

Navigate to the location of the sh file in a command prompt and enter the command “bash <jobName>_run.sh”. The job shown here is simpledeploy, thus simpledeploy_run.sh. Note the text “Standard was called!”.

To change the context variable at run time, add it after the original command so:

“bash <jobName>_run.sh –context_param input=<not standard>”. Any value that is not “standard” will result in the text “Manual was called!”

image 6

Step 4: Schedule a Recurring Task

Many people will want to schedule their talend jobs as recurring tasks to run every day, week, etc. In unix, this can be done with a cron job.

To open up the list of cron jobs, type “crontab -e”.

Here you can schedule tasks on each line with the following format:

1 2 3 4 5 /path/to/command arg1 arg2

Each number means:

1: Minutes, 2: Hours, 3: Day, 4: Month, 5: Day of the Week

The following entry runs every day at 11:00 PM:

image 7

Save and quit to have this job run forever.

Looking For More Talend Resources and Help?
Ask An Expert