Using tREST and JSON components to access API data

In this post, we are going to walk you through the basic steps required to get started on accessing API information with the tREST component. This allows for simple extraction of data using any API interface. In this specific example, we will pull JSON data about a Spotify track search.

Step 1: Open Talend

Open Talend and create or open an existing project

Step 2: Create a new job

• Right click on Job Designs in the Repository window and select “Create job”
• Name the job “trest_extract”

Step 3: Design your job

Now, create a tREST component on your canvas.

Step 4: Find your API information

View the API information you’d like to use, in this case it is the Spotify Web API from mashape.com

From here we can see the URL and header information to access the API method we’d like to use

Alter the URL to the artist you’d like to see track information about (in this instance, we chose Lorde)

 Step 5: Enter in API information

Click the green “plus” icon to add Header information, and copy in the header pair from the API method

 Step 6: Create JSON components

Now that we can access the API, we need to save the JSON data it will retrieve. For this use a tFileOutputJSON component (if we were working with Talend Big Data, tExtractJSONFields would be a better choice)

 

Step 7: Save schema information

We want to save this schema, so in the component view, click the “…” button next to “Edit schema”

From here, click the “View Schema” option, and then save the schema to be used later.

Step 8: Load JSON data from file

To read this file, we now want a tFileInputJSON component to read the file we saved.

In the component tab, switch the schema type to “Repository”, unclick the “Read By XPath” checkbox, and edit the filename to match your tFileOutputJSON component.

Select a schema by clicking the “…” button next to the blank schema name to select the schema to use. Then navigate to your saved metadata information and select it.

 

After doing this, the mapping area should be filled in, though with blank JSONPath queries.

Since we don’t want to pull the Error Code data, you can also edit the metadata you just selected, and unclick the “Used Column” checkbox for “ERROR_CODE”.

 Step 9: Choose JSON Path

Now we must execute a JSON query to retrieve the information we want, but for the purposes of this demo, we’ll just retrieve the whole document. From the website http://goessner.net/articles/JsonPath/ we can find how to work the JSONPath query, and find the way to use these queries. A description table follows:

We can use the last query to retrieve all elements, so we’ll copy that into our component.

  Step 9: Run Job and View Output

Lastly, we want to insert a tlogrow component and be sure our two subjobs are connected with both main rows and a OnSubjobOk trigger to determine the order in which they run.

Switch to the run tab and click the “Run” button to see the fruits of your labor!

 

Your output should return the JSON document from the API:

 

  1. Sanjay Reply

    hi ,
    I am new in elt please help me, how to pass json file in trestclient component.

    Thanks in advanced

  2. Amit Sonavane Reply

    Hi,

    I have followed your example as it is for my work and instead of Spotify Web API from mashape.com url I have used stripe API URL “https://api.stripe.com/v1/charges?limit=3”. I have following question –
    1. Connection from tREST_1 to tFileInputJSON_1 I am giving Filename as “C:/Talend-Studio-20141207_1530-V5.6.1/workspace2/out.txt” — I am getting following error as

    Exception in component tFileInputJSON_1
    java.io.FileNotFoundException: C:\Talend-Studio-20141207_1530-V5.6.1\workspace2\out.txt (The system cannot find the file specified)

    And when I create empty out.txt manually in workspcace it gives me following error as —

    Exception in component tFileInputJSON_1
    java.lang.RuntimeException: fail to parse the json file : C:/Talend-Studio-20141207_1530-V5.6.1/workspace2/out.txt

    And when I add the json data in out.txt it works fine but when i put the limit of 3 it gives me 5 records as the out.txt contains 5 records.

    I have to use data from API not from tect file.
    Could you please help me on this?
    Your help highly appreciated!!!
    -Amit

    • Snifira Reply

      Hi Amit,
      Well I had the same issue as you.
      Actually the file “out.txt” should be “out.json” (a little mistake in the screenshot :p), and if you read the explanation before the screenshot, it’s written “edit the filename to MATCH your tFileOutputJSON component”.
      The 2nd subjob (tFileOutputJSON and tlogRow) reads what the 1st subjob (tRest and tFileOutputJSON) had written in the “out.json” file, and prints its content as log.

      N.B: the link between TRest and the tFileInputJSON components is a “OnSubjobOk”, which means that the 2nd subjob won’t be executed unless the first one has been successfully executed.

      Cheers

  3. swati jaiswal Reply

    Hi,

    Can you give me one post example for sending data by using tRest component?

    I set http body with json data but having an error of 400 (Bad Request).

    JSON data is –

    “{
    ‘user_id’: ‘swati1036′,
    ’email’: ‘w@serenity.io’,
    ‘name’: ‘Serenity Washburne’,
    ‘signed_up_at’: 1392731325,
    ‘last_seen_ip’ : ‘1.2.3.6’
    }”

    So please suggest me.

  4. Travis Reply

    Im a newbie so I haven’t use the tREST at all, but how did you find the header info to put into the component? Was that some sort of browser extension to see the values?

  5. VKarumanchi Reply

    Hi,
    At the time we are running the job we need to close the file( the file whic is refering should not be opend while teland job is running)

  6. Prakash Rajendran Reply

    Could you please give an example to use tREST component to upload a file?

Leave a Reply

*

captcha *