Time for action—capturing errors while calculating the age of a film

In this tutorial you will use the output of the denormalizing process from the previous chapter. You will calculate the age of the films and classify them according to their age.

  1. Get the file with the films. You can take the transformation that denormalized the data and generate the file with a Text file output step, or you can take a sample file from the Packt website.
  2. Create a new transformation and read the file with a Text file input step.
  3. Do a preview of the data. You will see the following:
    Time for action—capturing errors while calculating the age of a film
  4. After the Text file input step, add a Get System Info step.
  5. Edit the step, add a new field named today, and choose Today 00:00:00 as its value.
  6. Add a JavaScript step.
  7. Edit the step and type the following piece of code:
    var diff;
film_date = str2date('01/01/' + Year, 'dd/MM/yyyy'),
diff = dateDiff(film_date,today,"y");
  8. Click on Get variables to add diff as a new field.
  9. Add a Number range step, edit it, and fill its window as follows:
    Time for action—capturing errors while calculating the age of a film
  10. With a Sort rows step, sort the data by diff.
  11. Finally, add a Group by step and double-click to edit it.
  12. As group field put age_of_film. In the Aggregates grid create a field named number_of_films to hold the number of films with that age. Put film as the Subject and select Number of values (N) as the Type.
  13. Add a Dummy step at the end and do a preview. You will be surprised by an error like this:
    Time for action—capturing errors while calculating the age of a film
  14. Look at the logging window. It looks like this:
    Time for action—capturing errors while calculating the age of a film
  15. Now drag Write to log step to the canvas from the Utility category.
  16. Create a hop from the JavaScript step to this new step.
  17. Select the JavaScript step, right-click it to bring up a contextual menu, and select Define error handling....
  18. The error handling settings window appears. Fill it like shown:
    Time for action—capturing errors while calculating the age of a film
  19. Click on OK.
  20. Save the transformation and do a new preview on the Dummy step. You will see this:
    Time for action—capturing errors while calculating the age of a film
  21. The logging window will show you this:
    ... - Bad rows.0 -
... - Bad rows.0 - ------------> Linenr 1-------------------------
... - Bad rows.0 - null
... - Bad rows.0 -
... - Bad rows.0 - Javascript error:
... - Bad rows.0 - Could not apply the given format dd/MM/yyyy on the string for 01/01/null : Format.parseObject(String) failed (script#4)
... - Bad rows.0 -
... - Bad rows.0 - --> 4:0
... - Bad rows.0 - SCR-001
... - Bad rows.0 -
... - Bad rows.0 - ====================
... - Bad rows.0 -
... - Bad rows.0 - ------------> Linenr 2-------------------------
... - Bad rows.0 - null
... - Bad rows.0 -
... - Bad rows.0 - Javascript error:
... - Bad rows.0 - Could not apply the given format dd/MM/yyyy on the string for 01/01/null : Format.parseObject(String) failed (script#4)
... - Bad rows.0 -
... - Bad rows.0 - --> 4:0
...

    The date was cut from the log for clarity of the log messages.

  22. Now do a preview on the Write to log step. This is what you see:
    Time for action—capturing errors while calculating the age of a film

What just happened?

You created a transformation to read a list of films and group them according to their age, that is, how old the movie is. You were surprised by an unexpected error caused by the rows in which the year was undefined. Then you implemented error handling to capture that error and to avoid the abortion of the transformation. With the treatment of the error, you split the stream in two:

  • The rows that caused the error went to a new stream that wrote to the log information about the error
  • The rows that passed the JavaScript step without problem went through the main path

Using PDI error handling functionality

With the error handling functionality, you can capture errors that otherwise would cause the transformation to halt. Instead of aborting, the rows that cause the errors are sent to a different stream for further treatment.

You don't need to implement error handling in every step, but in those where it's more likely to have errors when running the transformation. A typical situation where you should consider handling errors is in a JavaScript step. A code that works perfectly when designing might fail while executing against real data, where the most common errors are related to data type conversions or indexes out of range. Another common use of error handling is when working with databases (you will see more on this later in the book).

To configure the error handling, you have to right-click the step and select Define Error handling.

Note

Note that not all steps support error handling. The Define Error handling option is available only when clicking on steps that support it.

After opening the settings window, you have to fill it just as you did in the tutorial. You have to specify the target step for the bad rows along with the name of the extra fields being added, as part of the treatment of errors:

Field

Description

Nr of errors fieldname

Name for the field that will have the number of errors

Error fields fieldname

Name for the field that will have the name of the field(s) that caused the errors

Error codes fieldname

Name for the field that will have the error code

Error descriptions fieldname

Name for the field that will have the error description

The first two are trivial. The last two deserve an explanation. The values for the error code and description fields are the same as those you see in the Logging tab when you don't trap the error. In the tutorial there was a JavaScript error with code SCR-001 and description JavaScript error: Could not apply the given format.... You saw this code as well as its description in the Logging tab when you didn't trap the error and the transformation crashed, and in the preview you made at the end of the error stream. This particular error was a JavaScript one, but the kind of error you get depends always on the kind of step where it occurs.

You are not forced to fill all the textboxes in the error setting window. Only the fields for which you provide a name will be added to the dataset. By doing a preview on the target step, you can see the extra fields that were added.

Aborting a transformation

You can handle the errors by detecting and sending bad rows to an extra stream. But when the errors are too many or when the errors are severe, the best option is to cancel the whole transformation. Let's see how to force the abortion of a transformation in such a situation.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.14.142.194