Time for action—generating files with top scores

In this tutorial, you will read the examination global file and generate four files—one for each particular skill. The files will contain the top 10 scores for each skill. The scores will not be the original, but converted to a scale with values in the range 0-5.

Note

As you must be already quite confident with PDI, some explanations in this section will not have the full details. On the contrary, the general explanation will be focused on the structure of the jobs and transformations.

  1. Create a new transformation and save it in the transformations folder under the name top_scores.ktr.
  2. Use a Text file input step to read the global examination file generated in the previous tutorial.
  3. After the Text file input step, add the following steps and link them in the same order:
    • A Select values step to remove the unused fields— file_processed and process_date.
    • A Split Fields to split the name of the students in two—name and last name.
    • A Formula step to convert name and last name to uppercase.
    • With the same Formula step, change the scale of the scores. Replace each skill field writing, reading, speaking, and listening with the same value divided by 20—for example, [writing]/20. You have already done this in Chapter 3.
  4. Do a preview on completion of the final step to check that you are doing well. You should see this:
    Time for action—generating files with top scores
  5. After the last Formula step, add and link in this order the following steps:
    • A Sort rows step to order the rows in descending order by the writing field.
    • A JavaScript step to filter the first 10 rows. Remember that you learned to do this in the chapter devoted to JavaScript. You do it by typing the following piece of code:
      trans_Status = CONTINUE_TRANSFORMATION;
if (getProcessCount('r')>10) trans_Status = SKIP_TRANSFORMATION;
    • An Add sequence step to add a field named seq_w. Leave the defaults so that the field contains the values 1, 2, 3 ...
    • A Select values step to rename the field seq_w as position and the field writing as score. Specify this change in the Select & Alter tab, and check the option Include unspecified fields, ordered.
    • A Text file output step to generate a file named writing_top10.txt at the location specified by the ${LABSOUTPUT} variable. In the Fields tab, put the following fields— position, student_code, student_name, student_lastname, and score.
  6. Save the transformation, as you've added a lot of steps and don't want to lose your work.
  7. Repeat step number 5, but this time sort by the reading field, rename the sequence seq_r as position and the field reading as score, and send the data to the reading_top10.txt file.

    Tip

    To save time, you can copy all those steps, paste them, and do the proper adjustments.

  8. Repeat the same procedure for the speaking field and the listening field.
  9. This is how the transformation looks like:
    Time for action—generating files with top scores
  10. Save the transformation.
  11. Run the transformation. Four files should have been generated. All the files should look similar. Let's check the writing_top10.txt file (the names and values may vary depending on the examination files that you have appended to the global file):
    Time for action—generating files with top scores

What just happened?

You read the big file with examination results and generated four files with information about the top scores—one file for each skill.

Beyond having used the Add sequences step for the first time, there was nothing new. However, there are several improvements you can do to this transformation. The next tutorials are meant to teach you some tricks.

Pop quiz—using the Add Sequence step

In the previous tutorial, you used different names for the sequences and then you renamed all of them to position. Which of the following options gives you the same results you got in the tutorial?

a. Using position as the name of the sequence in all Add sequence steps

b. Joining the four streams with a single Add sequence step and then splitting the stream back into four streams by using the Distribute method you learned in Chapter 4

c. Joining the four streams with a single Add sequence step and then splitting the stream back into four streams by using a Switch case step that distributes the rows properly

d. All of them

e. None of them

Reusing part of your transformations

As you noticed, the sequence of steps used to get the ranks are almost identical for the four skills. You could have avoided copying and pasting or doing the same work several times by moving those steps to a subtransformation. Let's do it.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.21.246.223