The calculation engine

The calculation engine, as its name refers to, is responsible for all calculations performed in the Qlik Sense applications. 

Qlik Sense is an on-demand calculation engine, meaning everything is calculated on the fly. This means that charts will get calculated or recalculated whenever the user makes selections. 

One thing that is important to clarify is the fact that charts are not always recalculated. If a selection throws, as a result, a calculation that had previously been calculated, the Qlik Sense engine will get the calculated result from the cache instead of having to calculate everything again. This, as you can image, improves response speed enormously. When selections are repetitive, you will see an improvement in response time.

I would like to clarify that even though Qlik Sense stores the results based on selections, nothing is pre-calculated.

In the previous section, we explained how the logical inference engine applies the State Vectors and flags the records in the data table(s) associated with the current selections. The calculation engine is then responsible for taking those flagged records and performing the following actions, in order:

  • Look for values (find values and/or combinations)
  • Aggregate

First, the engine must find the rows in the data tables to aggregate. This is done by searching for the binary value 1 in the Output State Vector in the data table. 

Once all rows are located, the calculation engine needs to go to the symbol table to look for the real value in the field.

A single field expression, such as sum([Gross Amount]), is a simple aggregation, as it only uses one field of one data table.

The next image shows a table with the expression field:

For a simple aggregation like Sum([Gross Amount]), only one data table is required (the one with [Gross Amount] in it), so this step is purely limited to looking up the values of [Amount] in its symbol table.

For an aggregation with multiple fields in it, such as Sum([Gross Amount]*[VAT])the calculation engine must first get all combinations of [Gross Amount] and [VAT].

The following image highlights the two fields used in the expression:

If both fields are in the same data table, this is straightforward, as all combinations already exist in the table. So again, it is just a lookup function, albeit with two fields instead of one. However, if [Gross Amount] and [VAT] reside in two different tables, the engine first has to build a temporary lookup table with all combinations of the two fields, as well as looking up the values.

This lookup and find combinations step is single-threaded (per object), so may be a bottleneck to performance.

As a rule of thumb, arranged fields are frequently aggregated together into the same table if the business logic allows. By doing this, performance will improve, as no temporary lookup table is needed for calculations.

Secondly, once the calculation engine has looked up the real values and created the temporary lookup table if needed, it performs the final aggregations over the records in the table.

The aggregation step is multi-threaded, and thus highly performant.

Here, we enumerate different examples of work done by the calculation engine:

  • Populating a filter pane of raw values. This does not require any aggregation, as it only uses the symbol table. Only the lookup step is performed.
  • Populating a chart. This requires both the lookup and the aggregation steps. The aggregation is made separately for every dimensional value.
  • Calculating an Aggr() expression. This also requires both the lookup and the aggregation steps. The aggregation is made separately for every dimensional value.
  • Using GetSelectedCount(<field>) in a Text and Image object. This does not require any aggregation, as it only uses the symbol table. Again, only the lookup step is performed.

Understanding the Qlik Sense engine is essential if you want to have a working application. Everything that has been explained so far will hopefully help you create an optimized data model, which will, in return, make your application more performant.

Besides understanding the internal logic of the QIX engine, there are many useful techniques to create a professional data model. Keep on reading!

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.138.37.20