Query folding

Query folding is one of the most powerful and important capabilities of the M language as it translates M expressions into SQL statements that can be executed by the source system. With query folding, M serves as an abstraction layer to implement both common and complex data cleansing and transformation operations while still leveraging source system resources. When implementing any remaining logic or data transformations via M functions, a top priority of the dataset designer is to ensure that these operations are folded to the data source.

In the following M query, a Table.RemoveColumns() M function is applied against the SQL view for the Internet Sales fact table to exclude three columns that are not needed for the dataset:

Power Query Editor: View Native Query

The additional step is translated to a SQL query that simply doesn't select the three columns. The specific SQL statement passed to the source system can be accessed by right-clicking the final step in the Query Settings pane and selecting View Native Query. If the View Native Query option is grayed out, this indicates that the specific step or transformation is executed with local resources.

Selecting one of the APPLIED STEPS in the Query Settings pane displays a preview of the results of the query of the given step. Particularly for queries with several steps, the ability to quickly walk through the transformations or view the query results at any given step is very helpful in analyzing and debugging M queries. Note that the names of the variables used in the M query will be reflected in the APPLIED STEPS pane, further underscoring the importance of applying intuitive variable names in M queries.

Query folding is limited by the data source of the M expression with relational databases, such as SQL Server and Oracle, supporting the most query folding. Alternatively, no query folding is possible when an Excel workbook or a text file is the data source of a query. The M queries against these file sources will use local M engine resources and thus the volume of data imported as well as the complexity of the query should be limited. Other sources, such as SharePoint lists, Active Directory, and Exchange, support some level of query folding, though significantly less than relational databases.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.190.217.253