© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2023
P. A. CarterPro SQL Server 2022 Administrationhttps://doi.org/10.1007/978-1-4842-8864-1_22

22. Query Store

Peter A. Carter1  
(1)
SOUTHAMPTON, UK
 

The Query Store captures the history of queries, their plans, and statistics. It allows DBAs to easily view the plans that were used by queries and troubleshoot performance issues. In this chapter, we will discuss how Query Store can be enabled and configured. We will also examine how the Query Store can be used to diagnose and resolve performance issues.

Enabling and Configuring Query Store

Query Store is enabled and configured at the database level, so the first thing we will do is create a database, called Chapter22. This can be achieved using the script in Listing 22-1.
--Create the database
CREATE DATABASE Chapter22 ;
GO
USE Chapter22
GO
--Create and populate numbers table
DECLARE @Numbers TABLE
(
        Number        INT
)
;WITH CTE(Number)
AS
(
        SELECT 1 Number
        UNION ALL
        SELECT Number + 1
        FROM CTE
        WHERE Number < 100
)
INSERT INTO @Numbers
SELECT Number FROM CTE;
--Create and populate name pieces
DECLARE @Names TABLE
(
        FirstName        VARCHAR(30),
        LastName        VARCHAR(30)
);
INSERT INTO @Names
VALUES('Peter', 'Carter'),
                 ('Michael', 'Smith'),
                 ('Danielle', 'Mead'),
                 ('Reuben', 'Roberts'),
                 ('Iris', 'Jones'),
                 ('Sylvia', 'Davies'),
                 ('Finola', 'Wright'),
                 ('Edward', 'James'),
                 ('Marie', 'Andrews'),
                 ('Jennifer', 'Abraham');
--Create and populate Customers table
CREATE TABLE dbo.Customers
(
      CustomerID        INT            NOT NULL     IDENTITY   PRIMARY KEY,
      FirstName         VARCHAR(30)    NOT NULL,
      LastName          VARCHAR(30)    NOT NULL,
      BillingAddressID  INT            NOT NULL,
      DeliveryAddressID INT            NOT NULL,
      CreditLimit       MONEY          NOT NULL,
      Balance           MONEY          NOT NULL
);
SELECT * INTO #Customers
FROM
         (SELECT
                 (SELECT TOP 1 FirstName FROM @Names ORDER BY NEWID()) FirstName,
                 (SELECT TOP 1 LastName FROM @Names ORDER BY NEWID()) LastName,
                 (SELECT TOP 1 Number FROM @Numbers ORDER BY NEWID()) BillingAddressID,
                 (SELECT TOP 1 Number FROM @Numbers ORDER BY NEWID()) DeliveryAddressID,
                 (SELECT TOP 1 CAST(RAND() * Number AS INT) * 10000
                 FROM @Numbers
                 ORDER BY NEWID()) CreditLimit,
                 (SELECT TOP 1 CAST(RAND() * Number AS INT) * 9000
                 FROM @Numbers
                 ORDER BY NEWID()) Balance
        FROM @Numbers a
        CROSS JOIN @Numbers b
) a;
INSERT INTO dbo.Customers
SELECT * FROM #Customers;
GO
CREATE INDEX idx_LastName ON dbo.Customers(LastName)
GO
Listing 22-1

Create the Chapter22 Database

In SQL Server 2022, Query Store is enabled, by default, for all new databases that are created. If you need to turn Query Store on, however, either because you have turned it off or because you have restored a database from an older version of SQL Server, you can do so, using the command in Listing 22-2.
ALTER DATABASE Chapter22 SET QUERY_STORE = ON
Listing 22-2

Enable Query Store

As well as enabling Query Store using an ALTER DATABASE command, this method is also used to configure Query Store properties. Table 22-1 details the Query Store–related SET options that are available.
Table 22-1

Query Store SET Options

SET Option

Description

ON

Enables the Query Store for the specified database.

OFF

Disables the Query Store for the specified database.

CLEAR

Clears the contents of the Query Store.

OPERATION_MODE

Can be configured as READ_ONLY, or the default of READ_WRITE. When configured as READ_WRITE, data is written to the Query Store and can be read from the Query Store. When set to READ_ONLY, data can still be read from the Query Store, but no further data will be written to it. The Query Store will automatically switch to READ_ONLY, if it runs out of space for storing new data.

CLEANUP_POLICY

Accepts STALE_QUERY_THRESHOLD_DAYS and a number of days. This determines how many days data for each query should be retained.

DATA_FLUSH_INTERVAL_SECONDS

Query Store data is flushed to disk asynchronously. DATA_FLUSH_INTERVAL_SECONDS specifies how frequently data should be flushed.

MAX_STORAGE_SIZE_MB

Specifies the maximum amount of space that can be used by Query Store data. If this value overflows, the Query Store will switch to READ_ONLY operational mode.

INTERVAL_LENGTH_MINUTES

Specifies the time interval at which runtime execution statistics data is aggregated.

SIZE_BASED_CLEANUP_MODE

Specifies if an automatic cleanup is triggered, when the Query Store reaches 90% of its maximum size threshold. Can be set to AUTO or OFF. When set to AUTO, the oldest, least expensive queries will be removed, until the size drops to around 80% of the Query Store’s maximum size threshold.

QUERY_CAPTURE_MODE

Specifies which queries data should be captured for. Can be configured as ALL, NONE, AUTO, or CUSTOM. When set to ALL, statistics will be captured for all queries. When set to NONE, statistics will only be captured for queries which already exist in the Query Store. When configured as AUTO, SQL Server will capture statistics for the most expensive queries, based on execution count and resource utilization. When configured as CUSTOM, then you will have more granular control over which queries have statistics captured. When CUSTOM is specified, QUERY_CAPTURE_POLICY should also be specified.

MAX_PLANS_PER_QUERY

Specifies the maximum number of plans that will be captured for each query, defaulting to 200.

WAIT_STATS_CAPTURE_MODE

Specifies if wait statistics should be captured for queries. This is, of course, very useful, but comes at the expense of extra disk space consumed.

QUERY_CAPTURE_POLICY

Can be used when QUERY_CAPTURE_MODE is set to CUSTOM. Please see Table 22-2 for details of the available options that can be passed.

Table 22-2 details the options that can be configured for QUERY_CAPTURE_POLICY.
Table 22-2

QUERY_CAPTURE_POLICY Options

Option

Description

STALE_CAPTURE_POLICY_THRESHOLD

Specifies an evaluation period, which is used by the other parameters, to determine if a query should have its statistics captured. Can be specified in hours, or days

EXECUTION_COUNT

Specifies the minimum number of executions within the evaluation period that should occur, for a query to be captured

TOTAL_COMPILE_CPU_TIME_MS

Specifies the total amount of CPU Compilation Time within the evaluation period that should occur, for a query to be captured

TOTAL_EXECUTION_CPU_TIME_MS

Specifies the total amount of CPU Execution Time within the evaluation period that should occur, for a query to be captured

Let’s use the command in Listing 22-3 to configure the Query Store for the Chapter22 database. Here, we are configuring the Query Store to retain data for 30 days, persist data to disk every five minutes, capture wait statistics for queries, and run an automatic cleanup when 90% of a 2GB threshold is reached.
ALTER DATABASE Chapter22
SET QUERY_STORE = ON (
      OPERATION_MODE = READ_WRITE,
      CLEANUP_POLICY = ( STALE_QUERY_THRESHOLD_DAYS = 30 ),
      DATA_FLUSH_INTERVAL_SECONDS = 300,
      MAX_STORAGE_SIZE_MB = 2048,
      SIZE_BASED_CLEANUP_MODE = AUTO,
      WAIT_STATS_CAPTURE_MODE = ON
) ;
Listing 22-3

Configure the Query Store

The Query Store can also be configured using SSMS. To do this, drill through Databases in Object Explorer and select Properties from the context menu of the database, for which you wish to configure the Query Store. In the Properties page, you can select the Query Store tab, which is illustrated in Figure 22-1.
Figure 22-1

Query Store Properties tab

Working with Query Store Data

Before we can see how to work with Query Store data, we will first need to run some queries, so that the Query Store has some data to collect. The script in Listing 22-4 will run a number of queries against the Chapter22 database.
SELECT
      CustomerID
      , FirstName
      , LastName
      , CreditLimit
      , Balance
FROM dbo.Customers
WHERE LastName = 'Carter'
GO
SELECT TOP (1000) [CustomerID]
      ,[FirstName]
      ,[LastName]
      ,[BillingAddressID]
      ,[DeliveryAddressID]
      ,[CreditLimit]
      ,[Balance]
FROM [Chapter22].[dbo].[Customers]
GO
SELECT *
FROM dbo.Customers A
INNER JOIN dbo.Customers B
      ON a.CustomerID = b.CustomerID
UNION
SELECT *
FROM dbo.Customers A
INNER JOIN dbo.Customers B
      ON a.CustomerID = b.CustomerID
GO
Listing 22-4

Query the Chapter22 Database

Query Store Reports

Now we have run some queries, the Query Store will have captured data about those queries. We can view this data through a series of SSMS reports. The standard reports are
  • View Regressed Queries

  • View Overall Resource Consumption

  • View Queries with Forced Plans

  • View Queries with High Variation

  • Query Wait Statistics

  • View Tracked Queries

These reports can be accessed by drilling through Databases ➤ [Database Name] and then selecting the appropriate report from the context menu of the Query Store node. In this chapter, we will examine some of the reports that I find most useful, but I encourage you to experiment with all of them.

Tip

Because of the nature of the reports, when you first set up Query Store, there will be very little data. The data points will grow as time moves forward after Query Store is enabled.

The View Overall Resource Consumption report is illustrated in Figure 22-2. This report consists of four charts, representing execution duration, execution count, CPU Time, and Logical Reads, aggregated over time intervals.
Figure 22-2

Overall Resource Consumption

Hovering your mouse over a bar will cause a box with further details to be displayed, as you can see in Figure 22-3.
Figure 22-3

Viewing additional details

Clicking the Standard Grid button will replace the bar charts with a grid, with one row per time interval, showing aggregated runtime statistics. This grid can be copied to Excel for further analysis. The grid is illustrated in Figure 22-4.
Figure 22-4

Standard grid view

Tip

You will notice that the execution count is far higher than the number of queries we have run. This is because SQL Server runs queries in the background, as part of many different processes, including queries that are run when you navigate SSMS.

Figure 22-5 illustrates the Top Resource Consumers report. The top-left corner of this report shows a bar chart, representing the most resource-intensive queries. The Metric drop-down defaults to Duration, but many different metrics can be selected, including degree of parallelism (DOP), CPU Time, Row Count, Wait Time even CLR Time, or Log Memory Used, to name but a few.
Figure 22-5

Top Resource Consumers

The top-right corner of the report shows a scatter chart, detailing resource utilization and execution time of each plan. This is helpful when a query has been executed using multiple plans, as you can easily assess the most efficient plans for a query.

The bottom half of the plan shows the accrual execution plan that was used. As with any other graphical query plan representation in SSMS, hovering over a physical operator will show cost information for that operator.

The front page of the Query Wait Statistics report can be seen in Figure 22-6. The bar chart at the top of the screen shows a summary of resources that the highest cumulative wait times have been against.
Figure 22-6

Query Wait Statistics report

Each bar in the bar chart is clickable. For example, if I were to click the CPU bar, then the drill-through report in Figure 22-7 would be displayed.
Figure 22-7

CPU drill-through report

By default, the top half of the grid will show the most expensive queries (labelled by Plan ID), based on Total Wait Time, but as you can see, the Based on drop-down list can be changed to base the results on average, min, max wait time, or the standard deviation.

The lower half of the report shows the graphical representation of the Query Plan that has been selected in the top half of the report.

Query Store T-SQL Objects

As well as graphical reports, SQL Server also exposes a number of catalog views which can be used to retrieve Query Store data. Table 22-3 details the catalog views which are exposed.
Table 22-3

Query Store Catalog Views

Catalog View

Description

Query_store_plan

Stores information about every query plan that is associated with a query.

Query_store_query

Stores query information and aggregated runtime statistics.

Query_store_wait_stats

Stores wait statistic details. Please see Table 22-4 for mappings of each wait type.

Query_store_query_text

Stores the SQL handle and SQL text of each query.

Query_store_runtime_stats

Stores the runtime statistics for each query.

Table 22-4 details how each wait type category maps to underlying wait types.
Table 22-4

Wait Statistics Mappings

Wait Category

Wait Types

CPU

SOS_SCHEDULER_YIELD

Worker Thread

THREADPOOL

Lock

LCK_M_*

Latch

LATCH_*

Buffer Latch

PAGELATCH_*

Buffer IO

PAGEIOLATCH_*

SQL CLR

CLR*

SQLCLR*

Mirroring

DBMIRROR*

Transaction

ACT*

DTC*

TRAN_MARKLATCH_*

MSQL_XACT_*

TRANSACTION_MUTEX

Idle

SLEEP_*

LAZYWRITER_SLEEP

SQLTRACE_BUFFER_FLUSH

SQLTRACE_INCREMENTAL_FLUSH_SLEEP

SQLTRACE_WAIT_ENTRIES

FT_IFTS_SCHEDULER_IDLE_WAIT

XE_DISPATCHER_WAIT

REQUEST_FOR_DEADLOCK_SEARCH

LOGMGR_QUEUE

ONDEMAND_TASK_QUEUE

CHECKPOINT_QUEUE

XE_TIMER_EVENT

Preemptive (Preemptive IO)

PREEMPTIVE_*

Service Broker

BROKER_*

Transaction Log IO

LOGMGR

LOGBUFFER

LOGMGR_RESERVE_APPEND

LOGMGR_FLUSH

LOGMGR_PMM_LOG

CHKPT

WRITELOG

Network IO

ASYNC_NETWORK_IO

NET_WAITFOR_PACKET

PROXY_NETWORK_IO

EXTERNAL_SCRIPT_NETWORK_IOF

Parallelism

CXPACKET

EXCHANGE

Memory

RESOURCE_SEMAPHORE, CMEMTHREAD

CMEMPARTITIONED, EE_PMOLOCK

MEMORY_ALLOCATION_EXT

RESERVED_MEMORY_ALLOCATION_EXT

MEMORY_GRANT_UPDATE

User Wait

WAITFOR

WAIT_FOR_RESULTS

BROKER_RECEIVE_WAITFOR

Tracing

TRACEWRITE

SQLTRACE_LOCK

SQLTRACE_FILE_BUFFER

SQLTRACE_FILE_WRITE_IO_COMPLETION

SQLTRACE_FILE_READ_IO_COMPLETION

SQLTRACE_PENDING_BUFFER_WRITERS, SQLTRACE_SHUTDOWN, QUERY_TRACEOUT

TRACE_EVTNOTIFF

Full Text Search

FT_RESTART_CRAWL

FULLTEXT GATHERER

MSSEARCH

FT_METADATA_MUTEX

FT_IFTSHC_MUTEX

FT_IFTSISM_MUTEX

FT_IFTS_RWLOCK

FT_COMPROWSET_RWLOCK

FT_MASTER_MERGE

FT_PROPERTYLIST_CACHE

FT_MASTER_MERGE_COORDINATOR

PWAIT_RESOURCE_SEMAPHORE_FT_PARALLEL_QUERY_SYNC

Other IO

ASYNC_IO_COMPLETION, IO_COMPLETION

BACKUPIO, WRITE_COMPLETION

IO_QUEUE_LIMIT

IO_RETRY

Replication

SE_REPL_*

REPL_*, HADR_*

PWAIT_HADR_*, REPLICA_WRITES

FCB_REPLICA_WRITE, FCB_REPLICA_READ

PWAIT_HADRSIM

Log Rate Governor

LOG_RATE_GOVERNOR

POOL_LOG_RATE_GOVERNOR

HADR_THROTTLE_LOG_RATE_GOVERNOR

INSTANCE_LOG_RATE_GOVERNOR

*Denotes a wildcard, where all waits that match the portion to the left of the * are included

Tip

Idle and User Wait are unlike other Wait Type Categories, in the respect that they are not waiting for a resource, they are waiting for work to do.

For example, the query in Listing 22-5 will return the three highest wait categories, by total wait time, excluding idle and user waits, which are “healthy” waits.
SELECT TOP 3
        wait_category_desc
      , SUM(total_query_wait_time_ms) TotalWaitTime
FROM sys.query_store_wait_stats
WHERE wait_category_desc NOT IN ('Idle', 'User Wait')
GROUP BY wait_category_desc
ORDER BY SUM(total_query_wait_time_ms) DESC
Listing 22-5

Return Top 3 Wait Types

Resolving Issues with Query Store

Figure 22-8 illustrates the Regressed Query report. This report displays queries, the impact of which, in terms of duration, or execution count have increased over time. These queries are illustrated with a bar chart in the top left of the report. The top right of the report illustrates the plans that have been used to execute the query, with their associated duration. At the bottom of the report, we can see the actual execution plan for the current selection of query, and plan in the top half of the screen.
Figure 22-8

Regressed Query report

In this particular example, we can see that the query has run using two distinct plans and that the plan with an ID of 168 took significantly less time to execute than the plan with an ID of 110. Therefore, with the more efficient plan selected, we can use the Force Plan button, in the middle of the report, to ensure that the more efficient plan is always used to execute this query. Once we have used this feature, the Unforce Plan button will become active, allowing us to undo our action, if required.

Tip

The Force and Unforce buttons are available on all Query Store reports, where a specific plan is selectable.

As well as using the GUI to impart performance improvements on queries, there are also a number of system stored procedures exposed. These procedures are detailed in Table 22-5 and can be used to manage both plans and the Query Store itself.
Table 22-5

Query Store Stored Procedures

Procedure

Description

Sp_quey_store_flush_db

Flushes Query Store data to disk

Sp_query_store_force_plan

Forces a query to use a specific plan

Sp_query_store_unforce_plan

Removes a forced plan from a query

Sp_query_store_reset_exec_stats

Clears the runtime statistics from the Query Store, for a specific query

Sp_query_store_remove_plan

Removes a specific plan from the Query Store

Sp_query_store_remove_query

Removes a query and all associated information from the Query Store

For example, to Unforce the plan that we forced upon our query in the previous example, we could use the query in Listing 22-6.
EXEC sp_query_store_unforce_plan @query_id=109, @plan_id=168
Listing 22-6

Unforce a Query Plan

Query Store Hints

Unlike many other languages, T-SQL is a descriptive and declarative language. This means, when writing a query, that instead of telling the database engine exactly what to do, you describe the results that you want to receive and the query optimizer figures out the best way to deliver those results. The query optimizer is incredibly good at deciding the best way to do something, but if it does make a mistake, it can have a negative impact on query performance.

This is where query hints come in. A query hint allows you to force the optimizer down a specific route. For example, you might tell it that it should recompile the query every time it runs. Alternatively, you may tell it to always use a certain type of physical join operation, or to always use a specific MAXDOP.

The challenge with query hints is that you are not always able to modify the code, which means that you cannot add a query hint to the query. This may be because the code belongs to a third party and support agreements do not allow for code changes, or it could be because there is an urgent performance issue and code release procedures are simply too cumbersome to resolve the issue in a timely manner.

To address this, SQL Server introduced plan guides, where you could specify a statement and one or more query hints that should always be applied, whenever that query runs. You could even use plan freezing, where you specified the USE PLAN hint, and then pass a specific query plan that you wanted to be used. The problem with plan guides is that they are far from user friendly and generally required a highly experienced DBA to create them. SQL Server 2022 resolves this issue with query store hints, which give DBAs the opportunity to specify query hints in a much more friendly way.

If you are trying to address a performance issue with a query and need to specify a query hint, then you can use query store functionality to add a query hint. For example, imagine that your performance troubleshooting has uncovered that a Nested Loops join operator is not the ideal choice for the plan illustrated in Figure 22-9.
Figure 22-9

Suboptimal hash match

Tip

I have used a simple example to illustrate the concept. In reality, this specific plan would be highly unlikely to need a query hint, as Intelligent Query Processing would alter the physical join operator dynamically, based on the inputs.

Because the optimizer is so good, I always advise working with the optimizer, rather than against it. Therefore, in this case, instead of using a query hint to force the optimizer to use a Merge Join, which is the natural inclination of many DBAs, I would advise using query hints to tell the optimizer to use either a Merge Join operation or a Hash Match operation. This translates to “Nested Loops is wrong. Use anything except this.” The optimizer can then select the best option from its remaining choices.

We will then need to retrieve the Query ID from the query store. We can do this using the reports discussed earlier in this chapter, or we can use the query store catalog views to retrieve it. The query in Listing 22-7 will retrieve a list of queries and Query IDs for tracked queries that have been run against the dbo.Customers table. The query also returns useful compilation statistics.
SELECT
        q.query_id
      , t.query_sql_text
      , q.avg_compile_duration
      , q.avg_optimize_duration
      , q.last_execution_timeFROM sys.query_store_query_text T
INNER JOIN sys.query_store_query q ON
    t.query_text_id = q.query_text_id
WHERE query_sql_text like '%dbo.Customers%'
Listing 22-7

Retrieve Query IDs

Once we have the Query ID, we can use the sp_query_store_set_hints stored procedure, as shown in Listing 22-8, where the Query ID of the query with the suboptimal plan is 86.
EXEC sys.sp_query_store_set_hints
      @query_id= 86
    , @query_hints = N'OPTION(HASH JOIN, MERGE JOIN)'
Listing 22-8

Apply a Query Store Hint

A nice safety feature of query store hints is that it will ignore any hint that stops it from running, as opposed to blocking the query. Therefore, you cannot inadvertently break a query. The query store hints will also override any query hints or plan guides that conflict with it.

You can retrieve a list of query store hints, including any failures to apply hints, from the query_store_query_hints catalog view, as demonstrated in Listing 22-9.
SELECT
        query_id
      , query_hint_text
      , last_query_hint_failure_reason_desc
FROM sys.query_store_query_hints
Listing 22-9

Retrieve a List of Query Store Hints

The command in Listing 22-10 demonstrates how to remove the query store hint associated with our problematic query earlier.
EXEC sys.sp_query_store_clear_hints @query_id = 86
Listing 22-10

Remove Query Store Hints

Summary

The Query Store is a very powerful feature of SQL Server, which allows DBAs to monitor the performance of specific queries and their plans. Because the data is flushed to disk, this means that the data persists, even after an instance is restarted.

Six standard reports are available, which allow DBAs to view information such as regressed queries, the most resource-intensive queries, or even wait statistics, by query.

As well as viewing problematic queries, DBAs can also force the most performant plan to be used. This can be achieved through the interactive reports, or via automation, using T-SQL. The automated approach would examine the Query Store data through the catalog views and then pass the relevant data to the parameters of system stored procedures.

SQL Server gives DBAs granular control over which queries are captured by Query Store. This is implemented with custom capture policies, where details such as execution count and execution time can be used to capture only the queries with the highest performance impact. This not only saves disk space but also avoids “noise” when working with the captured data.

In SQL Server 2022, query store has been enhanced to include query store hints, which make it easy and safe for a DBA to use query hints to resolve performance issues, without the need to edit the code. SQL Server 2022 also introduces the ability to use query store on Secondary Availability Replicas, but at the time of writing, this feature was in preview, and not supported for production workloads. Further discussion of Availability Groups can be found in Chapter 15.

Tip

SQL Server 2022 introduces query store enhancements, which allow for feedback mechanisms which can automatically optimize query performance. These feedback mechanisms are discussed in various chapters throughout this book.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.218.151.81