{"id":86758,"date":"2020-03-25T14:44:19","date_gmt":"2020-03-25T14:44:19","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=86758"},"modified":"2026-04-14T13:17:52","modified_gmt":"2026-04-14T13:17:52","slug":"reduce-cpu-of-large-analytic-queries-without-changing-code","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/t-sql-programming-sql-server\/reduce-cpu-of-large-analytic-queries-without-changing-code\/","title":{"rendered":"SQL Server Batch Mode on Rowstore: Reduce CPU on Analytic Queries"},"content":{"rendered":"<p><strong>SQL Server 2019 introduced Batch Mode on Rowstore &#8211; a query processing improvement that applies batch mode execution to tables without a columnstore index. Previously, batch mode (which processes data in chunks of ~900 rows rather than one row at a time) was only available on tables with a columnstore index. SQL Server 2019 extends this to heap and B-tree tables for qualifying analytic aggregate queries, reducing CPU usage by 20\u201330% on typical workloads without any query rewrites. The feature is enabled automatically by the SQL Server 2019 query optimizer and available in Enterprise and Developer editions. This article demonstrates the improvement with a benchmarked test against the WideWorldImportersDW database and explains how to disable it selectively for queries that don&#8217;t benefit.<\/strong><\/p>\n<p>When Microsoft came out with columnstore in SQL Server 2012, they introduced a new way to process data called <em>Batch Mode<\/em>. Batch mode processes a group of rows together as a batch, instead of processing the data row by row. By processing data in batches, SQL Server uses less CPU than row by row processing. To take advantage of batch mode, a query had to reference a table that contained a column store index. If your query only involved tables that contain data in row stores, then your query would not use batch mode. That has now changed. With the introduction of version 15.x of SQL Server, aka SQL Server 2019, Microsoft introduced a new feature call <em>Batch Mode on Rowstore. <\/em><\/p>\n<p>Batch Mode on Rowstore is one of many new features that was introduced in the Azure SQL Database and SQL Server 2019 to help speed up rowstore queries that don\u2019t involve a column store. The new Batch Mode on Rowstore feature can improve performance of large analytic queries that scan many rows, where these queries aggregate, sort or group selected rows. Microsoft included this new batch mode feature in the <a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/relational-databases\/performance\/intelligent-query-processing?view=sql-server-ver15\"><em>Intelligent Query Processing <\/em>(IQP)<\/a>. See Figure 1 for a diagram from Microsoft\u2019s documentation that shows all the IQP features introduced in Azure SQL Database and SQL Server 2019. It also shows the features that originally were part of Adaptive Query Processing included in the older generation of Azure SQL Database and SQL Server 2017.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1020\" height=\"495\" class=\"wp-image-86759\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-46.png\" \/><\/p>\n<p class=\"caption\">Figure 1: Intelligent Query Processing<\/p>\n<p>Batch Mode on Rowstore can help speed up your big data analytic queries but might not kick in for smaller OLTP queries (more on this later). Batch mode has been around for a while and supports columnstore operators, but it wasn\u2019t until SQL Server version 15.x that batch mode worked on Rowstores without performing a <a href=\"https:\/\/sqlworkbooks.com\/2017\/05\/batch-mode-hacks-for-rowstore-queries-in-sql-server\/\">hack<\/a>. Before seeing the new Batch Mode on Rowstore feature in action, let me first explain how batch mode processing works.<\/p>\n<h2>How Batch Mode Processing Works<\/h2>\n<p>When the database engine processes a transact SQL statement, the underlying data is processed by one or more operators. These operators can process the data using two different modes: Row or Batch. At a high level, row mode can be thought of as processing rows of data, one row at a time. Whereas, batch mode processes multiple rows of data together in a batch. The processing of batches of rows at a time versus row by row can reduce CPU usage.<\/p>\n<p>When batch mode is used for rowstore data, the rows of data are scanned and loaded into a vector storage structure, known as a batch. Each batch is a 64K internal storage structure. This storage structure can contain between 64 and 900 rows of data, depending on the number of columns involved in the query. Each column used by the query is stored in a continuous column vector of fixed size elements, where the <em>qualifying rows vector<\/em> indicates which rows are still logically part of the batch (see Figure 2 which came from a <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2013\/06\/Apollo3-Sigmod-2013-final.pdf\">Microsoft Research paper<\/a>).<\/p>\n<p>Rows of data can be processed very efficiently when an operation uses batch mode, as compared to row mode processing. For instance, when a batch mode filter operation needs to qualify rows that meet a given column filter criteria, all that is needed is to scan the vector that contains the filtered column and mark the row appropriately in the qualifying rows vector, based on whether or not the column value meets the filter criteria.<\/p>\n<p>\u00a0<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"317\" height=\"296\" class=\"wp-image-86760\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-47.png\" \/><\/p>\n<p class=\"caption\">Figure 2: A row batch is stored column-wise and contains one vector for each column plus a bit vector indicating qualifying rows<\/p>\n<p>SQL Server executes fewer instructions per row when using batch mode over row mode. By reducing the number of instructions when using batch mode, queries typically use less CPU than row mode queries. Therefore, if a system is CPU bound, then batch mode might help reduce the environment\u2019s CPU footprint.<\/p>\n<p>In a given execution plan, SQL Server might use both batch and row mode operators, because not all operators can process data in batch mode. When mixed-mode operations are needed, SQL Server needs to transition between batch mode and row mode processing. This transition comes at a cost. Therefore, SQL Server tries to minimize the number of transitions to help optimize the processing of mixed-mode execution plans.<\/p>\n<p>For the engine to consider batch mode for a rowstore, the database compatibility level must be set to 150. With the compatibility level set to 150, the database engine performs a few heuristic checks to make sure the query qualifies to use batch mode. One of the checks is to make sure the rowstore contains a significate number of rows. Currently, it appears that the magic number seems to be 131,072. Dmitry Pilugin wrote an excellent <a href=\"http:\/\/www.queryprocessor.com\/batch-mode-on-row-store\/\">post<\/a> on this magic number. I also verified that this is still the magic number for the RTM release of SQL Server 2019. That means that batch mode doesn\u2019t kick in for smaller tables (less than 131,072 rows), even if the database is set to compatibility mode 150. Another heuristic check verifies that the rowstore is using either a b-tree or heap for its storage structure. Batch mode doesn\u2019t kick in if the table is an in-memory table. The cost of the plan is also considered. If the database optimizer finds a cheaper plan that doesn\u2019t use Batch Mode on Rowstore, then the cheaper plan is used.<\/p>\n<p>To see how this new batch mode feature works on a rowstore, I set up a test that ran a couple of different aggregate queries against the WideWorldImportersDW database.<\/p>\n<h2>Batch Mode on Rowstore In Action<\/h2>\n<p>This section demonstrates running a simple test aggregate query to summarize a couple of columns of a table that uses heap storage. The example runs the test aggregate query twice. The first execution uses compatibility level 140, so the query must use row mode operators to process the test query. The second execution runs under compatibility mode 150 to demonstrate how batch mode improves the query processing for the same test query.<\/p>\n<p>After running the test query, I\u2019ll explain how the graphical execution plans show the different operators used between the two test query executions. I\u2019ll also compare the CPU and Elapsed time used between the two queries to identify the performance improvement using batch mode processing versus row mode processing. Before showing my testing results, I\u2019ll first explain how I set up my testing environment.<\/p>\n<h3>Setting up Testing Environment<\/h3>\n<p>I used the WideWorldImportersDW database as a starting point for my test data. To follow along, you can download the database backup for this DB <a href=\"https:\/\/github.com\/Microsoft\/sql-server-samples\/releases\/download\/wide-world-importers-v1.0\/WideWorldImportersDW-Full.bak\">here<\/a>. I restored the database to an instance of SQL Server 2019 RTM running on my laptop. Since the <em>Fact.[Order] <\/em>table in this database isn\u2019t that big, I ran the code in Listing 1 to create a bigger fact table named <em>Fact.OrderBig. <\/em>The test query aggregates data using this newly created fact table.<\/p>\n<p class=\"caption\">Listing 1: Code to create the test table <em>Fact.OrderBig<\/em><\/p>\n<pre class=\"lang:tsql theme:ssms2012-simple-talk\">USE WideWorldImportersDW;\nGO\nCREATE TABLE Fact.[OrderBig](\n\t[Order Key] [bigint],\n\t[City Key] [int] NOT NULL,\n\t[Customer Key] [int] NOT NULL,\n\t[Stock Item Key] [int] NOT NULL,\n\t[Order Date Key] [date] NOT NULL,\n\t[Picked Date Key] [date] NULL,\n\t[Salesperson Key] [int] NOT NULL,\n\t[Picker Key] [int] NULL,\n\t[WWI Order ID] [int] NOT NULL,\n\t[WWI Backorder ID] [int] NULL,\n\t[Description] [nvarchar](100) NOT NULL,\n\t[Package] [nvarchar](50) NOT NULL,\n\t[Quantity] [int] NOT NULL,\n\t[Unit Price] [decimal](18, 2) NOT NULL,\n\t[Tax Rate] [decimal](18, 3) NOT NULL,\n\t[Total Excluding Tax] [decimal](18, 2) NOT NULL,\n\t[Tax Amount] [decimal](18, 2) NOT NULL,\n\t[Total Including Tax] [decimal](18, 2) NOT NULL,\n\t[Lineage Key] [int] NOT NULL);\nGO\nINSERT INTO Fact.OrderBig\n   SELECT * FROM Fact.[Order];\nGO 100<\/pre>\n<p>The code in Listing 1 created the <em>Fact.OrderBig <\/em>table that is 100 times the size of the original <em>Fact.[Order]<\/em> table with 23,141,200 rows.<\/p>\n<h3>Comparison Test Script<\/h3>\n<p>To do a comparison test between batch mode and row mode, I ran two different test queries found in Listing 2.<\/p>\n<p class=\"caption\">Listing 2: Test script<\/p>\n<pre class=\"lang:tsql theme:ssms2012-simple-talk\">USE WideWorldImportersDW;\nGO\n-- Turn on time statistics\nSET STATISTICS IO, TIME ON;\n-- Clean buffers so cold start performed \nDBCC DROPCLEANBUFFERS\nGO\n-- Prepare Database Compatibility level for Test #1 \nALTER DATABASE WideWorldImportersDW SET COMPATIBILITY_LEVEL = 140;\nGO\n-- Test #1\nSELECT [Customer Key], \n       SUM(Quantity) AS TotalQty,\n       AVG(Quantity) AS AvgQty, \n       AVG([Unit Price]) AS AvgUnitPrice\nFROM Fact.[OrderBig]\nWHERE [Customer Key] &gt; 10 and [Customer Key] &lt; 100\nGROUP BY [Customer Key]\nORDER BY [Customer Key];\nGO\n-- Clean buffers so cold start performed\nDBCC DROPCLEANBUFFERS\nGO\n-- Prepare Database Compatibility level for Test #2\nALTER DATABASE WideWorldImportersDW SET COMPATIBILITY_LEVEL = 150;\nGO\n-- Test #2\nSELECT [Customer Key], \n     SUM(Quantity) AS TotalQty,\n\tAVG(Quantity) AS AvgQty, \n\tAVG([Unit Price]) AS AvgUnitPrice\nFROM Fact.[OrderBig]\nWHERE [Customer Key] &gt; 10 and [Customer Key] &lt; 100\nGROUP BY [Customer Key]\nORDER BY [Customer Key];\nGO<\/pre>\n<p>The code in Listing 2 executes two different tests, collects some performance statistics, and cleans the data buffer cache between each test. Both tests run the same simple aggregate query against the <em>Fact.OrderBig <\/em>table. Test #1 runs the aggregate <code>SELECT<\/code> statement using compatibility level 140, whereas Test #2 runs the same aggregate <code>SELECT<\/code> statement using compatibility level 150. By setting the compatibility level to 140, Test #1 uses row mode processing. Whereas Test #2, uses compatibility level 150, so batch mode can be considered for the test query. Additionally, I turned on the TIME statistics so I could measure performance (CPU and Elapsed time) between each test. By doing this, I can validate the performance note in Figure 3, that was found in this Microsoft <a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/relational-databases\/performance\/intelligent-query-processing?view=sql-server-ver15#batch-mode-on-rowstore\">documentation<\/a>.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"915\" height=\"121\" class=\"wp-image-86761\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-48.png\" \/><\/p>\n<p class=\"caption\">Figure 3: Documentation Note on Performance<\/p>\n<p>When I ran my test script in Listing 2, I executed it from a SQL Server Management Studio (SSMS) query window. In that query window, I enabled the <em>Include Actual Execution Plan <\/em>option so that I could compare the execution plans created for both of my tests. Let me review the execution artifacts created when I ran my test script in Listing 2.<\/p>\n<h3>Review Execution Artifacts<\/h3>\n<p>When I ran my test script, I collected CPU and Elapsed Time statistics as well as the actual execution plans for each execution of my test aggregate query. In this section, I\u2019ll review the different execution artifacts to compare the differences between row mode and batch mode processing.<\/p>\n<p>The CPU and Elapsed time statistics, as well as the actual execution plan for when I ran my first test query, which was using compatibility level 140, can be found in Figure 4 and Figure 5 respectfully.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"414\" height=\"77\" class=\"wp-image-86762\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-49.png\" \/><\/p>\n<p class=\"caption\">Figure 4: CPU and Elapsed Time Statistics for Test #1<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1705\" height=\"198\" class=\"wp-image-86763\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-50.png\" \/><\/p>\n<p class=\"caption\">Figure 5: Actual Execution Plan under Compatibility Level 140 for Query 1<\/p>\n<p>Figure 6 and 7 below, show the time statistics and the actual execution plan when I ran my test query under compatibility level 150. <strong><br \/><\/strong> <img loading=\"lazy\" decoding=\"async\" width=\"419\" height=\"81\" class=\"wp-image-86764\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-51.png\" \/><\/p>\n<p class=\"caption\">Figure 6: Execution Statistics for Test #2<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1695\" height=\"189\" class=\"wp-image-86765\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-52.png\" \/><\/p>\n<p class=\"caption\">Figure 7: Execution Plan for Test #2<\/p>\n<p>The first thing to note is that the plan that ran under compatibility level 150 (Figure 7) is more streamlined than the one that ran under compatibility mode 140 (Figure 6). From just looking at the execution plan for the second test query, I can\u2019t tell whether or not the query (which ran under compatibility mode 150) uses batch mode or not. To find out, you must right-click on the <em>SELECT<\/em> icon in the execution for the Test #2 query (Figure 7) and then select the <em>Properties<\/em> item from the context menu. Figure 8 shows the properties of this query.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"479\" height=\"857\" class=\"wp-image-86766\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-53.png\" \/><\/p>\n<p class=\"caption\">Figure 8: Properties for Compatibility Level 150 Query (Test #2)<\/p>\n<p>Notice that the property <em>BatchModeOnRowstoreUsed <\/em>is <em>True<\/em>. This property is a new showplan attribute that Microsoft added in SSMS version 18. When this property is true, it means that some of the operators used in processing Test #2 did use a batch mode operation on the Rowstore <em>Fact.OrderBig<\/em> table.<\/p>\n<p>To review which operators used Batch Mode on Rowstore, you must review the properties of each operator. Figure 9 has some added annotations to the execution plan that shows which operators used batch mode processing and which ones used row mode processing.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1275\" height=\"772\" class=\"wp-image-86767\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-54.png\" \/><\/p>\n<p class=\"caption\">Figure 9: Execution Plan for Batch Mode query with Operator property annotations<\/p>\n<p>If you look at the <em>Table Scan (Heap)<\/em> operator, you can see that the <em>Fact.OrderBig<\/em> table is a <em>RowStore<\/em> by reviewing the <em>Storage<\/em> Property. You can also see that this operation used batch mode by looking at the <em>Actual Execution Mode<\/em> property. All the other operators ran in batch mode, except the <em>Parallelism<\/em> operator, which used row mode.<\/p>\n<p>The test table (<em>Fact.OrderBig)<\/em> contains 23,141,200 rows and the test query referenced 3 different columns. The query didn\u2019t need all those rows because it was filtered to include the rows where the <em>customerid<\/em> was greater than 10 and less than 100. To determine the number of batches the query created, look at the properties of the table scan operator in the execution plan, which is shown in Figure 10.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"418\" height=\"744\" class=\"wp-image-86768\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-55.png\" \/><\/p>\n<p class=\"caption\">Figure 10: Number of batches used for Test #2.<\/p>\n<p>The <em>Actual Number of Batches<\/em> property in Figure 8 shows that the table scan operator of the test #2 query created 3,587 batches. To determine the number of rows in each batch, use the following formula: <em>Actual Number of Rows<\/em> divided by the <em>Actual Number of Batches<\/em>. By using this formula, I got, on average, 899.02 rows per batch.<\/p>\n<p>The cost estimate for each of the queries is the same, 50%. Therefore, to measure performance between batch mode and row mode, I\u2019ll have to look at the TIME statistics.<\/p>\n<h2>Comparing Performance of Batch Mode and Row Mode<\/h2>\n<p>To compare performance between running batch mode and row mode queries, I ran my test script in Listing 2 ten different times. I then averaged the CPU and Elapsed times between my two different tests and then graphed the results in the chart found in Figure 11.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"660\" height=\"432\" class=\"wp-image-86769\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-56.png\" \/><\/p>\n<p class=\"caption\">Figure 11: CPU and Elapsed time Comparison between Row Mode and Batch Mode<\/p>\n<p>The chart in Figure 11 shows that the row mode test query used a little more than 30% more CPU over the batch mode test query. Both the batch and row mode queries ran about the same elapsed time. Just like the note (Figure 4) above suggested, this first test showed considerable CPU improvement could be gained when a simple aggregate query uses Batch Mode processing. But not all queries are created equal when it comes to performance improvements using Batch Mode versus Row Mode.<\/p>\n<h2>Not All Queries are Created Equal When It Comes to Performance<\/h2>\n<p>The previous test showed a 30% improvement in CPU but little improvement in Elapsed Time. The resource (CPU and Elapsed Time) improvements using Batch Mode operations versus Row mode depend on the query. Here is another contrived test that shows some drastic improvements in Elapsed Time, using the new Batch Mode on Rowstore feature. The test script I used for my second performance test can be found in Listing 3.<\/p>\n<p class=\"caption\">Listing 3: Stock Item Key Query Test Script<\/p>\n<pre class=\"lang:tsql theme:ssms2012-simple-talk \">-- Turn on time statistics\nSET STATISTICS IO, TIME ON; \n-- Clean buffers so cold start performed \nDBCC DROPCLEANBUFFERS\nGO\n-- Prepare Database Compatibility level for Test #1 \nALTER DATABASE WideWorldImportersDW SET COMPATIBILITY_LEVEL = 140;\nGO\nSELECT [Stock Item Key],[City Key],[Order Date Key],[Salesperson Key],\n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key]) AS StockAvgQty, \n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key],[City Key]) \n        AS StockCityAvgQty,\n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key],[City Key],\n        [Order Date Key]) AS StockCityDateAvgQty,  \n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key],[City Key],\n        [Order Date Key],[Salesperson Key]) \n        AS StockCityDateSalespersonAvgQty\nFROM Fact.OrderBig\nWHERE [Customer Key] &gt; 10 and [Customer Key] &lt; 100\n-- Clean buffers so cold start performed \nDBCC DROPCLEANBUFFERS\nGO\n-- Prepare Database Compatibility level for Test #2 \nALTER DATABASE WideWorldImportersDW SET COMPATIBILITY_LEVEL = 150;\nGO\nSELECT [Stock Item Key],[City Key],[Order Date Key],[Salesperson Key],\n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key]) AS StockAvgQty, \n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key],[City Key]) \n        AS StockCityAvgQty,\n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key],[City Key],\n        [Order Date Key]) AS StockCityDateAvgQty,  \n    AVG(Quantity) OVER(PARTITION BY [Stock Item Key],[City Key],\n        [Order Date Key],[Salesperson Key]) \n        AS StockCityDateSalespersonAvgQty\nFROM Fact.OrderBig\nWHERE [Customer Key] &gt; 10 and [Customer Key] &lt; 100<\/pre>\n<p>In Listing 3, I used the <code>OVER<\/code> clause to create four different aggregations, where each aggregation had a different <code>PARTITION<\/code> specification. To gather the performance statistics for Listing 3 queries, I ran this script ten different times. Figure 12 shows the numbers for CPU and Elapsed Time numbers graphically.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"663\" height=\"440\" class=\"wp-image-86770\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/03\/word-image-57.png\" \/><\/p>\n<p class=\"caption\">Figure 12: CPU and Elapsed Time comparison for Window Function Query test<\/p>\n<p>As you can see by creating the different aggregation in Listing 3, I once again saw a big performance improvement in CPU (around 72%). This time, I also got a big improvement in Elapsed Time (a little more than 45%) when batch mode was used. My testing showed that not all queries are created equal when it comes to performance. For this reason, I recommend you test all the queries in your environment to determine how each query performs using this new Batch Mode on Rowstore feature. If you happen to find some queries that perform worse using batch mode, then you can either rewrite the queries to perform better or consider disabling batch mode for those problem queries.<\/p>\n<h2>Disabling Batch Mode on Row Store<\/h2>\n<p>If you find you have a few queries that don\u2019t benefit from using batch mode, and you don\u2019t want to rewrite them, then you might consider turning off the Batch Mode on Rowstore feature with a query hint.<\/p>\n<p>If you use the <code>DISALLOW_BATCH_MODE<\/code> hint, you can disable Batch Mode on Rowstore feature for a given query. The code in Listing 4 shows how I disabled batch mode for the first test query I used in this article.<\/p>\n<p class=\"caption\">Listing 4: Using \u201cDISALLOW BATCH MODE\u201d hint to disable batch mode for a single query<\/p>\n<pre class=\"lang:tsql theme:ssms2012-simple-talk\">SELECT [Customer Key], \n       SUM(Quantity) AS TotalQty,\n       AVG(Quantity) AS AvgQty, \n       AVG([Unit Price]) AS AvgUnitPrice\nFROM Fact.[OrderBig]\nWHERE [Customer Key] &gt; 10 and [Customer Key] &lt; 100\nGROUP BY [Customer Key]\nORDER BY [Customer Key]\nOPTION(USE HINT('DISALLOW_BATCH_MODE'));<\/pre>\n<p>When I ran the query in Listing 4 against the WideWorldImportersDW database running in compatibility mode 150, the query didn\u2019t invoke any batch mode operations. I verified this by reviewing the properties of each operator. They all processed using a row mode operation. The value of using the <code>DISALLOW_BATCH_MODE<\/code> hint is I can disable the batch mode feature for a single query. This means it\u2019s possible to be selective on which queries will not consider batch mode when your database is running under compatibility level 150.<\/p>\n<p>Alternatively, you could disable the Batch Mode on Rowstore feature at the database level, as shown in Listing 5.<\/p>\n<p class=\"caption\">Listing 5: Disabling Batch Mode at the database level<\/p>\n<pre class=\"lang:tsql theme:ssms2012-simple-talk\">-- Disable batch mode on rowstore\nALTER DATABASE SCOPED CONFIGURATION SET BATCH_MODE_ON_ROWSTORE = OFF;<\/pre>\n<p>Disabling the batch mode feature at the database level still allows other queries to take advantages of the other new 15.x features. This might be an excellent option to use if you wanted to move to version 15.x of SQL Server while you complete testing of all of your large aggregation queries to see how they are impacted by the batch mode feature. Once testing is complete, reenable batch mode by running the code in Listing 6.<\/p>\n<p class=\"caption\">Listing 6: Enabling Batch Mode at the database level<\/p>\n<pre class=\"lang:tsql theme:ssms2012-simple-talk\">-- Enable batch mode on rowstore\nALTER DATABASE SCOPED CONFIGURATION SET BATCH_MODE_ON_ROWSTORE = ON;<\/pre>\n<p>By using the hint or database scoped configure method to disable batch mode, I have control over how I want this new feature to affect the performance of my row mode query operations. It is great that the team at Microsoft allows these different methods to disable\/enable the Batch Mode on Rowstore feature<em>. <\/em>By allowing these different options for enable\/disabling batch mode on rowstore, I have more flexibility in how I roll out the batch mode feature across a database.<\/p>\n<h2>Which Editions Support Batch Mode?<\/h2>\n<p>Before you get too excited about how this feature might help the performance of your large analytic queries, I have to tell you the bad news. Batch Mode on Rowstore is not available to all version of SQL. Like many cool new features that have come out in the past, they are first introduced in Enterprise edition only, and then over time, they might become available in other editions. Batch Mode on Rowstore is no exception. As of the RTM release of SQL Server 2019, the Batch Mode on Rowstore feature is only available in Enterprise Edition, as documented <a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/sql-server\/editions-and-components-of-sql-server-version-15?view=sql-server-ver15\">here.<\/a> Also note that developer edition supports Batch Mode on Rowstore<em>, <\/em>but of course cannot be used for production work. Be careful when doing performance testing of this new feature on the developer edition of SQL Server 2019 if you plan to roll out your code into any production environment except Enterprise. If you want to reduce your CPU footprint using this new feature, then you better get out your checkbook and upgrade to Enterprise edition, or just wait until Microsoft rolls this feature out to other editions of SQL Server. It also works on Azure SQL Database.<\/p>\n<h2>Reduce CPU of Large Analytic Queries Without Changing Code<\/h2>\n<p>If you have large analytic queries that perform aggregations, you might find that using the new Batch Mode on Rowstore feature improves CPU and Elapsed time without changing any code if your query environment meets a few requirements. The first requirement is that your query needs to be running using SQL Server version 15.x (SQL Server 2019) or better. The second requirement is you need to be running on an edition of SQL Server that supports the Batch Mode on Rowstore feature. Additionally, the table being queried needs to have at least 131,072 rows and be stored in a b-tree or heap before batch mode is considered for the table.<\/p>\n<p>I am impressed by how much less CPU and Elapsed time was used for my test aggregation queries. If you have a system that runs lots of aggregate queries, then migrating to SQL Server 2019 might be able to eliminate your CPU bottlenecks and get some of your queries to run faster at the same time.<\/p>\n\n\n<section id=\"faq\" class=\"faq-block my-5xl\">\n    <h2>FAQs: Reduce CPU of Large Analytic Queries Without Changing Code<\/h2>\n\n                        <h3 class=\"mt-4xl\">1. What is Batch Mode on Rowstore in SQL Server 2019?<\/h3>\n            <div class=\"faq-answer\">\n                <p>Batch Mode on Rowstore is a SQL Server 2019 query processing feature that allows the database engine to use batch mode execution &#8211; processing groups of ~900 rows simultaneously rather than one row at a time &#8211; on tables without columnstore indexes. It applies automatically to qualifying analytic aggregate queries and can reduce CPU usage significantly. Previously available only with columnstore indexes, Batch Mode on Rowstore extends this performance improvement to conventional heap and B-tree table storage.<\/p>\n            <\/div>\n                    <h3 class=\"mt-4xl\">2. How much does Batch Mode on Rowstore reduce CPU in SQL Server 2019?<\/h3>\n            <div class=\"faq-answer\">\n                <p>The improvement varies by query, but typical analytic aggregate workloads see 20\u201330% CPU reduction. Elapsed time improvements are smaller because batch mode primarily optimises CPU efficiency rather than I\/O. Queries that benefit most are those with large aggregations over many rows &#8211; SUM, AVG, COUNT across millions of rows. Queries with complex filters, small result sets, or non-aggregate operations see less benefit.<\/p>\n            <\/div>\n                    <h3 class=\"mt-4xl\">3. Which SQL Server editions support Batch Mode on Rowstore?<\/h3>\n            <div class=\"faq-answer\">\n                <p>Batch Mode on Rowstore is available in SQL Server 2019 Enterprise and Developer editions only. It is not available in Standard, Express, or Web editions of SQL Server 2019. On SQL Server 2017 and earlier, Batch Mode is only available on tables with a columnstore index, regardless of edition. Check your edition before planning workload migrations that depend on this feature.<\/p>\n            <\/div>\n                    <h3 class=\"mt-4xl\">4. How do I disable Batch Mode on Rowstore for a specific query?<\/h3>\n            <div class=\"faq-answer\">\n                <p>Add the query hint OPTION (USE HINT (&#8216;DISALLOW_BATCH_MODE&#8217;)) to the SELECT statement. This forces row mode processing for that specific query without affecting other queries. Use this selectively for queries where you have confirmed batch mode is not helping or is causing unexpected plan changes. A database-level option (ALTER DATABASE SCOPED CONFIGURATION) can also disable it for the entire database if needed during testing.<\/p>\n            <\/div>\n            <\/section>\n","protected":false},"excerpt":{"rendered":"<p>SQL Server 2019 Batch Mode on Rowstore reduces CPU usage of large analytic aggregate queries by 20\u201330% without rewriting any code. Learn how it works, see a benchmarked comparison, and find out which editions support it.&hellip;<\/p>\n","protected":false},"author":78478,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":true,"footnotes":""},"categories":[143529,143531],"tags":[48385,5842],"coauthors":[11330],"class_list":["post-86758","post","type-post","status-publish","format-standard","hentry","category-performance-sql-server","category-t-sql-programming-sql-server","tag-cpu-monitoring","tag-sql-monitor"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/86758","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/78478"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=86758"}],"version-history":[{"count":6,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/86758\/revisions"}],"predecessor-version":[{"id":109624,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/86758\/revisions\/109624"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=86758"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=86758"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=86758"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=86758"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}