{"id":1250,"date":"2012-01-04T00:00:00","date_gmt":"2011-12-06T00:00:00","guid":{"rendered":"https:\/\/test.simple-talk.com\/uncategorized\/window-functions-in-sql-server-part-3-questions-of-performance\/"},"modified":"2021-08-16T15:02:02","modified_gmt":"2021-08-16T15:02:02","slug":"window-functions-in-sql-server-part-3-questions-of-performance","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/learn\/window-functions-in-sql-server-part-3-questions-of-performance\/","title":{"rendered":"SQL Server Window Function Performance: Execution Plans for Running Aggregates, LEAD, LAG, FIRST_VALUE, LAST_VALUE"},"content":{"rendered":"<div id=\"pretty\">\n<p class=\"start\">Are Window Functions as fast as the equivalent alternative SQL methods? In this article I&#8217;ll be exploring the performance of the window functions. After you read this article I hope you&#8217;ll be in a better position to decide whether it is really worth changing your application code to start using window functions.<\/p>\n<p>Before I start the tests, I would like to explain the most important factor that affects the performance of window functions. A window spool operator has two alternative ways of storing the frame data, with the <strong><em>in<\/em><\/strong><strong><em>&#8211;<\/em><\/strong><strong><em>memory<\/em><\/strong> worktable or with a <strong><em>disk<\/em><\/strong><strong><em>&#8211;<\/em><\/strong><strong><em>based<\/em><\/strong> worktable. You&#8217;ll have a huge difference on performance according to the way that the query processor is executing the operator.<\/p>\n<p>The in-memory worktable is used when you define the frame as ROWS and it is lower than 10000 rows. If the frame is greater than 10000 rows, then the window spool operator will work with the on-disk worktable.<\/p>\n<p>The on-disk based worktable is used with the default frame, that is, <em>&#8220;range<\/em><em>&#8230;<\/em><em>&#8220;<\/em> and a frame with more than 10000 rows.<\/p>\n<p>It&#8217;s possible to check whether the window spool operator is using the on-disk worktable by looking at the results of the <strong>SET STATISTICS IO<\/strong>. Let&#8217;s see a small example of the window spool using the on-disk based worktable:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">USE tempdb\r\nGO\r\nIF OBJECT_ID('tempdb.dbo.#TMP')\u00a0\u00a0IS NOT NULL\r\n\u00a0\u00a0DROP TABLE #TMP\r\nGO\r\nCREATE TABLE #TMP (ID INT, Col1 CHAR(1), Col2 INT)\r\nGO\r\nINSERT INTO #TMP VALUES(1,'A', 5), (2, 'A', 5), (3, 'B', 5), (4, 'C', 5), (5, 'D', 5)\r\nGO\r\nSET STATISTICS IO ON\r\nSELECT *,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SUM(Col2) OVER(ORDER BY Col1 RANGE UNBOUNDED PRECEDING) \"Range\" \u00a0\u00a0FROM #TMP\r\nSET STATISTICS IO OFF <\/pre>\n<p>\u00a0<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">Table 'Worktable'. Scan count 5, logical reads 29, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.\r\nTable 'Worktable'. Scan count 1, logical reads 1, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.<\/pre>\n<p>\u00a0<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">SET STATISTICS IO ON\r\nSELECT *,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SUM(Col2) OVER(ORDER BY Col1 ROWS UNBOUNDED PRECEDING) \"Rows\" \u00a0\u00a0FROM #TMP\r\nSET STATISTICS IO OFF <\/pre>\n<p>\u00a0<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">Table 'Worktable'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.\r\nTable 'Worktable'. Scan count 1, logical reads 1, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. <\/pre>\n<p>As you can see, the worktable has some reads when you use a &#8220;range&#8221;, but it doesn&#8217;t happen with the in-memory worktable which is used when you use &#8220;Rows&#8221;.<\/p>\n<p>Optionally you also can look at the xEvent called <em>window_spool_ondisk_warning<\/em><em>. <\/em>Fortunately on SQL Server 2012 we&#8217;ve a very nice interface that we can use to deal with xEvents. In SSMS, you can easily capture an event, and then look at the live data captured. Here you can see a screen-capture of this interface:<\/p>\n<div>\n<table>\n<tbody>\n<tr>\n<td><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image001.png\" alt=\"1403-image001.png\" \/><\/td>\n<td><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image003.png\" alt=\"1403-image003.png\" \/><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>There are many aspects of the window spool operator that could be improved, I just opened some connect items about this and I&#8217;d appreciate it if you could take a minute or so to vote on this items.<\/p>\n<ul>\n<li><a href=\"https:\/\/connect.microsoft.com\/SQLServer\/feedback\/details\/710074\/window-spool-worktable-doesnt-spill-to-tempdb\">Window Spool worktable doesn&#8217;t spill to tempdb<\/a><\/li>\n<li><a href=\"https:\/\/connect.microsoft.com\/SQLServer\/feedback\/details\/710076\/default-window-spool-worktable-in-memory\">Default window spool worktable in memory<\/a><\/li>\n<li><a href=\"https:\/\/connect.microsoft.com\/SQLServer\/feedback\/details\/710079\/granting-memory-to-process-window-spool-worktable\">Granting memory to process window spool worktable<\/a><\/li>\n<li><a href=\"https:\/\/connect.microsoft.com\/SQLServer\/feedback\/details\/679342\/unnecessary-sort\">Unnecessary Sort<\/a><\/li>\n<\/ul>\n<p>For now, when the logic of the query allows it, it is a good practice to <strong><em>always<\/em><\/strong> define the window frame as ROWS in order to try to use the in-memory worktable.<\/p>\n<h2>Performance tests<\/h2>\n<p>I&#8217;ll now show some performance tests that I did with the most-used windows functions such as LEAD, LAG, LAST_VALUE and the OVER clause.<\/p>\n<p>To execute the tests, I used the free SQLQueryStress toll created by Adam Machanic (<a href=\"http:\/\/sqlblog.com\/blogs\/adam_machanic\/\">blog<\/a> | <a href=\"http:\/\/twitter.com\/\">twitter<\/a>), you can download it <a href=\"http:\/\/dataeducation.com\/sqlquerystress-the-source-code\/\">here<\/a>, I also will use the sample database AdventureWorks2008R2.<\/p>\n<p>Let&#8217;s start by testing the performance of the clause OVER with the running aggregation solution, let&#8217;s see how it would scale with many users demanding information from the server.<\/p>\n<p>First let&#8217;s create a table with 453772 rows:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">USE AdventureWorks2008R2\r\nGO\r\nSET NOCOUNT ON;\r\nIF OBJECT_ID('TestTransactionHistory') IS NOT NULL\r\n\u00a0\u00a0DROP TABLE TestTransactionHistory\r\nGO\r\nSELECT IDENTITY(INT, 1,1) AS TransactionID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ProductID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ReferenceOrderID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ReferenceOrderLineID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TransactionDate,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TransactionType,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Quantity,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ActualCost,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ModifiedDate\r\n\u00a0\u00a0INTO TestTransactionHistory\r\n\u00a0\u00a0FROM Production.TransactionHistory\r\n\u00a0\u00a0CROSS JOIN (VALUES (1), (2), (3) , (4)) AS Tab(Col1)\r\nGO\r\nCREATE UNIQUE CLUSTERED INDEX ixClusterUnique ON TestTransactionHistory(TransactionID)\r\nGO <\/pre>\n<h2>Running Aggregation, Clause OVER + Order By<\/h2>\n<p>Now let&#8217;s run the old solution to the running aggregation query:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Creating a index to help the subquery\r\nCREATE INDEX ix1 ON TestTransactionHistory (ProductID, TransactionDate) INCLUDE(Quantity)\r\n\r\n-- Old Solution\r\n-- Test running 200 threads, 5 iterations per session\r\n-- AVG Elapsed time: 00:02:02.3009\r\n-- AVG Logical Reads: 8665\r\nSELECT TransactionID, \r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ProductID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Quantity,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 (SELECT SUM(Quantity) \r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0FROM TestTransactionHistory b\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WHERE b.ProductID = a.ProductID\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 AND b.TransactionDate &lt;= a.TransactionDate) AS \"Running Total\" \u00a0\u00a0FROM TestTransactionHistory a\r\nWHERE ProductID = ROUND(((999 - 930 -1) * RAND() + 930), 0)\r\nGO <\/pre>\n<p>Let&#8217;s look at the execution plan:<\/p>\n<p class=\"illustration\"><a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image004.png\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image004small.png\" alt=\"1403-image004small.png\" \/><\/a><\/p>\n<p>This is a pretty straightforward plan: For each row read in the table, it executes the seek to calculate the running totals.<\/p>\n<p>I&#8217;m filtering by a random <strong>ProductID<\/strong> to simulate many users requiring different products at the same time. Let&#8217;s run it in on SQLQueryStress:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image006.png\" alt=\"1403-image006.png\" \/><\/p>\n<p>Now let&#8217;s run it using the clause OVER with ORDER BY:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Clause Over with ORDER BY, DEFAULT FRAME RANGE UNBOUNDED PRECEDING\r\n-- Test running 200 threads, 5 iterations per session\r\n-- AVG Elapsed time: 00:00:14.5668\r\n-- AVG Logical Reads: 8957\r\nSELECT TransactionID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ProductID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Quantity,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SUM(Quantity) OVER (PARTITION BY ProductID ORDER BY TransactionDate, TransactionID RANGE UNBOUNDED PRECEDING) \"Running Total\" \u00a0\u00a0FROM TestTransactionHistory\r\nWHERE ProductID = ROUND(((999 - 930 -1) * RAND() + 930), 0)\r\nGO <\/pre>\n<p>The execution plan is:<\/p>\n<p class=\"illustration\"><a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image008.png\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image008small.png\" alt=\"1403-image008small.png\" \/><\/a><\/p>\n<p>As you can see, there is no double access to the table here. There is just a window spool doing the magic of keeping the rows to be aggregated.<\/p>\n<p>The results from SQLQueryStress:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image010.png\" alt=\"1403-image010.png\" \/><\/p>\n<p>Now let&#8217;s specify the ROWS frame in the query and run again:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Clause Over with ORDER BY, ROWS WINDOW FRAME, ROWS UNBOUNDED PRECEDING\r\n-- Test running 200 threads, 5 iterations per session\r\n-- AVG Elapsed time: 00:00:02.7381\r\n-- AVG Logical Reads: 8\r\nSELECT TransactionID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ProductID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Quantity,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SUM(Quantity) OVER (PARTITION BY ProductID ORDER BY TransactionDate, TransactionID ROWS UNBOUNDED PRECEDING) \"Running Total\" \u00a0\u00a0FROM TestTransactionHistory\r\nWHERE ProductID = ROUND(((999 - 930 -1) * RAND() + 930), 0)\r\nGO <\/pre>\n<p>The results from SQLQueryStress:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image012.png\" alt=\"1403-image012.png\" \/><\/p>\n<p>As you can see in the elapsed time, the difference between the old solution and the solution using the worktable is very good.<\/p>\n<div>\n<table>\n<tbody>\n<tr>\n<td valign=\"top\">\n<p><b>Query<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Client Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>CPU Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Actual Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Logical Reads\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>old solution<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,4312<\/p>\n<\/td>\n<td valign=\"top\">\n<p>1,4528<\/p>\n<\/td>\n<td valign=\"top\">\n<p>1,2097<\/p>\n<\/td>\n<td valign=\"top\">\n<p>9514,3370<\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>RANGE query<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,1148<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,04<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0931<\/p>\n<\/td>\n<td valign=\"top\">\n<p>9206,5540<\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>ROWS query<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0385<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0075<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0241<\/p>\n<\/td>\n<td valign=\"top\">\n<p>9,0800<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>What about the LEAD and LAG function?<\/p>\n<h2>LEAD<\/h2>\n<p>First let&#8217;s see the execution plan for a query to return the next row from the <strong>TransactionHistory<\/strong> table.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Old Solution\r\n-- Test running 50 threads, 5 iterations per session\r\n-- AVG Elapsed time: 00:00:33.7419\r\n-- AVG Logical Reads: 11860\r\nSELECT a.TransactionID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Tab.\"Next TransactionID\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 a.ProductID\r\n\u00a0\u00a0FROM TestTransactionHistory a\r\nOUTER APPLY (SELECT TOP 1 b.TransactionID AS \"Next TransactionID\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0FROM TestTransactionHistory b\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WHERE b.ProductID = a.ProductID\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 AND b.TransactionID &gt; a.TransactionID\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ORDER BY b.TransactionID) AS Tab\r\nWHERE a.ProductID = ROUND(((999 - 930 -1) * RAND() + 930), 0)\r\nORDER BY a.ProductID, a.TransactionID\r\nGO <\/pre>\n<p class=\"illustration\"><a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image014.png\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image014small.png\" alt=\"1403-image014small.png\" \/><\/a><\/p>\n<p>As you can see, an easy way to do get the next row of a table is using the OUTER APPLY clause. Let&#8217;s see the performance of this query:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image016.png\" alt=\"1403-image016.png\" \/><\/p>\n<p>Now let&#8217;s see the same query but now using the LEAD function to get the next row.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Using LEAD function\r\n-- Test running 50 threads, 5 iterations per session\r\n-- AVG Elapsed time: 00:00:00.9020\r\n-- AVG Logical Reads: 9\r\nSELECT TransactionID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 LEAD(TransactionID) OVER(ORDER BY ProductID, TransactionID) AS \"Next TransactionID\",\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ProductID\r\n\u00a0\u00a0FROM TestTransactionHistory\r\nWHERE ProductID = ROUND(((999 - 930 -1) * RAND() + 930), 0)\r\nORDER BY ProductID, TransactionID <\/pre>\n<p>Execution Plan:<\/p>\n<p class=\"illustration\"><a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image018.png\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image018small.png\" alt=\"1403-image018small.png\" \/><\/a><\/p>\n<p>As you can see from the execution plan, we could improve the query by creating an index on the columns <strong>ProductID<\/strong> and <strong>TransactionID<\/strong> to remove the sort operator, but in order to compare the performance, let&#8217;s see the results of the execution on this query:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image020.png\" alt=\"1403-image020.png\" \/><\/p>\n<p>Again, the window function is much better than the old subquery solution.<\/p>\n<div>\n<table>\n<tbody>\n<tr>\n<td valign=\"top\">\n<p><b>Query<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Client Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>CPU Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Actual Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Logical Reads\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>Subquery solution<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>2,4539<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,4108<\/p>\n<\/td>\n<td valign=\"top\">\n<p>1,7373<\/p>\n<\/td>\n<td valign=\"top\">\n<p>12129,2840<\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>Window Function<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0222<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0077<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0148<\/p>\n<\/td>\n<td valign=\"top\">\n<p>9,1640<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>Now let&#8217;s see the LAG function.<\/p>\n<h2>LAG<\/h2>\n<p>First the old approach to get the previews row:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Old Solution\r\n-- Test running 50 threads, 5 iterations per session\r\n-- AVG Elapsed time: 00:00:31.4037\r\n-- AVG Logical Reads: 11125\r\nSELECT a.TransactionID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Tab.\"Previews TransactionID\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 a.ProductID\r\n\u00a0\u00a0FROM TestTransactionHistory a\r\nOUTER APPLY (SELECT TOP 1 b.TransactionID AS \"Previews TransactionID\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0FROM TestTransactionHistory b\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WHERE b.ProductID = a.ProductID\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 AND b.TransactionID &lt; a.TransactionID\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ORDER BY b.TransactionID DESC) AS Tab\r\nWHERE a.ProductID = ROUND(((999 - 930 -1) * RAND() + 930), 0)\r\nORDER BY a.ProductID, a.TransactionID\r\nGO <\/pre>\n<p>The results from SQLQueryStress:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image022.png\" alt=\"1403-image022.png\" \/><\/p>\n<p>Now using the LAG function:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Using LAG function\r\n-- Test running 50 threads, 5 iterations per session\r\n-- AVG Elapsed time: 00:00:00.8690\r\n-- AVG Logical Reads: 9\r\nSELECT TransactionID,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 LAG(TransactionID) OVER(ORDER BY ProductID, TransactionID) AS \"Previews TransactionID\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ProductID\r\n\u00a0\u00a0FROM TestTransactionHistory\r\nWHERE ProductID = ROUND(((999 - 930 -1) * RAND() + 930), 0)\r\nORDER BY ProductID, TransactionID <\/pre>\n<p>The results from SQLQueryStress:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image024.png\" alt=\"1403-image024.png\" \/><\/p>\n<p>Once again, the performance on windowing functions is much better.<\/p>\n<div>\n<table>\n<tbody>\n<tr>\n<td valign=\"top\">\n<p><b>Query<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Client Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>CPU Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Actual Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Logical Reads\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>Subquery solution<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>2,9293<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,4106<\/p>\n<\/td>\n<td valign=\"top\">\n<p>1,9069<\/p>\n<\/td>\n<td valign=\"top\">\n<p>11001,4880<\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>Lag Window function<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>0.0193<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0062<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0117<\/p>\n<\/td>\n<td valign=\"top\">\n<p>8,2160<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<div class=\"note\">\n<p class=\"note\">Note: The LEAD and LAG functions doesn&#8217;t accept a frame, in this case the default of window spool is to use the in memory worktable.<\/p>\n<\/div>\n<h2>First_Value and Last_Value<\/h2>\n<p>Let&#8217;s suppose I want to return all the employees from a specific department. I also want to know which employee had the largest number of hours not working because he\/she was sick and who that employee was.<\/p>\n<p>To write it without window functions, I could use subqueries to return the data of most hours sick and who was the employee; something like the following query:<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Old Solution\r\n-- Test running 200 threads, 100 iterations per session\r\n-- AVG Elapsed time: 00:00:14.7488\r\n-- AVG Logical Reads: 391\r\nDECLARE @i INT = ROUND(((16 - 1 -1) * RAND() + 1), 0);\r\nWITH CTE_1\r\nAS\r\n(\r\n\u00a0\u00a0SELECT Person.FirstName + ' ' + Person.LastName AS EmployeeName,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Employee.JobTitle,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Employee.SickLeaveHours,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Employee.HireDate,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Department.Name AS DepartmentName\r\n\u00a0\u00a0\u00a0\u00a0FROM HumanResources.Employee\r\n\u00a0\u00a0 INNER JOIN HumanResources.EmployeeDepartmentHistory\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0ON Employee.BusinessEntityID = EmployeeDepartmentHistory.BusinessEntityID\r\n\u00a0\u00a0 INNER JOIN HumanResources.Department\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0ON EmployeeDepartmentHistory.DepartmentID = Department.DepartmentID\r\n\u00a0\u00a0 INNER JOIN Person.Person\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0ON Employee.BusinessEntityID = Person.BusinessEntityID\r\n\u00a0\u00a0 WHERE EmployeeDepartmentHistory.EndDate IS NULL\r\n\u00a0\u00a0\u00a0\u00a0 AND Department.DepartmentID = @i\r\n)\r\nSELECT a.*,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 (SELECT TOP 1 b.SickLeaveHours\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0FROM CTE_1 b\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WHERE a.DepartmentName = b.DepartmentName\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ORDER BY b.SickLeaveHours DESC) AS MostSickEmployee,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 (SELECT TOP 1 c.EmployeeName\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0FROM CTE_1 c\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WHERE a.DepartmentName = c.DepartmentName\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ORDER BY c.SickLeaveHours DESC, c.HireDate DESC) AS MostSickEmployeeName\r\n\u00a0\u00a0FROM CTE_1 a\r\nORDER BY a.DepartmentName, a.EmployeeName\r\nGO <\/pre>\n<p>This is the result:<\/p>\n<p class=\"illustration\"><a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image026.png\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image026small.png\" alt=\"1403-image026small.png\" \/><\/a><\/p>\n<p>The employee that had the most time off-sick was &#8220;Brian LaMee&#8221;. I had to use a CTE to make the query a little easier to understand, and I wrote two subqueries on this CTE.<\/p>\n<p>Let&#8217;s see the performance:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image028.png\" alt=\"1403-image028.png\" \/><\/p>\n<p>Now how it would be with window functions?<\/p>\n<p>When you first try the functions FIRST_VALUE and LAST_VALUE, it may seem confusing and you might wonder if the functions MAX and MIN wouldn&#8217;t return the same values of FIRST_VALUE and LAST_VALUE, let&#8217;s see an example.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- New Solution\r\n-- Test running 200 threads, 100 iterations per session\r\n-- AVG Elapsed time: 00:00:11.9196\r\n-- AVG Logical Reads: 189\r\nSELECT Person.FirstName + ' ' + Person.LastName AS EmployeeName,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Employee.JobTitle,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Employee.SickLeaveHours,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Employee.HireDate,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Department.Name AS DepartmentName,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 MAX(Employee.SickLeaveHours) OVER(PARTITION BY Department.Name ORDER BY Employee.SickLeaveHours ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS MostSickEmployee,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 LAST_VALUE(Person.FirstName + ' ' + Person.LastName) OVER(PARTITION BY Department.Name ORDER BY Employee.SickLeaveHours, Employee.HireDate ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS MostSickEmployeeName\r\n\u00a0\u00a0FROM HumanResources.Employee\r\nINNER JOIN HumanResources.EmployeeDepartmentHistory\r\n\u00a0\u00a0\u00a0\u00a0ON Employee.BusinessEntityID = EmployeeDepartmentHistory.BusinessEntityID\r\nINNER JOIN HumanResources.Department\r\n\u00a0\u00a0\u00a0\u00a0ON EmployeeDepartmentHistory.DepartmentID = Department.DepartmentID\r\nINNER JOIN Person.Person\r\n\u00a0\u00a0\u00a0\u00a0ON Employee.BusinessEntityID = Person.BusinessEntityID\r\nWHERE EmployeeDepartmentHistory.EndDate IS NULL\r\n\u00a0\u00a0 AND Department.DepartmentID = ROUND(((16 - 1 -1) * RAND() + 1), 0)\r\nORDER BY Department.Name, Person.FirstName + ' ' + Person.LastName <\/pre>\n<p>And the results:<\/p>\n<p class=\"illustration\"><a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image030.png\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image030small.png\" alt=\"1403-image030small.png\" \/><\/a><\/p>\n<p>The same results were returned, and the in fact the function MAX can be used to return the LAST_VALUE of the value in a partition, in this case the partition is the department, so the MAX of a window partitioned by department with a frame that goes from UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING will return the last value based on the ORDER BY of the column specified in the MAX function.<\/p>\n<p>The LAST_VALUE function will return the last value based on the ORDER BY clause specified in the OVER clause. If you didn&#8217;t understand, then try to change the LAST_VALUE for the MAX and see that the results will not be the ones you might expect.<\/p>\n<p>Now let&#8217;s see the performance:<\/p>\n<p class=\"illustration\"><img decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1403-image032.png\" alt=\"1403-image032.png\" \/><\/p>\n<p>This time we can&#8217;t see a big difference between the queries, but that&#8217;s probably because the tables I used are very small.<\/p>\n<div>\n<table>\n<tbody>\n<tr>\n<td valign=\"top\">\n<p><b>Query<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Client Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>CPU Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Actual Seconds\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p><b>Logical Reads\/Iteration (Avg)<\/b><\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>CTE solution<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0268<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0017<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0029<\/p>\n<\/td>\n<td valign=\"top\">\n<p>411,3282<\/p>\n<\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">\n<p><b>Window function<\/b><\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0253<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0011<\/p>\n<\/td>\n<td valign=\"top\">\n<p>0,0024<\/p>\n<\/td>\n<td valign=\"top\">\n<p>209,5995<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<div class=\"note\">\n<p class=\"note\"><strong>Note:<\/strong> The performance will vary in your SQL Server environment since the codes are all random.<\/p>\n<\/div>\n<h2>Conclusion<\/h2>\n<p>In general the window functions will perform better than the old solutions using subqueries that we are used to writing.<\/p>\n<p>I recommend you to always test both solutions the old and the new using the window function, if the new solution is not performing better than the old one you should always check whether the window spool operator is working with the worktable on-disk.<\/p>\n<p>That&#8217;s all folks.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Performance analysis of SQL Server window functions &#8211; comparing execution plans for running aggregations (OVER + ORDER BY), LEAD and LAG offset functions, and FIRST_VALUE \/ LAST_VALUE analytic functions. With benchmarks comparing window functions to pre-SQL-2012 subquery-based equivalents.&hellip;<\/p>\n","protected":false},"author":65554,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[143525],"tags":[4178,5543,5542,4149,5541,4150,4151,5540],"coauthors":[6809],"class_list":["post-1250","post","type-post","status-publish","format-standard","hentry","category-learn","tag-bi","tag-lag","tag-lead","tag-learn-sql-server","tag-performance-tests","tag-sql","tag-sql-server","tag-window-functions"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/1250","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/65554"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=1250"}],"version-history":[{"count":4,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/1250\/revisions"}],"predecessor-version":[{"id":76586,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/1250\/revisions\/76586"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=1250"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=1250"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=1250"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=1250"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}