{"id":1713,"date":"2013-10-17T00:00:00","date_gmt":"2013-10-17T00:00:00","guid":{"rendered":"https:\/\/test.simple-talk.com\/uncategorized\/calculating-values-within-a-rolling-window-in-transact-sql\/"},"modified":"2021-09-29T16:21:41","modified_gmt":"2021-09-29T16:21:41","slug":"calculating-values-within-a-rolling-window-in-transact-sql","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/t-sql-programming-sql-server\/calculating-values-within-a-rolling-window-in-transact-sql\/","title":{"rendered":"Calculating Values within a Rolling Window in Transact SQL"},"content":{"rendered":"<div id=\"pretty\">\n<h1>Calculating Values within a Rolling Window in SQL<\/h1>\n<p>Any time that you need to combine values across several rows in SQL, the problem  can be challenging, particularly when it comes to performance.&#160; We will  focus upon the rolling twelve-month totals problem, but our methods can be applied to any time window (e.g., 3 months)  or to averages and other aggregations across those time windows as well. <\/p>\n<p>A rolling total for a month is the total for that month plus the previous months  within the time window, or <code>NULL<\/code> if you don&#8217;t have the values for all the previous months within the time window .<\/p>\n<p>In previous versions of SQL Server, you had to jump through a few hoops to come  up with a method that performs well, but SQL 2012 offers some new features that make it simpler. &#160;In either case, there are several valid solutions. &#160;Which is fastest and most efficient?&#160; We&#8217;ll try to answer this question &#160;in this article.<\/p>\n<p>We will be working in SQL 2012. &#160;If you  would like to follow along, you can use the <b><i>Sample Queries.sql<\/i><\/b> resource you&#8217;ll find attached.<\/p>\n<h2>Data Setup and Statement of the Business Problem<\/h2>\n<p>Often times you&#8217;ll find yourself with many transactions within a month, but in  our case we&#8217;ll assume you&#8217;ve already grouped your transactions for each month.&#160; We&#8217;ll assign our <code>PRIMARY KEY<\/code> to a <code>DATE<\/code> data type, and include some values over which we want to accumulate  rolling twelve month totals.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">CREATE TABLE #RollingTotalsExample\n(\n&#160;&#160;&#160; [Date] &#160;&#160;&#160;&#160;DATE PRIMARY KEY\n&#160;&#160;&#160; ,[Value] &#160;&#160;INT\n);\n&#160;\nINSERT INTO #RollingTotalsExample\nSELECT '2011-01-01',626\nUNION ALL SELECT '2011-02-01',231 UNION ALL SELECT '2011-03-01',572\nUNION ALL SELECT '2011-04-01',775 UNION ALL SELECT '2011-05-01',660\nUNION ALL SELECT '2011-06-01',662 UNION ALL SELECT '2011-07-01',541\nUNION ALL SELECT '2011-08-01',849 UNION ALL SELECT '2011-09-01',632\nUNION ALL SELECT '2011-10-01',906 UNION ALL SELECT '2011-11-01',961\nUNION ALL SELECT '2011-12-01',361 UNION ALL SELECT '2012-01-01',461\nUNION ALL SELECT '2012-02-01',928 UNION ALL SELECT '2012-03-01',855\nUNION ALL SELECT '2012-04-01',605 UNION ALL SELECT '2012-05-01',83\nUNION ALL SELECT '2012-06-01',44 UNION ALL SELECT '2012-07-01',382\nUNION ALL SELECT '2012-08-01',862 UNION ALL SELECT '2012-09-01',549\nUNION ALL SELECT '2012-10-01',632 UNION ALL SELECT '2012-11-01',2\nUNION ALL SELECT '2012-12-01',26;\n&#160;\nSELECT * FROM #RollingTotalsExample;\n<\/pre>\n<p>Since a valid, rolling twelve month total can&#8217;t occur until you have at least  twelve months of data in your set, we seek to generate a NULL value for our rolling total column for the first 11 rows  in the returned results.&#160; It is only in the 12th month of 2011  that we will have twelve months of data in which to calculate the rolling total.&#160; With our sample data, we can calculate that total as 7776 (or we can run the query below if manual calculations  aren&#8217;t your thing).<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">SELECT SUM(Value)\nFROM #RollingTotalsExample\nWHERE [Date] &lt;= '2011-12-01';\n<\/pre>\n<p>Calculating a rolling twelve month total is akin to calculating a running total  from all prior rows, with just a few calculation tricks we&#8217;ll show a little later on.<\/p>\n<h2>Solutions That Work in SQL Server 2005 Onwards<\/h2>\n<h3>Solution #1: Using A Tally Table<\/h3>\n<p>Hopefully you&#8217;ll know what a Tally table is, but to make a long story short it  is simply a table that has one integer column that is a sequential number from 1 to n, n being the number of rows you  need.&#160; Since we know that each monthly row in our test table must be summed  into exactly 12 other rows (ignoring end points), perhaps we can use a simple zero-based Tally table to do this.&#160; Let&#8217;s give it a try.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">WITH Tally (n) AS\n(\n&#160;&#160;&#160; SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3\n&#160;&#160;&#160; UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7\n&#160;&#160;&#160; UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11\n)\nSELECT [Date], Value, n, GroupingDate=DATEADD(month, n, [Date])\nFROM #RollingTotalsExample a\nCROSS APPLY Tally b\nWHERE [Date] = '2011-01-01';\n<\/pre>\n<p>This query returns 12 rows where the <code>GroupingDate<\/code> column represents the period  we want to sum the 2011-01-01 row into.&#160; It is then a relatively simple  matter to create the sum over the grouping like this.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">WITH Tally (n) AS\n(\n&#160;&#160;&#160; SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3\n&#160;&#160;&#160; UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7\n&#160;&#160;&#160; UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11\n)\nSELECT GroupingDate=DATEADD(month, n, [Date])\n&#160;&#160;&#160; ,Value=MAX(CASE n WHEN 0 THEN a.Value END)\n&#160;&#160;&#160; ,Rolling12Months=SUM(Value)\nFROM #RollingTotalsExample a\nCROSS APPLY Tally b\nGROUP BY DATEADD(month, n, [Date])\nORDER BY DATEADD(month, n, [Date]);\n<\/pre>\n<p>When we examine the results of this query, we find that there are a couple of  issues:<\/p>\n<ul>\n<li>The first 11 rows do not show <code>NULL<\/code> as we&#8217;d like.<\/li>\n<li>The last 11 rows represent dates that are outside of our data range, namely they are  \t\tlater than 2012-12-01.<\/li>\n<\/ul>\n<p>The first issue can be taken care of with a <code>ROW_NUMBER()<\/code>, while the second issue  is resolved with a <code>HAVING<\/code> clause.&#160;&#160;&#160; Since we must reference our  \t<code>GroupingDate<\/code> column in multiple places, we&#8217;ll put that calculation into a  \t<code>CROSS APPLY<\/code> so we only need to do it once.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Rolling twelve month totals using a Tally table\nWITH Tally (n) AS\n(\n&#160;&#160;&#160; SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3\n&#160;&#160;&#160; UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7\n&#160;&#160;&#160; UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11\n)\nSELECT GroupingDate\n&#160;&#160;&#160; ,Value=MAX(CASE n WHEN 0 THEN a.Value END)\n&#160;&#160;&#160; ,Rolling12Months=CASE \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHEN ROW_NUMBER() OVER (ORDER BY GroupingDate) &lt; 12 \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; THEN NULL \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ELSE SUM(Value) \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; END\nFROM #RollingTotalsExample a\nCROSS APPLY Tally b\nCROSS APPLY\n(\n&#160;&#160;&#160; SELECT GroupingDate=DATEADD(month, n, [Date])\n) c\nGROUP BY GroupingDate\nHAVING GroupingDate &lt;= MAX([Date])\nORDER BY GroupingDate;\n<\/pre>\n<p>For reference and comparison against later queries, here are the final (correct)  results.<\/p>\n<pre>GroupingDate&#160;&#160; Value&#160;&#160; Rolling12Months\n2011-01-01&#160;&#160;&#160;&#160; 626&#160;&#160;&#160;&#160; NULL\n2011-02-01&#160;&#160;&#160;&#160; 231&#160;&#160;&#160;&#160; NULL\n2011-03-01&#160;&#160;&#160;&#160; 572&#160;&#160;&#160;&#160; NULL\n2011-04-01&#160;&#160;&#160;&#160; 775&#160;&#160;&#160;&#160; NULL\n2011-05-01&#160;&#160;&#160;&#160; 660&#160;&#160;&#160;&#160; NULL\n2011-06-01&#160;&#160;&#160;&#160; 662&#160;&#160;&#160;&#160; NULL\n2011-07-01&#160;&#160;&#160;&#160; 541&#160;&#160;&#160;&#160; NULL\n2011-08-01&#160;&#160;&#160;&#160; 849&#160;&#160;&#160;&#160; NULL\n2011-09-01&#160;&#160;&#160;&#160; 632&#160;&#160;&#160;&#160; NULL\n2011-10-01&#160;&#160;&#160;&#160; 906&#160;&#160;&#160;&#160; NULL\n2011-11-01&#160;&#160;&#160;&#160; 961&#160;&#160;&#160;&#160; NULL\n2011-12-01&#160;&#160;&#160;&#160; 361&#160;&#160;&#160;&#160; 7776\n2012-01-01&#160;&#160;&#160;&#160; 461&#160;&#160;&#160;&#160; 7611\n2012-02-01&#160;&#160;&#160;&#160; 928&#160;&#160;&#160;&#160; 8308\n2012-03-01&#160;&#160;&#160;&#160; 855&#160;&#160;&#160;&#160; 8591\n2012-04-01&#160;&#160;&#160;&#160; 605&#160;&#160;&#160;&#160; 8421\n2012-05-01&#160;&#160;&#160;&#160; 83&#160;&#160;&#160;&#160;&#160; 7844\n2012-06-01&#160;&#160;&#160;&#160; 44&#160;&#160;&#160;&#160;&#160; 7226\n2012-07-01&#160;&#160;&#160;&#160; 382&#160;&#160;&#160;&#160; 7067\n2012-08-01&#160;&#160;&#160;&#160; 862&#160;&#160;&#160;&#160; 7080\n2012-09-01&#160;&#160;&#160;&#160; 549&#160;&#160;&#160;&#160; 6997\n2012-10-01&#160;&#160;&#160;&#160; 632&#160;&#160;&#160;&#160; 6723\n2012-11-01&#160;&#160;&#160;&#160; 2&#160;&#160;&#160;&#160;&#160;&#160; 5764\n2012-12-01&#160;&#160;&#160;&#160; 26&#160;&#160;&#160;&#160;&#160; 5429\n<\/pre>\n<h3>Solution #2: A More Traditional Approach<\/h3>\n<p>Even though the first solution is able to do a single Index scan of our table,  our suspicions are that using a Tally table may not be the optimal way to solve this problem.&#160; This is mainly because of the second issue we saw, specifically the extra rows that were generated with dates  past the end point of our input data set.&#160; Let&#8217;s look at a couple of more  traditional approaches, the first being an<code> INNER JOIN<\/code>.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Rolling twelve month total by using INNER JOIN\nSELECT a.[Date]\n&#160;&#160;&#160; ,Value=MAX(CASE WHEN a.[Date] = b.[Date] THEN a.Value END)\n&#160;&#160;&#160; ,Rolling12Months=CASE \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHEN ROW_NUMBER() OVER (ORDER BY a.[Date]) &lt; 12 \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; THEN NULL \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ELSE SUM(b.Value) \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; END\nFROM #RollingTotalsExample a\nJOIN #RollingTotalsExample b ON b.[Date] BETWEEN DATEADD(month, -11, a.[Date]) AND a.[Date]\nGROUP BY a.[Date]\nORDER BY a.[Date];\n<\/pre>\n<p>Clearly the above is a much simpler query than our solution using a Tally table,  but still suffers from the first issue of having to force<code> NULL<\/code><code>s<\/code> into the first 11 rows by using <code>ROW_NUMBER()<\/code>.&#160; A quick check of the execution plan indicates that it is making full use of the clustered index that is available  (one scan and one seek).&#160; We can get away with using a <code>BETWEEN <\/code>on our dates  because we&#8217;re looking at closed end points (normal best practice is to use <code>&gt;=<\/code> start point and  <code>&lt;<\/code> end point).<\/p>\n<p>I&#8217;ve never liked queries that <code>JOIN<\/code> a table onto itself; so we are still left  wondering how well this will perform because of what I&#8217;d classify as a semi-<a href=\"http:\/\/www.sqlservercentral.com\/articles\/T-SQL\/61539\/\">triangular  join<\/a>, which as the linked article points out is simply SQL Row-by-Agonizing-Row (RBAR) in disguise.<\/p>\n<p>If you haven&#8217;t already guessed from the other articles I&#8217;ve written, I love the  <code>CROSS APPLY<\/code> construct in SQL.&#160; So let&#8217;s see if we can do something similar  to the <code>INNER JOIN<\/code> approach to get to our twelve month rolling totals.<\/p>\n<h3>Solution #3: Using TOP and CROSS APPLY<\/h3>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Rolling twelve month total by using CROSS APPLY TOP\nSELECT a.[Date]\n&#160;&#160;&#160; ,a.Value\n&#160;&#160;&#160; ,Rolling12Months=CASE \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHEN ROW_NUMBER() OVER (ORDER BY a.[Date]) &lt; 12 \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; THEN NULL \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ELSE a.Value + b.Value \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; END\nFROM #RollingTotalsExample a\nCROSS APPLY\n( \n&#160;&#160;&#160; SELECT Value=SUM(Value)\n&#160;&#160;&#160; FROM\n&#160;&#160;&#160; (\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; SELECT TOP 11 b.[Date], Value \n&#160;&#160;&#160;&#160;&#160;&#160;&#160; FROM #RollingTotalsExample b\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHERE b.[Date] &lt; a.[Date]\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; ORDER BY b.[Date] DESC\n&#160;&#160;&#160; ) b\n) b\nORDER BY a.[Date];\n<\/pre>\n<p>Here we calculate the sum of the prior 11 rows in the <code>CROSS APPLY<\/code>, then add this  to the current row.&#160; Examination of the query plan for this solution  compared to the <code>INNER JOIN <\/code>method shows some differences, so it is possible the performance characteristics will also be  different.&#160; Unfortunately we still had to <code>NULL<\/code> out the first 11 rows with  our<code> ROW_NUMBER() <\/code>as before, but you should convince yourself that the final results are the same.<\/p>\n<h3>Solution #4: Using a Correlated Sub-query<\/h3>\n<p>From the prior method, we can extract the summed Value result from within the<code>  CROSS APPLY<\/code> and put it into a correlated sub-query instead.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Rolling twelve month total by using a correlated sub-query\nSELECT a.[Date]\n&#160;&#160;&#160; ,a.Value\n&#160;&#160;&#160; ,Rolling12Months=\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; CASE \n&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHEN ROW_NUMBER() OVER (ORDER BY a.[Date]) &lt; 12 \n&#160;&#160;&#160;&#160;&#160;&#160;&#160; THEN NULL \n&#160;&#160;&#160;&#160;&#160;&#160;&#160; ELSE a.Value + \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; (\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; SELECT Value=SUM(Value)\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; FROM\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; (\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; SELECT TOP 11 b.[Date], Value \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; FROM #RollingTotalsExample b\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHERE b.[Date] &lt; a.[Date]\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ORDER BY b.[Date] DESC\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ) b\n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; )\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; END\nFROM #RollingTotalsExample a\nORDER BY a.[Date];\n<\/pre>\n<p>This also produces a slight different query plan so we&#8217;ll be interested to see  how its performance results compare to other solutions proposed so far.<\/p>\n<p>So much for traditional solutions, and my apologies if I happened to overlook  one of your favorites, but feel free to code it up and add it to the performance test harness we&#8217;ll present later to see  how it fares.<\/p>\n<h3>Solution #5: Using a Quirky Update<\/h3>\n<p>If you&#8217;ve never heard of the Quirky Update (QU) and how it can be applied to  problems such as running totals, I strongly recommend you have a read of this outstanding article by <a href=\"http:\/\/mvp.microsoft.com\/en-us\/mvp\/Jeff%20Moden-4020758\">SQL MVP<\/a> <a href=\"https:\/\/www.simple-talk.com\/opinion\/opinion-pieces\/jeff-moden-dba-of-the-day\/\">Jeff Moden<\/a>, entitled <a href=\"http:\/\/www.sqlservercentral.com\/articles\/T-SQL\/68467\/\">Solving the Running Total and Ordinal Rank Problems<\/a>.&#160; <\/p>\n<p>Before we continue, we should note that there are those that insist the QU  method represents an undocumented behavior of SQL Server and so is not to be trusted.&#160; We can say that the syntax is clearly described by the <a href=\"http:\/\/technet.microsoft.com\/en-us\/library\/ms177523.aspx\">MS Books On Line entry for the UPDATE statement<\/a>  for SQL versions 2005, 2008 and 2012.&#160; In fact it goes back further than  that. I have successfully used it in SQL Server 2000 but it was inherited from Sybase and was in the first SQL Server  version ever released.&#160; To the naysayers I&#8217;ll say that the &#8220;undocumented&#8221;  behavior is at least consistent across all versions and there is probably little reason to suspect that it will be  deprecated or change in future versions of MS SQL.&#160; Consider yourself  warned!<\/p>\n<p>If you ever consider using a QU to solve any problem, you need to take careful  note of the many rules that apply (also included in the referenced article by Jeff).&#160; The main ones, which I&#8217;ve handled in this query, can be summarized as:<\/p>\n<ul>\n<li>The  table must have a clustered index that indicates the ordering of the source rows by the period as you wish it to be  traversed.<\/li>\n<li>The table must have a column into which you can place the aggregated running total.<\/li>\n<li>When you perform the update, you need to lock the table using the<code> TABLOCKX<\/code> query hint to make sure nobody else gets in  any<code> INSERT<\/code>s, <code>DELETE<\/code>s or<code> UPDATE<\/code>s before you&#8217;re through.<\/li>\n<li>You must prevent SQL from trying to parallelize the query using the <code>OPTION (MAXDOP 1) \t\t<\/code>hint.<\/li>\n<\/ul>\n<p>Since a rolling twelve month average is simply a running total in disguise, we  can add a column to our table and apply a QU query to do our calculation.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Performing a Quirky Update to get our running twelve month totals\nALTER TABLE #RollingTotalsExample ADD Rolling12Months INT NULL;\n&#160;\n-- Change these assignments on DECLARE to a separate statement for SQL 2005\nDECLARE @Lag1&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag2&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag3&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag4&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag5&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag6&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag7&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag8&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag9&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag10&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag11&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@Lag12&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@rt&#160;&#160;&#160;&#160;&#160;&#160;&#160; INT = 0\n&#160;&#160;&#160; ,@rn&#160;&#160;&#160;&#160;&#160;&#160;&#160; INT = NULL;\n&#160;\nUPDATE #RollingTotalsExample WITH(TABLOCKX)\nSET @rt = @rt + Value - @Lag12\n&#160;&#160;&#160; ,@rn = CASE WHEN @rn IS NULL THEN 1 ELSE @rn + 1 END\n&#160;&#160;&#160; ,Rolling12Months = CASE WHEN @rn &gt; 11 THEN @rt END\n&#160;&#160;&#160; ,@Lag12 = @Lag11\n&#160;&#160;&#160; ,@Lag11 = @Lag10\n&#160;&#160;&#160; ,@Lag10 = @Lag9\n&#160;&#160;&#160; ,@Lag9 = @Lag8\n&#160;&#160;&#160; ,@Lag8 = @Lag7\n&#160;&#160;&#160; ,@Lag7 = @Lag6\n&#160;&#160;&#160; ,@Lag6 = @Lag5\n&#160;&#160;&#160; ,@Lag5 = @Lag4\n&#160;&#160;&#160; ,@Lag4 = @Lag3\n&#160;&#160;&#160; ,@Lag3 = @Lag2\n&#160;&#160;&#160; ,@Lag2 = @Lag1\n&#160;&#160;&#160; ,@Lag1 = Value\nOPTION (MAXDOP 1);\n&#160;\nSELECT * FROM #RollingTotalsExample;\n<\/pre>\n<p>I must confess that this does look a little messy, with all of the variables you  need to <code>DECLARE<\/code>.&#160; Basically what we are doing is to keep track of the last  twelve (lagging) values, in order to remove the 12th one (where the <code>Rolling12Months <\/code>column is assigned) from  what is otherwise a QU running total as described in Jeff&#8217;s article. &#160;We have  high hopes for its speed given that it is known to be the fastest method for solving the running totals problem.<\/p>\n<p>Once again, you should convince yourself that the results are consistent with  prior solutions, and yes this solution still behaves the same in SQL 2012.&#160;  If you&#8217;re with me so far, you may also be asking yourself &#8220;what happens if I need to calculate multiple running twelve  month totals across different partitions?&#8221;&#160; This is relatively simple for  all the other solutions presented but does propose a bit of a challenge using the QU.&#160; The answer to this can be found in the attached resource file: <b><i>Quirky Update Partitioned.sql<\/i><\/b>.<\/p>\n<h2>SQL 2012 Solutions<\/h2>\n<p>Until now, everything we have done will work in SQL 2008.&#160; The only thing we&#8217;ve done that is not supported in SQL 2005 is the initializations of the variables we  <code>DECLARE<\/code>d  in the QU approach.&#160; Now let&#8217;s see what new features SQL 2012 has that can  be applied to this problem.<\/p>\n<h3>Solution #6: Using a Window Frame<\/h3>\n<p>Our first SQL 2012 solution (#6) shows how to use a window frame that starts 11  rows prior to the current row, up through the current row to SUM our desired results.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Rolling twelve months totals using SQL 2012 and a window frame\nSELECT [Date], Value\n&#160;&#160;&#160; ,Rolling12Months=CASE WHEN ROW_NUMBER() OVER (ORDER BY [Date]) &gt; 11\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; THEN SUM(Value) OVER (ORDER BY [Date] ROWS BETWEEN 11 PRECEDING AND CURRENT ROW)\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; END\nFROM #MyTable;\n<\/pre>\n<p>We still need to handle the <code>NULL<\/code>s for the first 11 rows specially, but otherwise  this solution is quite neat and concise.<\/p>\n<h3>Solution #7: Using the LAG Analytic Function<\/h3>\n<p>SQL 2012 also offers a new analytic function:<code> LAG<\/code>, which can be used to solve  this problem.&#160;<code> <\/code> <code>LAG <\/code>returns information from a prior row, offset by the  number passed as its second argument.&#160; The main benefit seems to be that  there&#8217;s no need for special handling on the initial 11 rows.<\/p>\n<pre class=\"theme:ssms2012 lang:tsql\">-- Rolling twelve months totals using SQL 2012 and multiple LAGs\nSELECT [Date], Value\n&#160;&#160;&#160; ,Rolling12Months=Value +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 1) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 2) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 3) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 4) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 5) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 6) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 7) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 8) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 9) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 10) OVER (ORDER BY [Date]) +\n&#160;&#160;&#160;&#160;&#160;&#160;&#160; LAG(Value, 11) OVER (ORDER BY [Date])\nFROM #MyTable;\n<\/pre>\n<p>Once again, the returned results are the same but the query plan is quite  different than for the prior SQL 2012 solution; however we&#8217;re not particularly optimistic that this approach will yield  a reasonably performing alternative because of the number of &#8220;look-backs&#8221; needed to make it work.<\/p>\n<h2>Performance Comparison of the Methods<\/h2>\n<p>The real test to see how multiple solutions perform  is to check actual execution times in a quiescent server using a test harness with many rows.&#160; Our test harness is shown, along with how Solution #1 and #2 have been modified (refer to comments in the code)  to:<\/p>\n<ul>\n<li>\n<p>Insert the results into a temp table, to avoid the elapsed time impact of  \t\treturning the rows to SQL Server Management Studio&#8217;s results grid.<\/p>\n<\/li>\n<li>\n<p>Remove the<code> DATE<\/code> arithmetic, because when generating multi-million row test  \t\tharnesses it is difficult to generate that many unique months, so the <code>[Date]<\/code> table column has been revised to be  \t\ta <code>BIGINT<\/code> data type. <\/p>\n<\/li>\n<\/ul>\n<pre class=\"theme:ssms2012 lang:tsql\">\t\tSET NOCOUNT ON;\nCREATE TABLE #RollingTotalsExample\n(\n&#160;&#160;&#160; -- [Date]&#160;&#160;&#160; DATE PRIMARY KEY\n&#160;&#160;&#160; -- Change data type of [Date] to BIGINT \n&#160;&#160;&#160; [Date]&#160;&#160;&#160;&#160; BIGINT PRIMARY KEY\n&#160;&#160;&#160; ,[Value]&#160;&#160; INT\n);\n&#160;\nWITH Tally (n) AS\n(\n&#160;&#160;&#160; SELECT TOP 1000000 ROW_NUMBER() OVER (ORDER BY (SELECT NULL))\n&#160;&#160;&#160; FROM sys.all_columns a CROSS JOIN sys.all_columns b\n)\nINSERT INTO #RollingTotalsExample\nSELECT n, 1+ABS(CHECKSUM(NEWID()))%1000\nFROM Tally;\nPRINT 'Number of test rows: ' + CAST(@@ROWCOUNT AS VARCHAR(12));\n&#160;\nPRINT 'Solution #1 - Tally Table';\nSET STATISTICS TIME ON;\n-- Rolling twelve month totals using a Tally table\nWITH Tally (n) AS\n(\n&#160;&#160;&#160; SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3\n&#160;&#160;&#160; UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7\n&#160;&#160;&#160; UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11\n)\nSELECT GroupingDate\n&#160;&#160;&#160; ,Value=MAX(CASE n WHEN 0 THEN a.Value END)\n&#160;&#160;&#160; ,Rolling12Months=CASE \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHEN ROW_NUMBER() OVER (ORDER BY GroupingDate) &lt; 12 \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; THEN NULL \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ELSE SUM(Value) \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; END\nINTO #Results_Soln1\nFROM #RollingTotalsExample a\nCROSS APPLY Tally b\nCROSS APPLY\n(\n&#160;&#160;&#160; -- Remove the DATE arithmetic\n&#160;&#160;&#160; SELECT GroupingDate=[Date] + n -- DATEADD(month, n, [Date])\n) c\nGROUP BY GroupingDate\nHAVING GroupingDate &lt;= MAX([Date])\nORDER BY GroupingDate;\nSET STATISTICS TIME OFF;\n&#160;\nPRINT 'Solution #2 - INNER JOIN' + CHAR(10);\nSET STATISTICS TIME ON;\n-- Rolling twelve month total by using INNER JOIN\nSELECT a.[Date]\n&#160;&#160;&#160; ,Value=MAX(CASE WHEN a.[Date] = b.[Date] THEN a.Value END)\n&#160;&#160;&#160; ,Rolling12Months=CASE \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHEN ROW_NUMBER() OVER (ORDER BY a.[Date]) &lt; 12 \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; THEN NULL \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ELSE SUM(b.Value) \n&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; END\nINTO #Results_Soln2\nFROM #RollingTotalsExample a\n-- Remove the DATE arithmetic\n--JOIN #RollingTotalsExample b ON b.[Date] BETWEEN DATEADD(month, -11, a.[Date]) AND a.[Date]\nJOIN #RollingTotalsExample b ON b.[Date] BETWEEN a.[Date]-11 AND a.[Date]\nGROUP BY a.[Date];\nSET STATISTICS TIME OFF;\n&#160;\nGO\nDROP TABLE #RollingTotalsExample;\nDROP TABLE #Results_Soln1;\nDROP TABLE #Results_Soln2;\n<\/pre>\n<p>When we run our test harness at 1,000,000 rows, we  get the following raw results, which would seem to eliminate Solutions #1 and #7 as contenders for the top prize in  elapsed execution time.&#160; You may want to also run this once at 4,000,000  rows (excluding solutions #1 and #7) to give SQL a chance to &#8220;learn&#8221; a &#8220;good&#8221; execution plan, prior to attempting to  recreate the test results we&#8217;ll show in a moment.<\/p>\n<pre>Number of test rows: 1000000\n \t&#160;\n \tSolution #1 - Tally Table\n \t&#160;SQL Server Execution Times:\n &#160;&#160; CPU time = 63039 ms,&#160; elapsed time = 188357 ms.\n \t&#160;\n \tSolution #2 - INNER JOIN\n \t&#160;\n \t&#160;SQL Server Execution Times:\n &#160;&#160; CPU time = 9251 ms,&#160; elapsed time = 8003 ms.\n \t&#160;\n \tSolution #3 - CROSS APPLY TOP\n \t&#160;\n \t&#160;SQL Server Execution Times:\n &#160;&#160; CPU time = 5397 ms,&#160; elapsed time = 5421 ms.\n \t&#160;\n \tSolution #4 - Correlated Sub-query\n \t&#160;\n \t&#160;SQL Server Execution Times:\n &#160;&#160; CPU time = 5382 ms,&#160; elapsed time = 5410 ms.\n \t&#160;\n \tSolution #5 - Quirky Update\n \t&#160;\n \t&#160;SQL Server Execution Times:\n &#160;&#160; CPU time = 2792 ms,&#160; elapsed time = 2871 ms.\n \t&#160;\n \tSolution #6 - SQL 2012 Window Frame\n \t&#160;\n \t&#160;SQL Server Execution Times:\n &#160;&#160; CPU time = 4041 ms,&#160; elapsed time = 4073 ms.\n \t&#160;\n \tSolution #7 - SQL 2012 Multiple LAGs\n \t&#160;\n \t&#160;SQL Server Execution Times:\n &#160;&#160; CPU time = 33275 ms,&#160; elapsed time = 33480 ms.<\/pre>\n<p>For the remaining solutions (#2 &#8211; #6), we have  graphed the CPU and Elapsed time results from 1M though 4M rows.<\/p>\n<p>  <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1887-clip_image002.png\" height=\"335\" width=\"601\" alt=\"1887-clip_image002.png\" \/><\/p>\n<p>  <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/imported\/1887-clip_image004.png\" height=\"335\" width=\"601\" alt=\"1887-clip_image004.png\" \/><\/p>\n<h2>Interpreting the Results<\/h2>\n<p>Elapsed and CPU times seem to be consistent across  the different methods with respect to their ordering.&#160; All appear to scale  in a linear fashion.<\/p>\n<p>The Quirky Update, assuming you can understand it and all of its associated  rules, seems to be the fastest available solution to solving this problem, even considering the new features available  in SQL 2012.<\/p>\n<p>In SQL 2012, the window frame approach is certainly neat, compact and elegant,  but slightly trails the Quirky Update solution across the rows we tested.&#160;  These test results seem to conform to an earlier test on <a href=\"http:\/\/blog.waynesheffield.com\/wayne\/archive\/2011\/08\/running-totals-in-denali-ctp3\/\">Running Totals in SQL  &#8220;Denali&#8221; CTP3<\/a> by Microsoft Certified Master <a href=\"http:\/\/www.amazon.com\/Wayne-Sheffield\/e\/B0095Y6OCA\">Wayne  Sheffield<\/a> in his blog.<\/p>\n<p>If you&#8217;re stuck with an earlier version of SQL (2005 or 2008), and for some  reason you can&#8217;t abide using a Quirky Update (e.g., if you don&#8217;t trust this undocumented behavior), the fastest  solutions available to you are either the <code>CROSS APPLY TOP<\/code> or using a correlated sub-query, as both of those seemed to be  in a close tie across the board.<\/p>\n<p>It seems that the &#8220;traditional&#8221;<code> INNER JOIN <\/code>is something to be avoided.&#160; It will probably only get worse if you need to do date arithmetic within the JOIN&#8217;s ON clause.&#160; Likewise, using either a Tally Table or multiple <code>LAG<\/code>s (SQL 2012) certainly was not the way to go.<\/p>\n<p>We did not explore <code>CURSOR<\/code>-based solutions, but you can back track to the article  referenced on running totals to get an idea of how they might perform in this case.&#160; I&#8217;ve also seen some solutions that employ a recursive Common Table Expression (rCTE), but I most certainly  wouldn&#8217;t bet on their performance compared to the QU or window frame solutions.<\/p>\n<p>There are many ways to calculate values within a rolling window in SQL and there  are some clear performance winners among them.&#160; We hope you found this guide  to the available methods interesting and informative.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Before the SQL Window functions were implemented, it was tricky to calculate rolling totals or moving averages efficiently in SQL Server. There are now a number of techniques, but which has the best performance? Dwain Camps gets out the metaphorical stopwatch.&hellip;<\/p>\n","protected":false},"author":221942,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[143531],"tags":[4150,4151,4252],"coauthors":[],"class_list":["post-1713","post","type-post","status-publish","format-standard","hentry","category-t-sql-programming-sql-server","tag-sql","tag-sql-server","tag-t-sql-programming"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/1713","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/221942"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=1713"}],"version-history":[{"count":6,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/1713\/revisions"}],"predecessor-version":[{"id":92530,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/1713\/revisions\/92530"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=1713"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=1713"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=1713"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=1713"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}