{"id":90268,"date":"2021-03-15T17:00:12","date_gmt":"2021-03-15T17:00:12","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=90268"},"modified":"2021-03-27T11:56:26","modified_gmt":"2021-03-27T11:56:26","slug":"count-distinct-and-window-functions","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/blogs\/count-distinct-and-window-functions\/","title":{"rendered":"Count Distinct and Window Functions"},"content":{"rendered":"<p>Or: How to make magic tricks with T-SQL<\/p>\n<p>Starting our magic show, let\u2019s first set the stage:<\/p>\n<p style=\"text-align: center\"><strong><em>Count Distinct doesn\u2019t work with Window Partition<\/em><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"401\" height=\"100\" class=\"wp-image-90269 aligncenter\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2021\/03\/word-image-2.png\" \/><\/p>\n<h2>Preparing the example<\/h2>\n<p style=\"padding-left: 30px\">In order to reach the conclusion above and solve it, let\u2019s first build a scenario. Using <strong>Azure SQL Database<\/strong>, we can create a sample database called <em>AdventureWorksLT<\/em>, a small version of the old sample <em>AdventureWorks<\/em> databases.<\/p>\n<p style=\"padding-left: 30px\">Let\u2019s use the tables <em>Product<\/em> and <em>SalesOrderDetail<\/em>, both in <em>SalesLT<\/em> schema. Each order detail row is part of an order and is related to a product included in the order. The product has a category and color.<\/p>\n<p style=\"padding-left: 30px\">One interesting query to start is this one:<\/p>\n<div style=\"padding-left: 60px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: silver\">*<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">unitprice<\/span>\u00a0<span style=\"color: silver\">*<\/span>\u00a0<span style=\"color: maroon\">orderqty<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">Total<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span>\u00a0<span style=\"color: maroon\">p<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">INNER<\/span>\u00a0<span style=\"color: blue\">JOIN<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">s<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">p<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span>\u00a0<span style=\"color: silver\">=<\/span>\u00a0<span style=\"color: maroon\">s<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span> <br \/>\n<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span>\u00a0 <\/span><\/div>\n<p style=\"padding-left: 30px\">This query results in the count of items on each order and the total value of the order.<\/p>\n<p style=\"padding-left: 30px\">Let\u2019s add some more calculations to the query, none of them poses a challenge:<\/p>\n<div style=\"padding-left: 60px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: silver\">*<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">unitprice<\/span>\u00a0<span style=\"color: silver\">*<\/span>\u00a0<span style=\"color: maroon\">orderqty<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">CategoriesPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span>\u00a0<span style=\"color: maroon\">p<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">INNER<\/span>\u00a0<span style=\"color: blue\">JOIN<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">s<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">p<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span>\u00a0<span style=\"color: silver\">=<\/span>\u00a0<span style=\"color: maroon\">s<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span> <br \/>\n<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span>\u00a0 <\/span><\/div>\n<p style=\"padding-left: 30px\">I included the total of different categories and colours on each order.<\/p>\n<h2>Identifying the Problem<\/h2>\n<p style=\"padding-left: 30px\">Now, let\u2019s imagine that, together this information, we also would like to know the number of distinct colours by category there are in this order.<\/p>\n<p style=\"padding-left: 30px\">The group by only has the <em>SalesOrderId.<\/em>\u00a0Due to that, our first natural conclusion is to try a window partition, like this one:<\/p>\n<div style=\"padding-left: 60px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: silver\">*<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">unitprice<\/span>\u00a0<span style=\"color: silver\">*<\/span>\u00a0<span style=\"color: maroon\">orderqty<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">CategoriesPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">OVER<\/span>\u00a0<span style=\"color: maroon\">(<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">partition<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span>\u00a0<span style=\"color: maroon\">p<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">INNER<\/span>\u00a0<span style=\"color: blue\">JOIN<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">s<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">p<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span>\u00a0<span style=\"color: silver\">=<\/span>\u00a0<span style=\"color: maroon\">s<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span> <br \/>\n<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span>\u00a0 <\/span><\/div>\n<p style=\"padding-left: 30px\">\nOur problem starts with this query. <strong><em>Count Distinct is not supported by window partitioning<\/em><\/strong>, we need to find a different way to achieve the same result.<\/p>\n<h2>Planning the Solution<\/h2>\n<p style=\"padding-left: 30px\">We are counting the rows, so we can use <strong>DENSE_RANK<\/strong> to achieve the same result, extracting the last value in the end, we can use a <strong>MAX<\/strong> for that. This works in a similar way as the distinct count because all the ties, the records with the same value, receive the same rank value, so the biggest value will be the same as the distinct count.<\/p>\n<p style=\"padding-left: 30px\">There are two ranking functions: <strong>RANK<\/strong> and <strong>DENSE_RANK<\/strong>. The difference is how they deal with ties.<\/p>\n<p style=\"padding-left: 60px\"><strong>RANK:<\/strong> After a tie, the count jumps the number of tied items, leaving a hole.<\/p>\n<p style=\"padding-left: 60px\"><strong>DENSE_RANK:<\/strong> No jump after a tie, the count continues sequentially<\/p>\n<p style=\"padding-left: 30px\">The following query makes an example of the difference:<\/p>\n<div style=\"padding-left: 60px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">productid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: #ff0080;font-weight: bold\">Rank<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">OVER<\/span>\u00a0<span style=\"color: maroon\">(<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ORDER<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">[rank]<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: #ff0080;font-weight: bold\">Dense_rank<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">OVER<\/span>\u00a0<span style=\"color: maroon\">(<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ORDER<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">[dense_rank]<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span> <br \/>\n<span style=\"color: blue\">WHERE<\/span>\u00a0\u00a0<span style=\"color: maroon\">color<\/span>\u00a0<span style=\"color: blue\">IS<\/span>\u00a0<span style=\"color: blue\">NOT<\/span>\u00a0<span style=\"color: blue\">NULL<\/span>\u00a0 <\/span><\/div>\n<p style=\"padding-left: 30px\"><img loading=\"lazy\" decoding=\"async\" width=\"325\" height=\"241\" class=\"wp-image-90270\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2021\/03\/word-image-3.png\" \/><\/p>\n<p style=\"padding-left: 30px\">The new query using <strong>DENSE_RANK<\/strong> will be like this:<\/p>\n<div style=\"padding-left: 60px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: silver\">*<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">unitprice<\/span>\u00a0<span style=\"color: silver\">*<\/span>\u00a0<span style=\"color: maroon\">orderqty<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">CategoriesPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: #ff0080;font-weight: bold\">Dense_rank<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">OVER<\/span>\u00a0<span style=\"color: maroon\">(<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">partition<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ORDER<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">color<\/span>\u00a0<span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span>\u00a0<span style=\"color: maroon\">p<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">INNER<\/span>\u00a0<span style=\"color: blue\">JOIN<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">s<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">p<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span>\u00a0<span style=\"color: silver\">=<\/span>\u00a0<span style=\"color: maroon\">s<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span> <br \/>\n<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span>\u00a0 <\/span><\/div>\n<p style=\"padding-left: 30px\">However, the result is not what we would expect:<\/p>\n<p style=\"padding-left: 30px\"><img loading=\"lazy\" decoding=\"async\" width=\"1132\" height=\"118\" class=\"wp-image-90271\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2021\/03\/word-image-4.png\" \/><\/p>\n<p style=\"padding-left: 30px\">The groupby and the over clause don\u2019t work perfectly together. The fields used on the over clause need to be included in the group by as well, so the query doesn\u2019t work.<\/p>\n<h2>Solving the Solution<\/h2>\n<p>The first step to solve the problem is to add more fields to the group by. Of course, this will affect the entire result, it will not be what we really expect. The query will be like this:<\/p>\n<div style=\"padding-left: 30px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: silver\">*<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">unitprice<\/span>\u00a0<span style=\"color: silver\">*<\/span>\u00a0<span style=\"color: maroon\">orderqty<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: black\">1<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: #ff0080;font-weight: bold\">Dense_rank<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">OVER<\/span>\u00a0<span style=\"color: maroon\">(<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">partition<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ORDER<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span>\u00a0<span style=\"color: maroon\">p<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">INNER<\/span>\u00a0<span style=\"color: blue\">JOIN<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">s<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">p<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span>\u00a0<span style=\"color: silver\">=<\/span>\u00a0<span style=\"color: maroon\">s<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span> <br \/>\n<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">color<\/span>\u00a0 <\/span><\/div>\n<p>There are two interesting changes on the calculation:<\/p>\n<ul>\n<li><em>CategoriesPerOrder<\/em> was removed, because the group by is some levels below this calculation, we can leave this for later.<\/li>\n<li><em>ColorPerOrder<\/em> is a fixed value, because we are grouping by colour.<\/li>\n<\/ul>\n<p>We need to make further calculations over the result of this query, the best solution for this is the use of <strong>CTE \u2013 Common Table Expressions<\/strong>.<\/p>\n<h3>2<sup>nd<\/sup> Query Level<\/h3>\n<p>The 2<sup>nd<\/sup> level of calculations will aggregate the data by <em>ProductCategoryId<\/em>, removing one of the aggregation levels.<\/p>\n<div style=\"padding-left: 30px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: green;font-style: italic\">;<\/span><span style=\"color: blue\">WITH<\/span>\u00a0<span style=\"color: maroon\">ranking<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">(<\/span><span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: silver\">*<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">unitprice<\/span>\u00a0<span style=\"color: silver\">*<\/span>\u00a0<span style=\"color: maroon\">orderqty<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: #ff0080;font-weight: bold\">Dense_rank<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">OVER<\/span>\u00a0<span style=\"color: maroon\">(<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">partition<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ORDER<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span>\u00a0<span style=\"color: maroon\">p<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">INNER<\/span>\u00a0<span style=\"color: blue\">JOIN<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">s<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">p<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span>\u00a0<span style=\"color: silver\">=<\/span>\u00a0<span style=\"color: maroon\">s<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n<span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">itemsperorder<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">total<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: black\">1<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">CategoriesPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">colorperorder<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Max<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">colorpercategory<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">ranking<\/span> <br \/>\n<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span>\u00a0 <\/span><\/div>\n<p>The calculations on the 2<sup>nd<\/sup> query are defined by how the aggregations were made on the first query:<\/p>\n<ul>\n<li><strong>ItemsPerOrder:<\/strong> We make a SUM on the results of the COUNT, this will aggregate the different counts.<\/li>\n<li><strong>Total:<\/strong> A simple <strong>SUM<\/strong> over the <strong>SUM<\/strong> already made.<\/li>\n<li><strong>CategoriesPerOrder:<\/strong> It can have a fixed number of 1, since we are still aggregating per category<\/li>\n<li><strong>ColorPerOrder:<\/strong> We make a <strong>SUM<\/strong> over the already existing <strong>COUNT<\/strong> from the previous query<\/li>\n<li><strong>ColorPerCategory:<\/strong> After making the <strong>DENSE_RANK<\/strong>, now we need to extract the <strong>MAX<\/strong> value to have the same effect as the <strong>COUNT DISTINCT<\/strong><\/li>\n<\/ul>\n<h3>3<sup>rd<\/sup> Query Level<\/h3>\n<p>On the 3<sup>rd<\/sup> step we reduce the aggregation, achieving our final result, the aggregation by <em>SalesOrderId<\/em><\/p>\n<div style=\"padding-left: 30px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: green;font-style: italic\">;<\/span><span style=\"color: blue\">WITH<\/span>\u00a0<span style=\"color: maroon\">cte<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">(<\/span><span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: silver\">*<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">unitprice<\/span>\u00a0<span style=\"color: silver\">*<\/span>\u00a0<span style=\"color: maroon\">orderqty<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Count<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: blue\">DISTINCT<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: #ff0080;font-weight: bold\">Dense_rank<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">OVER<\/span>\u00a0<span style=\"color: maroon\">(<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">partition<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span>\u00a0<span style=\"color: maroon\">productcategoryid<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ORDER<\/span>\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span>\u00a0<span style=\"color: maroon\">p<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">INNER<\/span>\u00a0<span style=\"color: blue\">JOIN<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">s<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">p<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span>\u00a0<span style=\"color: silver\">=<\/span>\u00a0<span style=\"color: maroon\">s<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">productid<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">cte2<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">AS<\/span>\u00a0<span style=\"color: maroon\">(<\/span><span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">itemsperorder<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">total<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: black\">1<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">CategoriesPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">colorperorder<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Max<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">colorpercategory<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">cte<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n<span style=\"color: blue\">SELECT<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">itemsperorder<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ItemsPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">total<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">Total<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">categoriesperorder<\/span><span style=\"color: maroon\">)<\/span>\u00a0<span style=\"color: maroon\">CategoriesPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">colorperorder<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerOrder<\/span><span style=\"color: silver\">,<\/span> <br \/>\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<span style=\"color: fuchsia;font-style: italic\">Sum<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">colorpercategory<\/span><span style=\"color: maroon\">)<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">ColorPerCategory<\/span> <br \/>\n<span style=\"color: blue\">FROM<\/span>\u00a0\u00a0\u00a0<span style=\"color: maroon\">cte<\/span> <br \/>\n<span style=\"color: blue\">GROUP<\/span>\u00a0\u00a0<span style=\"color: blue\">BY<\/span>\u00a0<span style=\"color: maroon\">salesorderid<\/span>\u00a0 <\/span><\/div>\n<p>Once again, the calculations are based on the previous queries. Some of them are the same of the 2nd query, aggregating more the rows. However, there are some different calculations:<\/p>\n<ul>\n<li><strong>CategoriesPerOrder:<\/strong> It becomes a SUM to achieve the result we would like<\/li>\n<li><strong>ColorPerCategory:<\/strong> It becomes a SUM, adding all the distinct count results of each category<\/li>\n<\/ul>\n<h2>The Execution Plan<\/h2>\n<p style=\"padding-left: 30px\">The execution plan generated by this query is not too bad as we could imagine. This query could benefit from additional indexes and improve the JOIN, but besides that, the plan seems quite ok.<\/p>\n<p style=\"padding-left: 30px\">There are other options to achieve the same result, but after trying them the query plan generated was way more complex.<\/p>\n<p style=\"padding-left: 30px\"><strong><img loading=\"lazy\" decoding=\"async\" width=\"1853\" height=\"279\" class=\"wp-image-90272\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2021\/03\/word-image-5.png\" \/><\/strong><\/p>\n<p style=\"padding-left: 30px\">The join is made by the field <em>ProductId<\/em>, so an index on <em>SalesOrderDetail<\/em> table by <em>ProductId<\/em> and covering the additional used fields will help the query<\/p>\n<p style=\"padding-left: 30px\">We can create the index with this statement:<\/p>\n<div style=\"padding-left: 60px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">CREATE<\/span>\u00a0<span style=\"color: blue\">INDEX<\/span>\u00a0<span style=\"color: maroon\">indorderdetail<\/span> <br \/>\n\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">salesorderdetail<\/span>\u00a0<span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">productid<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0<span style=\"color: maroon\">include<\/span>\u00a0<span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">orderqty<\/span><span style=\"color: silver\">,<\/span>\u00a0<span style=\"color: maroon\">unitprice<\/span><span style=\"color: maroon\">)<\/span>\u00a0 <\/span><\/div>\n<p style=\"padding-left: 30px\">You may notice on the new query plan the join is converted to a merge join, but the Clustered Index Scan still takes 70% of the query<\/p>\n<p style=\"padding-left: 30px\"><img loading=\"lazy\" decoding=\"async\" width=\"1810\" height=\"241\" class=\"wp-image-90273\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2021\/03\/word-image-6.png\" \/><\/p>\n<p style=\"padding-left: 30px\">The secret is that a covering index for the query will be a smaller number of pages than the clustered index, improving even more the query. The statement for the new index will be like this:<\/p>\n<div style=\"padding-left: 60px\"><span style=\"font-family: Courier New;font-size: 10pt\"> <span style=\"color: blue\">CREATE<\/span>\u00a0<span style=\"color: blue\">INDEX<\/span>\u00a0<span style=\"color: maroon\">indproduct<\/span> <br \/>\n\u00a0\u00a0<span style=\"color: blue\">ON<\/span>\u00a0<span style=\"color: maroon\">saleslt<\/span><span style=\"color: silver\">.<\/span><span style=\"color: maroon\">product<\/span><span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">productid<\/span><span style=\"color: maroon\">)<\/span> <br \/>\n\u00a0\u00a0<span style=\"color: maroon\">include<\/span>\u00a0<span style=\"color: maroon\">(<\/span><span style=\"color: maroon\">productcategoryid<\/span><span style=\"color: silver\">,<\/span>\u00a0<span style=\"color: maroon\">color<\/span><span style=\"color: maroon\">)<\/span>\u00a0 <\/span><\/div>\n<p style=\"padding-left: 30px\"><img loading=\"lazy\" decoding=\"async\" width=\"1815\" height=\"221\" class=\"wp-image-90274\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2021\/03\/word-image-7.png\" \/><\/p>\n<p style=\"padding-left: 30px\">What\u2019s interesting to notice on this query plan is the SORT, now taking 50% of the query. This doesn\u2019t mean the execution time of the SORT changed, this means the execution time for the entire query reduced and the SORT became a higher percentage of the total execution time.<\/p>\n<h2>Further Learning<\/h2>\n<p style=\"padding-left: 30px\">There will be T-SQL sessions on the Malta Data Saturday Conference, on April 24, <a href=\"https:\/\/mmdpug.azurewebsites.net\/\">register now<\/a><\/p>\n<p style=\"padding-left: 30px\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/t-sql\/queries\/select-over-clause-transact-sql?view=sql-server-ver15&amp;WT.mc_id=DP-MVP-4014132\">Select \u2013 Over clause<\/a><\/p>\n<p style=\"padding-left: 30px\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/t-sql\/queries\/with-common-table-expression-transact-sql?view=sql-server-ver15&amp;WT.mc_id=DP-MVP-4014132\">Common Table Expressions<\/a><\/p>\n<p style=\"padding-left: 30px\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/learn\/modules\/describe-sql-server-query-plans\/?WT.mc_id=DP-MVP-4014132\">Describe SQL Server Query Plans<\/a><\/p>\n<p style=\"padding-left: 30px\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/learn\/paths\/optimize-query-performance-sql-server\/?WT.mc_id=DP-MVP-4014132\">Optimize Query Performance in SQL Server<\/a><\/p>\n<h2>Conclusion<\/h2>\n<p style=\"padding-left: 30px\">Mastering modern T-SQL syntaxes, such as CTE\u2019s and Windowing can lead us to interesting magic tricks and improve our productivity<\/p>\n<p style=\"padding-left: 30px\">\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Or: How to make magic tricks with T-SQL Starting our magic show, let\u2019s first set the stage: Count Distinct doesn\u2019t work with Window Partition Preparing the example In order to reach the conclusion above and solve it, let\u2019s first build a scenario. Using Azure SQL Database, we can create a sample database called AdventureWorksLT, a&#8230;&hellip;<\/p>\n","protected":false},"author":50808,"featured_media":90463,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2],"tags":[124955,4162,137126,137125,4190],"coauthors":[6810],"class_list":["post-90268","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blogs","tag-dataplatform","tag-microsoft","tag-mvpbuzz","tag-sqlserver","tag-tsql"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/90268","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/50808"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=90268"}],"version-history":[{"count":1,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/90268\/revisions"}],"predecessor-version":[{"id":90275,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/90268\/revisions\/90275"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media\/90463"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=90268"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=90268"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=90268"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=90268"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}