{"id":519,"date":"2009-02-17T00:00:00","date_gmt":"2009-02-17T00:00:00","guid":{"rendered":"https:\/\/test.simple-talk.com\/uncategorized\/divided-we-stand-the-sql-of-relational-division\/"},"modified":"2021-09-29T16:22:07","modified_gmt":"2021-09-29T16:22:07","slug":"divided-we-stand-the-sql-of-relational-division","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/t-sql-programming-sql-server\/divided-we-stand-the-sql-of-relational-division\/","title":{"rendered":"Divided We Stand: The SQL of Relational Division"},"content":{"rendered":"
\n

I’ve just got a copy of THE MANGA GUIDE TO DATABASES (ISBN 978-1-59327-190-9); it is one of a series of “Manga Guides” (the others are Calculus and Statistics).  Since I consider myself a long time comic book fan, rather than a Dummy or an Idiot, I prefer these titles over their competition on the book shelf.  But beside the risk that the title poses to my personal dignity, the book is a really good introductory book.  You just have to get over the Japanese ‘database fairy’ Tico in the Kingdom of Kod.  <\/p>\n

The book has a section on basic RDBMS in which they mention Codd’s eight original operations on tables.  If you don’t remember them, they were the three classic set operators: UNION, INTERSECTION and DIFFERENCE and five relationship operators: JOIN, PROJECTION, SELECTION, CARTESIAN PRODUCT and DIVISION.  Since RDBMS is based on sets, you can see why the classic set operators are there.  What is hard to figure out is why it took so long to get INTERSECTION and DIFFERENCE into SQL.  We had JOIN, PROJECTION, SELECTION and CARTESIAN PRODUCT right away in SQL because you cannot do anything without them in an RDBMS. <\/p>\n

But the one in the collection that seems weird on first glance is relational division.  It can be expressed in terms of other operators, but the other seven are relatively primitive operations.  The idea is that a divisor table is used to partition a dividend table and produce a quotient or results table.  The quotient table is made up of those values of one column for which a second column had all of the values in the divisor. <\/p>\n

The name “relational division” comes from the symbol for a Cartesian product (aka CROSS JOIN), which is X or multiplication.  If you take the quotient table cross joined with the divisor table you get the dividend table.  In notation we have (quotient CROSS JOIN divisor = dividend) is like (a\/b = c) implies (b * c = a) in integer maths. <\/p>\n

This is easier to explain with an example.  We have a table of pilots and the planes they can fly (dividend); we have a table of planes in the hangar (divisor); we want the names of the pilots who can fly every plane (quotient) in the hangar.  To get this result, we divide the PilotSkills table by the planes in the hangar.<\/p>\n

CREATE TABLE PilotSkills\n(pilot_name CHAR(15) NOT NULL,\nplane_name CHAR(15) NOT NULL,\nPRIMARY KEY (pilot_name, plane_name));\n\n\/*\nPilotSkills\npilot_name    plane_name\n=========================\n'Celko'    'Piper Cub'\n'Higgins'  'B-52 Bomber'\n'Higgins'  'F-14 Fighter'\n'Higgins'  'Piper Cub'\n'Jones'    'B-52 Bomber'\n'Jones'    'F-14 Fighter'\n'Smith'    'B-1 Bomber'\n'Smith'    'B-52 Bomber'\n'Smith'    'F-14 Fighter'\n'Wilson'   'B-1 Bomber'\n'Wilson'   'B-52 Bomber'\n'Wilson'   'F-14 Fighter'\n'Wilson'   'F-17 Fighter' *\/\nCREATE TABLE Hangar\n(plane_name CHAR(15) NOT NULL PRIMARY KEY);\n\/*\nHangar\nplane_name\n=============\n'B-1 Bomber'\n'B-52 Bomber'\n'F-14 Fighter'\n\nPilotSkills DIVIDED BY Hangar\npilot_name\n=============================\n'Smith'\n'Wilson'\n*\/<\/pre>\n
In this example, Smith and Wilson are the two pilots who can fly everything in the hangar.  Notice that both Higgins and Celko know how to fly a Piper Cub, but we don’t have one right now.  In Codd’s original definition of relational division, it is not a problem to have more rows than are called for.<\/p>\n
Division with a Remainder<\/b><\/h2>\n
There are two kinds of relational division.  Division with a remainder allows the dividend table to have more values than the divisor, which was Dr. Codd’s original definition.  For example, if a pilot can fly more planes than just those we have in the hangar, this is fine with us.  The query can be written as …<\/p>\n
SELECT DISTINCT pilot_name\n  FROM PilotSkills AS PS1 \n  WHERE NOT EXISTS\n       (SELECT *\n          FROM Hangar\n         WHERE NOT EXISTS\n               (SELECT *\n                  FROM PilotSkills AS PS2\n                 WHERE (PS1.pilot_name = PS2.pilot_name)\n                   AND (PS2.plane_name = Hangar.plane_name)));\n<\/pre>\n
The quickest way to explain what is happening in this query is to imagine a World War II movie where a cocky pilot has just walked into the hangar, looked over the fleet, and announced, “There ain’t no planes in this hangar that I can’t fly!”  We want to find pilots for whom no plane exists in the hangar for which they have no skills.  You might want to read that double negative again – it is ugly English, but good logic. <\/p>\n
This query for relational division was made popular by Chris Date in his textbooks, but it is neither the only method nor always the fastest.  Another version of the division can be written so as to avoid three levels of nesting.  While it is not original with me, I have made it popular in my books.<\/p>\n
 SELECT PS1.pilot_name\n   FROM PilotSkills AS PS1, Hangar AS H1\n  WHERE PS1.plane_name = H1.plane_name\n  GROUP BY PS1.pilot_name \n  HAVING COUNT(PS1.plane_name) = (SELECT COUNT(plane_name) FROM Hangar);\n\n<\/pre>\n
There is a serious difference in the two methods.  Burn down the hangar, so that the divisor is empty.  Because of the NOT EXISTS() <\/code>predicates in Date’s query, all pilots are returned from a division by an empty set.  Because of the COUNT() <\/code>functions in my query, no pilots are returned from a division by an empty set.  <\/p>\n
Now we are getting philosophical as to how we want to maintain the “integer division” analogy.  Is an empty set “kind of like a zero” or not?  If so, then dividing by zero would be undefined (or infinity, depending on your math book and your age).  And dividing zero by anything is always zero.  <\/p>\n
In the sixth edition of his book, INTRODUCTION TO DATABASE SYSTEMS (Addison-Wesley; 1995; ISBN 0-191-82458-2), Chris Date defined another operator (DIVIDEBY …  PER) which produces the same results as my query, but with more complexity.<\/p>\n
Exact Division<\/b><\/h2>\n
The second kind of relational division is exact relational division.  The dividend table must match exactly to the values of the divisor without any extra values – i.e. a remainder, if you remember grade school math.<\/p>\n
SELECT PS1.pilot_name\n  FROM PilotSkills AS PS1\n       LEFT OUTER JOIN\n       Hangar AS H1\n       ON PS1.plane_name = H1.plane_name \n  GROUP BY PS1.pilot_name\nHAVING COUNT(PS1.plane_name) = (SELECT COUNT(plane_name) FROM Hangar)\n   AND COUNT(H1.plane_name) = (SELECT COUNT(plane_name) FROM Hangar);<\/pre>\n
This says that a pilot must have the same number of certificates as there planes in the hangar and these certificates all match to a plane in the hangar, not something else.  The “something else” is shown by a created NULL from the LEFT OUTER JOIN.<\/p>\n
Please do not make the mistake of trying to reduce the HAVING clause with a little false relational algebra to:<\/p>\n
HAVING COUNT(PS1.plane_name) = COUNT(H1.plane_name) <\/pre>\n
because it does not work; it will tell you that the hangar has (n) planes in it and the pilot_name<\/b> is certified for (n) planes, but not that those two sets of planes are equal to each other.<\/p>\n
Todd’s Division<\/b><\/h2>\n
A relational division operator proposed by Stephen Todd is defined on two tables with common columns that are joined together, dropping the JOIN column and retaining only those non-JOIN columns that meet a criterion.  Again, this is easier to explain with an example.  <\/p>\n
We are given a table, JobParts(job_nbr, part_nbr)<\/b>, and another table, SupParts(sup_nbr, part_nbr),<\/b> of suppliers and the parts that they provide.  We want to get the supplier-and-job pairs such that supplier ‘sn’ supplies all of the parts needed for ‘jn’.  This is not quite the same thing as getting the supplier-and-job pairs such that job_nbr ‘jn’ requires all of the parts provided by supplier sn.<\/p>\n
You want to divide the JobParts<\/b> table by the SupParts<\/b> table.  A rule of thumb: The remainder comes from the dividend, but all values in the divisor are present.<\/p>\n
 JobParts           SupParts             Result AS JobSups\n job_nbr part_nbr   sup_nbr  part_nbr    job_nbr sup_nbr\n ================   =================    ==============\n 'j1'  'p1'         's1'    'p1'           'j1'  's1'\n 'j1'  'p2'         's1'    'p2'           'j1'  's2'\n 'j2'  'p2'         's1'    'p3'           'j2'  's1'\n 'j2'  'p4'         's1'    'p4'           'j2'  's4'\n 'j2'  'p5'         's1'    'p5'           'j3'  's1'\n 'j3'  'p2'         's1'    'p6'           'j3'  's2'\n                    's2'    'p1'           'j3'  's3'\n                    's2'    'p2'           'j3'  's4'\n                    's3'    'p2'\n                    's4'    'p2'\n                    's4'    'p4'\n                    's4'    'p5'\n<\/pre>\n
Pierre Mullin submitted the following query to carry out the Todd division:<\/p>\n
SELECT DISTINCT JP1.job_nbr, SP1.sup_nbr_nbr\n  FROM JobParts AS JP1, SupParts AS SP1\n WHERE NOT EXISTS\n        (SELECT *\n           FROM JobParts AS JP2\n          WHERE JP2.job_nbr = JP1.job_nbr\n            AND JP2.part_nbr\n                NOT IN (SELECT SP2.part_nbr\n                          FROM SupParts AS SP2\n                         WHERE SP2.sup_nbr_nbr = SP1.sup_nbr_nbr));<\/pre>\n
This is really a modification of the query for Codd’s division, extended to use a JOIN on both tables in the outermost SELECT statement.  The IN predicate for the second subquery can be replaced with a NOT EXISTS predicate; it might run a bit faster, depending on the optimizer.<\/p>\n
Another related query is finding the pairs of suppliers who sell the same parts.  In this data, that would be the pairs (‘s1’, ‘p2’), (‘s3’, ‘p1’), (‘s4’, ‘p1’), (‘s5’, ‘p1’)<\/p>\n
SELECT S1.sup_nbr, S2.sup_nbr\n  FROM SupParts AS S1, SupParts AS S2\n WHERE S1.sup_nbr < S2.sup_nbr -- different suppliers\n   AND S1.part_nbr = S2.part_nbr -- same parts\n GROUP BY S1.sup_nbr, S2.sup_nbr\nHAVING COUNT(*) = (SELECT COUNT (*)  -- same count of parts\n                     FROM SupParts AS S3\n                    WHERE S3.sup_nbr = S1.sup_nbr)\n   AND COUNT(*) = (SELECT COUNT (*)\n                     FROM SupParts AS S4\n                    WHERE S4.sup_nbr = S2.sup_nbr);<\/pre>\n
This can be modified into Todd’s division easily be adding the restriction that the parts must also belong to a common job_nbr.  <\/p>\n
Division with JOINs<\/b><\/h2>\n
Standard SQL has several JOIN operators that can be used to perform a relational division.  Going back to my World War II movie, to find the pilots, who can fly the same planes as Higgins, use this query:<\/p>\n
SELECT SP1.Pilot_name\n  FROM (((SELECT plane_name FROM Hangar) AS H1\n         INNER JOIN\n         (SELECT pilot_name, plane_name FROM PilotSkills) AS SP1\n         ON H1.plane_name = SP1.plane_name)\n       INNER JOIN (SELECT *\n                     FROM PilotSkills\n                    WHERE pilot_name = 'Higgins') AS H2\n       ON H2.plane_name = H1.plane_name)\n       GROUP BY SP1.Pilot_name\nHAVING COUNT(*) >= (SELECT COUNT(*)\n                      FROM PilotSkills\n                     WHERE pilot_name = 'Higgins');<\/pre>\n
The first JOIN finds all of the planes in the hangar for which we have a pilot.  The next JOIN takes that set and finds which of those match up with (SELECT * FROM PilotSkills WHERE pilot_name = 'Higgins')<\/code> skills.  The GROUP BY<\/code> clause will then see that the intersection we have formed with the joins has at least as many elements as Higgins has planes.  The GROUP BY<\/code> also means that the SELECT DISTINCT<\/code> can be replaced with a simple SELECT<\/code>.  If the theta operator in the GROUP BY clause is changed from >= to =, the query finds an exact division.  If the theta operator in the GROUP BY clause is changed from >= to &l t;= or <, the query finds those pilots whose skills are a superset or a strict superset of the planes that Higgins flies.  This idea is useful when we get to Romley’s Davison at the end of this article. <\/p>\n
It might be a good idea to put the divisor into a VIEW or CTE for readability in this query and as a clue to the optimizer to calculate it once.  Some products will execute this form of the division query faster than the nested subquery version, because they will use the PRIMARY KEY information to pre-compute the joins between tables.<\/p>\n
Division with Set Operators<\/b><\/h2>\n
The Standard SQL set difference operator, EXCEPT, can be used to write a very compact version of Dr. Codd’s relational division.  The EXCEPT operator removes the divisor set from the dividend set.  If the result is empty, we have a match; if there is anything left over, it has failed.  Using the pilots-and-hangar-tables example, we would write<\/p>\n
SELECT DISTINCT Pilot_name FROM PilotSkills AS P1\nWHERE (SELECT plane_name FROM Hangar\n        EXCEPT\n        SELECT plane_name\n          FROM PilotSkills AS P2\n         WHERE P1.pilot_name = P2.pilot_name) IS NULL;<\/pre>\n
Again, informally, you can imagine that we got a skill list from each pilot, walked over to the hangar, and crossed off each plane he could fly.  If we marked off all the planes in the hangar, we would keep this guy.  Another trick is that an empty subquery expression returns a NULL, which is how we can test for an empty set.  The WHERE<\/code> clause could just as well have used a NOT EXISTS() <\/code>predicate instead of the IS NULL<\/code> predicate.<\/p>\n
Romley’s Division<\/b><\/h2>\n
This somewhat complicated relational division is due to Richard Romley at Salomon Smith Barney.  The original problem deals with two tables.  The first table has a list of managers and the projects they can manage.  The second table has a list of Personnel, their departments and the project to which they are assigned.  Each employee is assigned to one and only one department and each employee works on one and only one project at a time.  But a department can have several different projects at the same time, so a single project can span several departments.<\/p>\n
CREATE TABLE MgrProjects\n(mgr_name CHAR(10) NOT NULL,\n project_id CHAR(2) NOT NULL,\n PRIMARY KEY(mgr_name, project_id));\nINSERT INTO MgrProject \nVALUES ('M1', 'P1'), ('M1', 'P3'),\n       ('M2', 'P2'), ('M2', 'P3'),\n       ('M3', 'P2'),\n       ('M4', 'P1'), ('M4', 'P2'), ('M4', 'P3');\nCREATE TABLE Personnel\n(emp_id CHAR(10) NOT NULL,\n dept_id CHAR(2) NOT NULL,\n project_id CHAR(2) NOT NULL,\n UNIQUE (emp_id, project_id),\n UNIQUE (emp_id, dept),\n PRIMARY KEY (emp_id, dept_id, project_id));\n-- load department #1 data\nINSERT INTO Personnel \nVALUES ('Albert', 'D1', 'P1'),\n       ('Bob', 'D1', 'P1'),\n       ('Carl', 'D1', 'P1'),\n       ('Don', 'D1', 'P2'),\n       ('Ed', 'D1', 'P2'),\n       ('Frank', 'D1', 'P2'),\n       ('George', 'D1', 'P2');\n-- load department #2 data\nINSERT INTO Personnel\nVALUES ('Harry', 'D2', 'P2'),\n       ('Jack', 'D2', 'P2'),\n       ('Larry', 'D2', 'P2'),\n       ('Mike', 'D2', 'P2'),\n       ('Nat', 'D2', 'P2');\n-- load department #3 data\nINSERT INTO Personnel \nVALUES ('Oscar', 'D3', 'P2'),\n       ('Pat', 'D3', 'P2'),\n       ('Rich', 'D3', 'P3');<\/pre>\n
The problem is to generate a report showing for each manager each department whether he is qualified to manage none, some or all of the projects being worked on within the department.  To find who can manage some, but not all, of the projects, use a version of relational division.<\/p>\n
SELECT M1.mgr_name, P1.dept_name\n  FROM MgrProjects AS M1\n       CROSS JOIN\n       Personnel AS P1\n WHERE M1.project_id = P1.project_id\n GROUP BY M1.mgr_name, P1.dept_name\nHAVING COUNT(*) <> (SELECT COUNT(emp_id)\n                      FROM Personnel AS P2\n                     WHERE P2.dept_name = P1.dept_name);<\/pre>\n
The query is simply a relational division with a <> instead of an = in the HAVING clause.  Richard came back with a modification of my answer that uses a characteristic function inside a single aggregate function.<\/p>\n
SELECT DISTINCT M1.mgr_name, P1.dept_name\n  FROM (MgrProjects AS M1\n        INNER JOININ\n        Personnel AS P1\n        ON M1.project_id = P1.project_id)\n         INNER JOIN\n         Personnel AS P2\n         ON P1.dept_name = P2.dept_name\n GROUP BY M1.mgr_name, P1.dept_name, P2.project_id\nHAVING MAX (CASE WHEN M1.project_id = P2.project_id\n                THEN 1 ELSE 0 END) = 0;<\/pre>\n
This query uses a characteristic function while my original version compares a count of Personnel under each manager to a count of Personnel under each project_id.  The use of “GROUP BY M1.mgr_name, P1.dept_name, P2.project_id” with the “SELECT DISTINCT M1.mgr_name, P1.dept_name” is really the tricky part in this new query.  What we have is a three-dimensional space with the (x, y, z) axis representing (mgr_name, dept_name, project_id) and then we reduce it to two dimensions (mgr_name, dept_id) by seeing if Personnel on shared project_ids cover the department or not.<\/p>\n
That observation leads to the next changes.  We can build a table that shows each combination of manager, department and the level of authority they have over the projects they have in common.  That is the derived table T1 in the following query; (authority = 1) means the manager is not on the project and authority = 2 means that he is on the project_id:<\/p>\n
SELECT T1.mgr_name, T1.dept_name,\n       CASE SUM(T1.authority)\n       WHEN 1 THEN 'None'\n       WHEN 2 THEN 'All'\n       WHEN 3 THEN 'Some'\n       ELSE NULL END AS POWER\n  FROM (SELECT DISTINCT M1.mgr_name, P1.dept_name,\n               MAX (CASE WHEN M1.project_id = P1.project_id\n                        THEN 2 ELSE 1 END) AS authority\n          FROM MgrProjects AS M1\n               CROSS JOIN\n               Personnel AS P1\n         GROUP BY m.mgr_name, P1.dept_name, P1.project_id) AS T1\n GROUP BY T1.mgr_name, T1.dept_name;<\/pre>\n
Another version, using the airplane hangar example:<\/p>\n
SELECT PS1.pilot_name,\n       CASE WHEN COUNT(PS1.plane_name) \n                > (SELECT COUNT(plane_name) FROM Hanger)\n                 AND COUNT(H1.plane_name) = (SELECT COUNT(plane_name)FROM Hanger)\n            THEN 'more than all'\n            WHEN COUNT(PS1.plane_name) \n                  = (SELECT COUNT(plane_name) FROM Hanger)\n                 AND COUNT(H1.plane_name) ) \n                     = (SELECT COUNT(plane_name) FROM Hanger)\n            THEN 'exactly all '\n            WHEN MIN(H1.plane_name) IS NULL\n            THEN 'none'\n            ELSE 'some' END AS skill_level\n  FROM PilotSkills AS PS1\n       LEFT OUTER JOIN\n       Hanger AS H1\n       ON PS1.plane_name = H1.plane_name\n  GROUP BY PS1.pilot_name;<\/pre>\n
We can now sum the authority numbers for all the projects within a department to determine the power this manager has over the department as a whole.  If he had a total of one, he has no authority over Personnel on any project in the department.  If he had a total of two, he has power over all Personnel on all projects in the department.  If he had a total of three, he has both a 1 and a 2 authority total on some projects within the department.  Here is the final answer.<\/p>\n
  Results\n  mgr_name dept_power\n=====================\n  M1  D1   'Some'\n  M1  D2   'None'\n  M1  D3   'Some'\n  M2  D1   'Some'\n  M2  D2   'All '\n  M2  D3   'All '\n  M3  D1   'Some'\n  M3  D2   'All '\n  M3  D3   'Some'\n  M4  D1   'All '\n  M4  D2   'All '\n  M4  D3   'All '\n \n<\/pre>\n
At the end of THE MANGA GUIDE TO DATABASES, Tico the Japanese database fairy has left the Kingdom of Kod to spread Basic RDBMS and SQL across the world.  I guess I get to be the SQL ogre who comes behind Tico and tells them that they just started, and that they need more than comic books introduction to do real work.  I just wonder how I would look drawn in Manga Style – I am creepy enough in the Western SF novels and comics that I have been in.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
Businesses often require reports that require more than the classic set operators. Surprisingly, a business requirement can often be expressed neatly in terms of the DIVISION relationship operator: How can this be done with SQL Server? Joe Celko opens up the ‘Manga Guide to Databases’, meets the Database Fairy, and is inspired to explain DIVISION.…<\/p>\n","protected":false},"author":96214,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[143531],"tags":[4168,4150,4151,4252,4926],"coauthors":[],"class_list":["post-519","post","type-post","status-publish","format-standard","hentry","category-t-sql-programming-sql-server","tag-database","tag-sql","tag-sql-server","tag-t-sql-programming","tag-tsql-sql-example-relational-division"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/519","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/96214"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=519"}],"version-history":[{"count":4,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/519\/revisions"}],"predecessor-version":[{"id":40051,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/519\/revisions\/40051"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=519"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=519"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=519"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=519"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}