{"id":82165,"date":"2014-08-08T21:49:18","date_gmt":"2014-08-08T21:49:18","guid":{"rendered":"https:\/\/www.webstaging.red-gate.com\/simple-talk\/?p=73518"},"modified":"2018-12-12T11:59:12","modified_gmt":"2018-12-12T11:59:12","slug":"how-in-memory-database-objects-affect-database-design-hybrid-code","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/blogs\/how-in-memory-database-objects-affect-database-design-hybrid-code\/","title":{"rendered":"How In-Memory Database Objects Affect Database Design: Hybrid Code"},"content":{"rendered":"<p>In my first attempts at building my code, I strictly went with either native or on-disk code. I specifically wrote the on-disk code to only use features that worked in-memory. This lead to one majorly silly bit of code, used to create system assigned key values. How would I create a customer number that was unique. We can\u2019t use the Max(value) + 1 approach because it will be very hideous with MVCC isolation levels, since 100 connections might see the same value, leading to lots of duplication. You can\u2019t see other connections, so you would duplicate data quickly.&#160; I was also limited to not using sequence objects because they too are not allowed in native code.<\/p>\n<p>So, I used a random number generator, like this:<\/p>\n<p>DECLARE @CustomerNumber CHAR(10)    <br \/>WHILE 1=1     <br \/>BEGIN     <br \/>&#160;&#160;&#160; SET @customerNumber = &#8216;CU&#8217; + RIGHT(&#8216;00000000&#8217; + CAST(CAST(100000000 * RAND() AS INT) AS VARCHAR(8)),8)     <br \/>&#160;&#160;&#160; <br \/>&#160;&#160;&#160; IF NOT EXISTS (SELECT * FROM Customers.Customer WHERE CustomerNumber = @CustomerNumber)     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; BREAK<\/p>\n<p>END&#160;&#160;&#160; <\/p>\n<p>This actually worked quite well for the tens of thousand rows I generated. The likelihood of two connections getting the same random value, and getting past the NOT EXISTS block was very unlikely. Now, obviously the probability of clashes will rise greatly as you approach the limits of the 8 digit number, so you would have to monitor usage and change the CU to something else to increment the value. This is demo code, and as this blog is about, there are easier ways. But it was fun to write and test.<\/p>\n<p>Of course the worst part of this code isn\u2019t the random number generator, or even the looping (oh, the dreaded looping), no this code is not very optimal, because of the NOT EXISTS subquery.&#160; Best case we have to do one probe into the table to see if that value doesn\u2019t exist.&#160; While it was sub optimal in interpreted code, in native code, it got more silly looking because you can\u2019t break out of a while loop, and you can\u2019t use subqueries (nor the RIGHT function). So the code changed to the following awkward (yet operational) bit of code:<\/p>\n<p>DECLARE @customerNumber CHAR(10), @rowcount INT, @keepGoing BIT = 1, @baseNumber VARCHAR(30)    <br \/>WHILE @keepGoing = 1     <br \/>BEGIN     <br \/>&#160;&#160;&#160; SET @baseNumber = &#8216;00000000&#8217; + CAST(CAST(100000000 * RAND() AS INT) AS VARCHAR(8))     <br \/>&#160;&#160;&#160; SET @customerNumber = &#8216;CU&#8217; + SUBSTRING(@baseNumber,LEN(@baseNumber) &#8211; 8,8)     <br \/>&#160;&#160;&#160; <br \/>&#160;&#160;&#160; SELECT @rowcount = COUNT(*) FROM Customers.Customer WHERE CustomerNumber = @CustomerNumber     <br \/>&#160;&#160;&#160; IF @rowcount = 0     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; SET @keepGoing = 0<\/p>\n<p>END&#160;&#160;&#160; <\/p>\n<p>The inefficiency of this code is the same as the interpreted code: that query on CustomerNumber.&#160; Of course, the more typical solution to the problem of a system generated key would be to use a SEQUENCE object (possibly as a default), and format the number somehow. But you can\u2019t use sequences in native code, so instead of going fully native code, I am using a hybrid approach.<\/p>\n<p>First, I built a procedure that generated CustomerNumbers, it has a simple checkdigit appended to a 7 digit number (preceded by CU) (and sloughing off any customer numbers with 666 in the value from the string just to show what can be done).<\/p>\n<p>CREATE SEQUENCE Customers.Customer$CustomerNumber$Sequence   <br \/>AS INT    <br \/>START WITH 1    <br \/>go<\/p>\n<p>&#8211;use a sequence and format the output a bit to avoid lots of duplication    <br \/>CREATE&#160; PROCEDURE Customers.Customer$getNextCustomerNumber     <br \/>&#160;&#160;&#160; @customerNumber char(10) OUTPUT     <br \/>AS     <\/p>\n<p>&#160;&#160;&#160; &#8211;doing it in a loop is the simplest method when complex requirements.     <br \/>&#160;&#160;&#160; WHILE (1=1)     <br \/>&#160;&#160;&#160;&#160; BEGIN     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; &#8211;Get the base account number, which is just the next value from the stack     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; SET @customerNumber = &#8216;CU&#8217; + right(replicate (&#8216;0&#8217;,8) +     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; CAST(NEXT VALUE FOR Customers.Customer$CustomerNumber$Sequence as varchar(7)), 6)<\/p>\n<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160; &#8211;add a check digit to the account number (take some digits add together, take the first number)    <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; SELECT @customerNumber = CAST(@customerNumber AS varchar(8)) +     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; RIGHT(CAST(     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; CAST(SUBSTRING(@customerNumber, 3,1) AS TINYINT) +     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; POWER(CAST(SUBSTRING(@customerNumber, 5,1) AS TINYINT),2) +     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; CAST(SUBSTRING(@customerNumber, 8,1) AS TINYINT) * 3 +     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; CAST(SUBSTRING(@customerNumber, 9,1) AS TINYINT) * 2 +     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; CAST(SUBSTRING(@customerNumber, 10,1) AS TINYINT) +     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; CAST(SUBSTRING(@customerNumber, 11,1) AS TINYINT) * 3&#160; AS VARCHAR(10)),1)<\/p>\n<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160; &#8211;if the number doesn&#8217;t have these character string in it (including check digit)    <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; if&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; @customerNumber NOT LIKE &#8216;%00000%&#8217;     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; AND @customerNumber NOT LIKE &#8216;%666%&#8217;     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; BREAK &#8212; we are done     <br \/>&#160;&#160;&#160;&#160; END     <br \/>GO<\/p>\n<p>Pretty simple, just loops until a good number is found. Not looping through rows, but looping through sequence values, which is very fast and can be done by many simultaneous callers with almost no contention.&#160; If contention is an issue, you can choose your caching for a sequence object, which can avoid a couple of writes when it persists that you have burned through the previously cached values.<\/p>\n<p>So now, the code simply says:<\/p>\n<p>DECLARE @CustomerNumber CHAR(10)    <br \/>EXEC Customers.Customer$getNextCustomerNumber @CustomerNumber OUTPUT<\/p>\n<p>Rather than that loop. But I can\u2019t use that in a natively compiled procedure, so we create an interpreted procedure that calls this procedure, then calls the native procedure:<\/p>\n<p>CREATE PROCEDURE Customers.Customer$CreateAndReturn    <br \/>@FirstName NVARCHAR(30),     <br \/>@LastName NVARCHAR(30),     <br \/>@MiddleName NVARCHAR(30),     <br \/>@EmailAddress NVARCHAR(200)     <br \/>AS     <\/p>\n<p>BEGIN     <br \/>&#160;&#160;&#160; SET NOCOUNT ON<\/p>\n<p>&#160;&#160; &#8211;see if the customer exists\u2026 We don\u2019t do updates here    <br \/>&#160;&#160;&#160; DECLARE @customerId INT = (SELECT CustomerId     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; FROM&#160; Customers.Customer     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; WHERE EmailAddress = @EmailAddress) &#8211;we are assuming validation is done elsewhere<\/p>\n<p>&#160;&#160;&#160; IF @customerId IS NULL    <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; BEGIN     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; DECLARE @customerNumber CHAR(10)     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; EXEC&#160; Customers.Customer$getNextCustomerNumber @customerNumber OUTPUT<\/p>\n<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; EXEC @CustomerId = Customers.Customer$SimpleInMemCreate     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; @CustomerNumber = @CustomerNumber,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; @FirstName = @FirstName,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; @LastName = @LastName,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; @MiddleName = @MiddleName,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; @EmailAddress = @EmailAddress<\/p>\n<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160; END<\/p>\n<p>&#160;&#160;&#160;&#160;&#160; RETURN COALESCE(@customerId, -100)    <br \/>END     <br \/>GO<\/p>\n<p>I haven\u2019t added error handling just yet, but this is nearly the final version. The procedure to do the actual insert is just a simple insert using native compilation:<\/p>\n<p>CREATE PROCEDURE Customers.Customer$SimpleInMemCreate    <br \/>@CustomerNumber CHAR(10),     <br \/>@FirstName NVARCHAR(30),     <br \/>@LastName NVARCHAR(30),     <br \/>@MiddleName NVARCHAR(30),     <br \/>@EmailAddress NVARCHAR(200)     <br \/>WITH NATIVE_COMPILATION, SCHEMABINDING, EXECUTE AS OWNER     <br \/>AS BEGIN ATOMIC WITH     <br \/>(     <br \/>TRANSACTION ISOLATION LEVEL =&#160; SNAPSHOT, LANGUAGE =&#160; N&#8217;us_english&#8217;     <br \/>)     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160; DECLARE @customerId int     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; INSERT INTO Customers.Customer     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ( FirstName , MiddleName ,LastName ,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; CustomerNumber ,&#160; EmailAddress,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; RowCreateTime, RowLastModifiedTime     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; )     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; VALUES&#160; ( @FirstName , @MiddleName ,@LastName ,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; @CustomerNumber ,&#160; @EmailAddress,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; SYSDATETIME(), SYSDATETIME()     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; )     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; SELECT @customerId = SCOPE_IDENTITY()<\/p>\n<p>&#160;&#160;&#160;&#160;&#160; RETURN isnull(@customerId, -100)     <br \/>&#160;&#160;&#160; END     <br \/>GO     <\/p>\n<p>So we get the benefits of the compiled procedure (if there is any in the actual case, my demo code is fairly simplistic) coupled with anything in the interpreted code that could not be done in native mode. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>In my first attempts at building my code, I strictly went with either native or on-disk code. I specifically wrote the on-disk code to only use features that worked in-memory. This lead to one majorly silly bit of code, used to create system assigned key values. How would I create a customer number that was&#8230;&hellip;<\/p>\n","protected":false},"author":56085,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2],"tags":[],"coauthors":[19684],"class_list":["post-82165","post","type-post","status-publish","format-standard","hentry","category-blogs"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/82165","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/56085"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=82165"}],"version-history":[{"count":1,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/82165\/revisions"}],"predecessor-version":[{"id":82254,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/82165\/revisions\/82254"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=82165"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=82165"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=82165"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=82165"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}