{"id":96625,"date":"2023-05-15T10:30:23","date_gmt":"2023-05-15T10:30:23","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=96625"},"modified":"2023-05-09T13:50:55","modified_gmt":"2023-05-09T13:50:55","slug":"log-file-instant-file-initialization","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/t-sql-programming-sql-server\/log-file-instant-file-initialization\/","title":{"rendered":"Changing log growth strategy in SQL Server 2022"},"content":{"rendered":"<p>When I first saw a bullet item stating SQL Server 2022 would support <a href=\"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/database-administration-sql-server\/instant-file-initialization\/\" target=\"_blank\" rel=\"noopener\">instant file initialization<\/a> for log file growth, I was excited. When I later learned it only applies to <em>automatic<\/em> growths, and only those of 64 MB or less, I was a little less excited. With those limitations, I was skeptical this enhancement could supplant my long-standing practice of using 1 GB autogrowth for log files &#8211; at least ever since SSDs and other modern storage became more commonplace.<\/p>\n<p>But after playing with it, <strong>I&#8217;m a believer<\/strong>.<\/p>\n<p>Since the advent of the instant file initialization feature, <em>data<\/em> file growths are nearly instantaneous, because the newly-allocated space can be created empty. When <em>log<\/em> files grow, on the other hand, the new space must be zero-initialized to make sure SQL Server uses the transaction log correctly, as Paul Randal explains in <a href=\"https:\/\/www.sqlskills.com\/blogs\/paul\/search-engine-qa-24-why-cant-the-transaction-log-use-instant-initialization\/\" target=\"_blank\" rel=\"noopener\">Why can\u2019t the transaction log use instant file initialization?<\/a><\/p>\n<p><code style=\"font-family: consolas; background: #e4e4e4; padding: 3px 4px 1px 4px; border-radius: 3px;\">PREEMPTIVE_OS_WRITEFILEGATHER<\/code> is the wait type you&#8217;ll see if a session is waiting on log file initialization. You can check if an instance is generally suffering this pain by <a href=\"https:\/\/www.sqlskills.com\/blogs\/paul\/wait-statistics-or-please-tell-me-where-it-hurts\/\" target=\"_blank\" rel=\"noopener\">comparing against other prevalent waits<\/a> in your workload, by checking <code style=\"font-family: consolas; background: #e4e4e4; padding: 3px 4px 1px 4px; border-radius: 3px;\">sys.dm_os_wait_stats<\/code>:<\/p>\n<pre class=\"theme:tomorrow-night font:consolas font-size:14 line-height:16 decode-attributes:false tab-convert:true lang:tsql decode:true whitespace-before:1 whitespace-after:1\"> SELECT wait_type, waiting_tasks_count, wait_time_ms\r\n   FROM sys.dm_os_wait_stats\r\n   WHERE wait_type = N'PREEMPTIVE_OS_WRITEFILEGATHER';<\/pre>\n<p>In the days of spinning rust platters, growing a log file was extremely painful, particularly in highly concurrent, write-heavy workloads. For most of my career, this alone has cast log files in general in a negative light. Even on fast and modern storage, log file expansion can be quite disruptive, because all transactions need to wait on any file growth operations.<\/p>\n<p>In most cases, best practice has dictated that you just size your log files as large as possible, to avoid any unexpected growth events. But this isn&#8217;t always possible &#8211; on systems with many databases, for example, you only have so many drives, and you can&#8217;t size <em>all<\/em> of your log files to fill the disk up front. It is also sometimes difficult to predict which ones will grow more &#8211; or in more spiky ways &#8211; over time. In these cases, a smaller autogrowth setting for log files can still be a good strategy today, even without this enhancement.<\/p>\n<p><b>But let&#8217;s see the impact of the change<\/b>.<\/p>\n<p>On two different instances on the same machine, I&#8217;ll create the following four databases, each with an 8 GB data file and 32 MB log file &#8211; all on the same drive. The differences will just be the version and the log file&#8217;s <code style=\"font-family: consolas; background: #e4e4e4; padding: 3px 4px 1px 4px; border-radius: 3px;\">filegrowth<\/code> setting:<\/p>\n<ul>\n<li>SQL Server 2019:\n<ul>\n<li>1 GB autogrow<\/li>\n<li>64 MB autogrow<\/li>\n<\/ul>\n<\/li>\n<li>SQL Server 2022:\n<ul>\n<li>1 GB autogrow<\/li>\n<li>64 MB autogrow<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<pre class=\"theme:tomorrow-night font:consolas font-size:14 line-height:16 decode-attributes:false tab-convert:true lang:tsql decode:true whitespace-before:1 whitespace-after:1\">\/* values in brackets should all be set before running script \r\n{2019|2022} = version; {64|1024} = log file size *\/\r\nCREATE DATABASE ifi_{64|1024}\r\n ON \r\n (\r\n   name     = N'ifi_data',\r\n   filename = 'D:\\data\\ifi_{2019|2022}_{64|1024}.mdf',\r\n   size = 8192MB, filegrowth = 2048MB\r\n )\r\n LOG ON \r\n (\r\n   name     = N'ifi_log', \r\n   filename = 'D:\\data\\ifi_{2019|2022}_{64|1024}.ldf',\r\n   size = 32MB, filegrowth = {64|1024}MB\r\n );<\/pre>\n<p>Then I&#8217;ll set up a simple but log-heavy workload script to execute against each database, measuring the following:<\/p>\n<ul>\n<li>total workload duration<\/li>\n<li>total duration of <code style=\"font-family: consolas; background: #e4e4e4; padding: 3px 4px 1px 4px; border-radius: 3px;\">PREEMPTIVE_OS_WRITEFILEGATHER<\/code> waits<\/li>\n<li>number of log growth events<\/li>\n<li>total virtual log file fragments (VLFs)<\/li>\n<li>average VLF size<\/li>\n<li>ending log file size<\/li>\n<\/ul>\n<p>The workload itself is pretty simple:<\/p>\n<pre class=\"theme:tomorrow-night font:consolas font-size:14 line-height:16 decode-attributes:false tab-convert:true lang:tsql decode:true whitespace-before:1 whitespace-after:1 \"> DROP TABLE IF EXISTS dbo.dummy;\r\n\r\n SELECT TOP (0) ID = IDENTITY(int,1,1),\r\n  name = CONVERT(nchar(2048), N'x') INTO dbo.dummy;\r\n\r\n CREATE UNIQUE CLUSTERED INDEX CIX_dummy ON dbo.dummy(ID);\r\n\r\n INSERT dbo.dummy(name) SELECT TOP (500000) N'x' \r\n   FROM sys.all_objects AS s1\r\n   CROSS JOIN sys.all_objects AS s2;<\/pre>\n<p>You can monitor the impact of this &#8220;workload&#8221; in a number of ways, including a query against <code style=\"font-family: consolas; background: #e4e4e4; padding: 3px 4px 1px 4px; border-radius: 3px;\">sys.dm_os_wait_stats<\/code> similar to the one above, as well as an <a href=\"https:\/\/learn.microsoft.com\/en-us\/sql\/t-sql\/statements\/create-event-session-transact-sql?\">Extended Events session<\/a> including the following:<\/p>\n<pre class=\"theme:tomorrow-night font:consolas font-size:14 line-height:16 decode-attributes:false tab-convert:true lang:tsql decode:true whitespace-before:1 whitespace-after:1 \"> CREATE EVENT SESSION [log_size_changes] ON SERVER \r\n   ADD EVENT sqlos.wait_info_external \r\n     (WHERE ([wait_type] = N'PREEMPTIVE_OS_WRITEFILEGATHER')),\r\n   ADD EVENT sqlserver.database_file_size_change\r\n   ADD TARGET package0.event_file(SET filename = N'ifi_log_size_changes')\r\n   \/* ... *\/<\/pre>\n<p><em style=\"display: block; margin-top: -18px; font-size: 0.85rem;\">Just note that some events report durations in microseconds, and others in milliseconds.<\/em><\/p>\n<p>Or queries against the <a href=\"https:\/\/sqlblog.org\/2007\/01\/11\/reviewing-autogrow-events-from-the-default-trace\" target=\"_blank\" rel=\"noopener\">default trace<\/a>, if you <a href=\"https:\/\/sqlperformance.com\/2020\/04\/extended-events\/removing-default-trace-2\" target=\"_blank\" rel=\"noopener\">still have it enabled<\/a>:<\/p>\n<pre class=\"theme:tomorrow-night font:consolas font-size:14 line-height:16 decode-attributes:false tab-convert:true lang:tsql decode:true whitespace-before:1 whitespace-after:1\"> SELECT t.DatabaseName, \r\n        GrowthEvents    = COUNT(*), \r\n        AverageDuration = AVG(t.Duration\/1000.0), \r\n        TotalDuration   = SUM(t.Duration\/1000.0)\r\n FROM\r\n (\r\n   SELECT [path] = REVERSE(SUBSTRING(p, CHARINDEX(N'\\', p), 260)) + N'log.trc'\r\n   FROM (SELECT REVERSE([path]) FROM sys.traces WHERE is_default = 1) AS s(p)\r\n ) AS p\r\n CROSS APPLY sys.fn_trace_gettable(p.[path], DEFAULT) AS t\r\n WHERE t.EventClass = 93\r\n   \/* AND t.DatabaseName LIKE N'ifi%' *\/\r\n GROUP BY t.DatabaseName;<\/pre>\n<p>And checking the output of <code style=\"font-family: consolas; background: #e4e4e4; padding: 3px 4px 1px 4px; border-radius: 3px;\">sys.dm_db_log_info<\/code>\u00a0you can see more information about the number of virtual log files and their sizes:<\/p>\n<pre class=\"theme:tomorrow-night font:consolas font-size:14 line-height:16 decode-attributes:false tab-convert:true lang:tsql decode:true whitespace-before:1 whitespace-after:1\"> SELECT d.name, \r\n        VLFCount  = COUNT(li.database_id), \r\n        AvgSizeMB = AVG(li.vlf_size_mb)\r\n FROM sys.databases AS d\r\n CROSS APPLY sys.dm_db_log_info(d.database_id) AS li\r\n \/* WHERE d.name LIKE N'ifi%' *\/\r\n GROUP BY d.name;<\/pre>\n<p>After running the workload against each database, I observed the following:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-96709\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2023\/04\/ab-ifi-2022.png\" alt=\"Metrics observed during growth events\" width=\"1523\" height=\"443\" \/><\/p>\n<p>Those are favorable and promising results &#8211; we&#8217;re shaving roughly 20% off the workload duration simply by moving to 64 MB autogrowth, entirely due to eliminating <code style=\"font-family: consolas; background: #e4e4e4; padding: 3px 4px 1px 4px; border-radius: 3px;\">PREEMPTIVE_OS_WRITEFILEGATHER<\/code> waits and capitalizing on instant file initialization. In addition, we see a slight benefit from an improved VLF algorithm (<a href=\"https:\/\/learn.microsoft.com\/en-us\/sql\/sql-server\/what-s-new-in-sql-server-2022?view=sql-server-ver16#performance\" target=\"_blank\" rel=\"noopener\">recently documented<\/a>), resulting in fewer, larger VLFs for the exact same log growth events.<\/p>\n<p>Now, <strong>your mileage may vary<\/strong>. On more capable hardware, the percentage of time spent on initialization may be lower, so the benefit might be more subtle. Still, zeroing out a new portion of a log file will never be instantaneous for growth events larger than 64 MB (barring more enhancements in future versions, of course). So, as you start planning for SQL Server 2022, or even if you&#8217;re already there, this configuration option is worth testing against your workload and hardware. This is especially true if you&#8217;re not in a position to pre-size your log files bigger than you&#8217;ll ever need. After all, the fastest file change is the one that doesn&#8217;t have to happen in the first place; the next fastest is the one that can take advantage of instant file initialization.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>When I first saw a bullet item stating SQL Server 2022 would support instant file initialization for log file growth, I was excited. When I later learned it only applies to automatic growths, and only those of 64 MB or less, I was a little less excited. With those limitations, I was skeptical this enhancement&#8230;&hellip;<\/p>\n","protected":false},"author":341115,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[143527,53,143531],"tags":[],"coauthors":[158980],"class_list":["post-96625","post","type-post","status-publish","format-standard","hentry","category-database-administration-sql-server","category-featured","category-t-sql-programming-sql-server"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/96625","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/341115"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=96625"}],"version-history":[{"count":52,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/96625\/revisions"}],"predecessor-version":[{"id":96752,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/96625\/revisions\/96752"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=96625"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=96625"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=96625"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=96625"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}