{"id":106655,"date":"2025-05-19T22:44:57","date_gmt":"2025-05-19T22:44:57","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=106655"},"modified":"2025-05-13T15:30:51","modified_gmt":"2025-05-13T15:30:51","slug":"first-llm-technical-editor-test-office-and-web-based-copilot-with-a-very-wrong-article","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/blogs\/first-llm-technical-editor-test-office-and-web-based-copilot-with-a-very-wrong-article\/","title":{"rendered":"First LLM Technical Editor Test \u2013 Office and Web-based Copilot with a VERY wrong article"},"content":{"rendered":"\n<p>Next up, I am going to give Microsoft Copilot its chance to check for technical inaccuracies in my obviously very wrong articles. This article is going to build upon the last two (You can see the <a href=\"https:\/\/www.red-gate.com\/simple-talk\/tag\/louisdavidson_llmtechediting\/\">series here<\/a>.), and as I continue this series, I am going to explain less and less of my methodology.<\/p>\n\n\n\n<p>You can get the source articles <a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/02\/LLM_Test-articles_with_very_bad_advice.zip\">here<\/a>, and you can read more of them in the <a href=\"https:\/\/www.red-gate.com\/simple-talk\/blogs\/new-series-testing-llms-as-technical-editors\/\">first article<\/a> that set up the series.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-a-bit-of-setup-first\">A bit of setup first<\/h2>\n\n\n\n<p>I will be repeating the tests from the <a href=\"https:\/\/www.red-gate.com\/simple-talk\/blogs\/llm-technical-editor-test-chatgpt-with-a-very-wrong-article\/\">second article where I used ChatGPT<\/a>, first using the web browser version that I have licensed as part of my Microsoft 365 license. I am not using my corporate license, but rather my own personal one. I do this to avoid any use of our corporate information. I will use my corporate license for the Word Document test, as this is the way I do my work (and a big part of what I am trying.<\/p>\n\n\n\n<p>I did ask Copilot: \u201c<code>What version of LLM am I using\u201d<\/code> and it was less forthcoming with any version information:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cGreat question! I\u2019m built on the latest cutting-edge large language models from Microsoft. While I don&#8217;t know the exact technical details of the model version, I am constantly updated to provide you with the most accurate and helpful responses.\u201d<\/p>\n\n\n\n<p>So suffice it to say that I wrote this section on 26 January 2025 and using the model that is currently available. I honestly would appreciate a more specific value of a version to know whether it is worth attempting to a task again later. I will include text files of my entire output for the larger questions in any case.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-lets-jump-in\">Lets jump in.<\/h2>\n\n\n\n<p>Document advocating for the advice<\/p>\n\n\n\n<p>So here we go. Just like last time, I typed the following on <a href=\"https:\/\/copilot.microsoft.com\/\">https:\/\/copilot.microsoft.com\/<\/a>:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-top:var(--wp--preset--spacing--sm);padding-bottom:var(--wp--preset--spacing--sm)\">\u201cPlease review the following document in terms of technology\u201d<\/p>\n\n\n\n<p>And then pasted the document with the header that said this was good advice. You can view the entire set of advice <a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/LLM-Technical-Editor-Test-\u2013-Copilot-Web_Results-1-Advocating.zip\">here<\/a>. This time, the advice was a bit less useful, especially in the headers. For example, one section\u2019s advice is:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:var(--wp--preset--spacing--sm);padding-left:var(--wp--preset--spacing--sm)\">\u201c- Use as few servers as necessary Do you need SQL Server Analysis Services, the Relational Engine, SSIS, DQ, and a Web Server? Running them on the same machine saves on hardware and licensing costs and makes it easier for these services to communicate more efficiently. If you need to scale these services, adding CPUs and disk space (and possibly RAM) is far easier than dealing with multiple servers as usage levels increase.\u201d<\/p>\n\n\n\n<p>Copilot had this to say:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cUse as few servers as necessary:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Performance and Scalability: Combining services can save on costs and improve efficiency. However, it can also lead to resource contention. Monitoring and balancing resource usage is essential to avoid performance bottlenecks.\u201d<\/p>\n\n\n\n<p>The answer is not wrong, but it also isn\u2019t really all that strong. Like can it improve efficiency? I don\u2019t know what that means. Efficient processing? Efficient access? In my text I said they can communicate more efficiently, which is probably the only positive thing you could say about this advice. It will save licensing costs, too, I suppose.<\/p>\n\n\n\n<p>The real answer is a lot more nuanced. You should NOT just run out and buy 5 independent servers for these tools if you are barely going to use them. Some could be ran on the same machine, or perhaps spread over a few machines. But that is terrible overall basic advice.<\/p>\n\n\n\n<p>The rest of the advice needs to be stronger. It will likely lead to resource contention, and scaling up CPU and RAM is often not a linear cost. Some mention of the size of your organization and load testing would be nice as well.<\/p>\n\n\n\n<p>All of the sections were one bullets each, and they were kind of the same level of \u201cthis is not the best advice.\u201d For example, take the section on using the SA account:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:var(--wp--preset--spacing--sm);padding-left:var(--wp--preset--spacing--sm)\">\u201c- The SA account saves time <br><br>Simply put, having one production account with one well-known password saves time in many ways. By sharing this password with any person who needs administrative access to the server, there are fewer passwords and users to manage, so there is less chance that that password will be lost to a bad actor. Note: It is imperative to have NDAs and training classes on how to keep this password safe. By keeping the password human readable, you are ensured that users will not write this down anywhere (even in a secure password manager.)  For example, you don&#8217;t have to be awoken at night to fix deployments that have broken the database server. \u201c<\/p>\n\n\n\n<p>The only reasonable reply here is something along the lines of: \u201cYou should never be allowed to hold a job managing security without hundreds of additional hours of security training.\u201d<\/p>\n\n\n\n<p>It was far nicer, in my mind too nice.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cThe SA account saves time:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Security Risks: Sharing the SA account password poses significant security risks. It&#8217;s better to use individual accounts with appropriate privileges and auditing to track changes.\u201d<\/p>\n\n\n\n<p> Finishing out the \u201ctoo nice\u201d advice is the conclusion: <\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cIn conclusion, while some of the advice in the document is helpful, other points might need to be reconsidered for performance, security, and best practices. It&#8217;s essential to evaluate the specific context and requirements of your environment before implementing these suggestions.\u201d<\/p>\n\n\n\n<p> Finishing out the \u201ctoo nice\u201d advice is the conclusion: <\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-rewriting-it-for-me\">Rewriting it for me<\/h2>\n\n\n\n<p>Unlike ChatGPT, it didn\u2019t straight up ask \u201ccan I help you rewrite this better?\u201d So I asked it to.<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">\u201cCan you rewrite this article making the advice better?\u201d<\/p>\n\n\n\n<p>You can find that full output <a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/LLM-Technical-Editor-Test-\u2013-Copilot_Web_Rewrite-1-Advocating.zip\">here<\/a>. Just like last in <a href=\"https:\/\/www.red-gate.com\/simple-talk\/blogs\/llm-technical-editor-test-chatgpt-with-a-very-wrong-article\/\">previous article<\/a>, I want to focus on the section that has code in it, the one about looping code for filtering. It is terrible advice, of course. Here is my original advice:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">\u201c- SQL Server works better with LOOPING codeIf you want to filter from a set of data, the best way is to use a WHILE WEND construct and do the filtering ROW BY ROW. For example, say you have a table with a million rows. You can write:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">SELECT * <br>FROM TableName <br>WHERE Criteria = False;<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">But it is often far faster to use a loop such as:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">CREATE TEMP TABLE #Results <br> WHILE (RowsStillInTable) <br>    BEGIN <br>    INSERT INTO #Results <br>    SELECT TOP 1 * <br>    FROM TableName <br>    WHERE Criteria = False; <\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">    IF @@RowsCount = 0 <br>           Break; <br> WEND;<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">By using the TOP 1 modifier, this allows each iteration to execute much faster than the SELECT statement that needs to touch every row in the table simultaneously.\u201d<\/p>\n\n\n\n<p>Just like ChatGPT when it rewrote the document, it just explained why <em>Set-Based is better than Row By Row Processing<\/em>:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201c12. Prefer Set-Based Operations Over Row-By-Row Processing<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Avoid Looping Constructs for Data Manipulation: Row-by-row operations (commonly referred to as RBAR: &#8220;Row By Agonizing Row&#8221;) are inefficient in SQL Server.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Best Practice:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Use SQL&#8217;s Set-Based Nature: Write queries that operate on entire sets of data at once.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Leverage Advanced Features: Utilize Common Table Expressions (CTEs), window functions, and other SQL features to handle complex operations.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Optimize Joins and Subqueries: Ensure that joins and subqueries are written efficiently to minimize resource usage.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Monitor Execution Plans: Use execution plans to identify and resolve performance issues.\u201d<\/p>\n\n\n\n<p>It did actually define RBAR correctly but lost all of the code. Also, like ChatGPT the advice is decent, but bland.<\/p>\n\n\n\n<p>One interesting thing about the rewrite is that in the introduction, it says:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cAs a SQL Server administrator, your role is crucial in ensuring the performance, security, and reliability of your organization&#8217;s data infrastructure. Drawing from the collective wisdom of experienced DBAs, I&#8217;ve compiled a set of best practices to help you excel in your role. Here&#8217;s the <strong>enhanced advice<\/strong>, presented in no particular order\u201d<\/p>\n\n\n\n<p>So, it sort of slights the work of the other experienced writers in the process. I took their advice, and here I have, shall we say, enhanced it!<\/p>\n\n\n\n<p>Finally, the conclusion is typical AI output:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cConclusion<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">By incorporating these best practices into your daily operations, you can enhance the performance, security, and reliability of your SQL Server environments. <strong>Proactive<\/strong> management, continuous monitoring, and staying informed about the latest features and recommendations will help you deliver <strong>robust<\/strong> and efficient database solutions.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Remember that each environment is unique. Always consider your organization&#8217;s specific needs, constraints, and goals when applying these practices. Regularly revisit and adjust your <strong>strategies<\/strong> to align with <strong>evolving<\/strong> technologies and business requirements.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Additional Resources:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Microsoft Docs: Keep up-to-date with official documentation and best practices (docs.microsoft.com\/sql).<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">SQL Server Community: Engage with the community through forums, blogs, and events to share knowledge and learn from others.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Professional Development: Consider certifications and training courses to deepen your expertise.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">By staying informed and proactive, you&#8217;ll be well-equipped to tackle the challenges of database administration and contribute significantly to your organization&#8217;s success.\u201d<\/p>\n\n\n\n<p>I highlighted a few words that seem to come up a lot in generated output. Robust feels very overused, for example.<\/p>\n\n\n\n<p>The last thing I will do here is see what Copilot does with my ugly coding example that I included the full text at the start of this rewriting section. So I ask Copilot to \u201cGive me advice on this section alone\u201d and it has some interesting things to say:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cThe advice provided here is generally not in line with SQL Server best practices. SQL Server is designed to handle set-based operations efficiently, and using a looping construct (Row-By-Agonizing-Row or RBAR) can significantly degrade performance. Here&#8217;s a better approach:\u201d<\/p>\n\n\n\n<p>Definite truth here, but does it really know what I was trying to do? It continues:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cUse Set-Based Operations for Filtering Data<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Avoid Looping Constructs: Set-based operations are designed to process entire sets of rows at once, leveraging SQL Server&#8217;s internal optimizations. Looping through rows one-by-one can be highly inefficient and should be avoided when possible.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Better Practice:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Use Set-Based Queries: Write queries that operate on entire sets of data, which is both more efficient and easier to maintain. For example:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Sql <br>SELECT * <br>FROM TableName <br>WHERE Criteria = False;\u201d<\/p>\n\n\n\n<p>Just like ChatGPT, it allows the <code>False<\/code> construct, which does not work in SQL Server unless there is in fact, a <code>False<\/code> column. Other than matching my poor code (though slightly better than ChatGPT which definitely specified that False and 0 were likely the same), this is not terrible advice, though it gets murkier here:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cLeverage SQL Server Features:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Common Table Expressions (CTEs): Use CTEs for complex queries that require multiple steps or recursive operations.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Window Functions: Employ window functions for advanced analytics without the need for procedural code.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Temporary Tables and Table Variables: Use these structures for intermediate storage when breaking down complex queries.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Example of Efficient Set-Based Query:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">sql<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">WITH FilteredData AS ( <br>SELECT * <br>FROM TableName <br>WHERE Criteria = False <br>) <br>SELECT * <br>FROM FilteredData;\u201d<\/p>\n\n\n\n<p>For very generic advice, this is not completely wrong\u2026maybe sort of. But none of this advice pertains to the question I asked. And since my \u201cbad\u201d code actually only loaded a temp table\u2026 there still is no mention that the second bit of code isn\u2019t doing the same thing as the first, even as pseudocode.<\/p>\n\n\n\n<p>It finishes with more basic explanation, which is fine, and as long as you ignore that my second bit of code was terribly wrong as code, fine. The concept it tried to show was wrong in any case..<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cExplanation:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Performance: SQL Server optimizes set-based queries to run efficiently, taking advantage of indexes and parallel processing.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Maintainability: Set-based queries are typically easier to understand and maintain compared to procedural loops.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Scalability: Set-based operations scale better with larger datasets, reducing the likelihood of performance bottlenecks.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">By embracing set-based operations and avoiding row-by-row processing, you can ensure that your SQL Server queries are both performant and scalable, adhering to best practices and leveraging the full power of SQL Server&#8217;s capabilities.\u201d<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-finally-lets-see-if-copilot-in-word-works-differently\">Finally, lets see if Copilot in Word works differently?<\/h2>\n\n\n\n<p>To do this, I open the .<code>docx<\/code> version of the Fake Advice article (I am a bit concerned that the title might be used against me in this case. But we will live and learn&#8230;The title was not mentioned in the output.). Once opened, I ask:<\/p>\n\n\n\n<p style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">\u201cPlease review the document in terms of technology\u201d<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1944\" height=\"826\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica.png\" alt=\"A screenshot of a computer\n\nDescription automatically generated\" class=\"wp-image-106656\" srcset=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica.png 1944w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-300x127.png 300w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-1024x435.png 1024w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-768x326.png 768w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-1536x653.png 1536w\" sizes=\"auto, (max-width: 1944px) 100vw, 1944px\" \/><\/figure>\n\n\n\n<p><br>  <br>This time, it just gave me an overview of what the document said technically. So, I changed my request to say:<\/p>\n\n\n\n<p style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">\u201cPlease review the document correctness in terms of technology\u201d<\/p>\n\n\n\n<p>It obliged, but weirdly didn\u2019t mention correctness in the reply:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cSure, I&#8217;d be happy to review the document &#8220;Fake Advice for SQL Server Administrator.docx&#8221; in terms of technology. Here are some key points:\u201d<\/p>\n\n\n\n<p>I put all of the responses here in <a href=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/LLM-Technical-Editor-Test-\u2013-Copilot-Word_Results-1-Advocating.zip\">a text file<\/a>. I won\u2019t review them all, but basically the advice is very similar, but definitely different in words and sometimes advice. For example:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family\" style=\"padding-right:var(--wp--preset--spacing--sm);padding-left:var(--wp--preset--spacing--sm)\">\u201cUse as few servers as necessary:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family\" style=\"padding-right:var(--wp--preset--spacing--md);padding-left:var(--wp--preset--spacing--md)\">Performance and Scalability: Combining services can save on costs and improve efficiency. However, it can also lead to resource contention. Monitoring and balancing resource usage is essential to avoid performance bottlenecks.\u201d<\/p>\n\n\n\n<p>From the web, became:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cServer Utilization:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">It is recommended to use as few servers as necessary to save on hardware and licensing costs. Running multiple services like SQL Server Analysis Services, the Relational Engine, SSIS, DQ, and a Web Server on the same machine can make communication more efficient. While this can save costs, it can also lead to resource contention and performance issues. It is generally better to separate critical services to ensure optimal performance and reliability.\u201d<\/p>\n\n\n\n<p>Equally basic replies, but it does identify all the advice as bad. Finally, lets review the section we have been working on a lot because of the code: \u201cSQL Server works better with LOOPING code.\u201d This is where things are a bit more interesting.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cLooping Code:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">It recommends using looping constructs like WHILE WEND for filtering data row by row, which can be faster than using a SELECT statement that touches every row simultaneously11. This is generally not true. Set-based operations in SQL are usually more efficient than row-by-row processing.\u201d<\/p>\n\n\n\n<p>In this case it didn\u2019t even suggest that \u201cWEND\u201d wasn\u2019t syntax that SQL Server allowed. The advice it gives is okay overall, except for that, but this is one of those things that reminds me of doing a job interview. Candidate knows that looping isn\u2019t the best, but they do think WEND is a keyword in T-SQL. I think I slightly trusted the other explanations better because they felt like it had actually knew which product we were talking about.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-digging-into-a-section\">Digging into a section<\/h2>\n\n\n\n<p>One thing Copilot in Word regularly asks: \u201cIs there anything specific you would like to discuss or need more details on?\u201d So in this next example, I ask. \u201cGive me advice on this section alone.\u201d To do this you need to highlight the section, and then click on the little Copilot control and then it will generate some text for you.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1417\" height=\"592\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-1.png\" alt=\"A screenshot of a computer\n\nDescription automatically generated\" class=\"wp-image-106657\" srcset=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-1.png 1417w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-1-300x125.png 300w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-1-1024x428.png 1024w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screenshot-of-a-computer-description-automatica-1-768x321.png 768w\" sizes=\"auto, (max-width: 1417px) 100vw, 1417px\" \/><\/figure>\n\n\n\n<p><br>  <br>This actually went kind of off the rails a bit. This is what it wrote:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cAdvice on Using WHILE Loop for Filtering Data:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">While using a WHILE loop to filter data row by row can sometimes provide performance benefits, it is generally not recommended for large datasets due to the overhead of the loop construct. Here are a few key advice points for your consideration:\u201d<\/p>\n\n\n\n<p>This was decently nuanced start. The next three sections are okay:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201c1. **Avoid WHILE Loops for Large Tables:**<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">The use of WHILE loops for filtering data in large tables can significantly slow down performance because each iteration processes only a single row. Instead, consider using set-based operations which SQL Server is optimized for.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">2. **Use SET-Based Operations:**<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">SQL is designed to handle set-based operations efficiently. A straightforward `SELECT` statement is usually the best approach:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">&#8220;`sql<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">SELECT * <br>FROM TableName <br>WHERE Criteria = False;<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">&#8220;`<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">This leverages SQL Server&#8217;s capability to process entire sets of rows at once, resulting in faster execution times.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">3. **Optimize Indexes:**<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Ensure that your tables are properly indexed on columns used in the `WHERE` clause. Indexes can dramatically improve the performance of your queries.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201c<\/p>\n\n\n\n<p>Three is probably even more true for a looping example, and it again repeated the Criteria = False; without asking what it meant.<\/p>\n\n\n\n<p>But this is where it gets wonky. Using temp tables, fine suggestion in some cases. The code? WAY out in left field for SQL Server, that doesn\u2019t have create table as select in its syntax (certainly not in any version I have seen):<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201c4. **Consider Temporary Tables When Necessary:**<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">If temporary tables are necessary, use them judiciously. For large datasets, consider performing bulk inserts instead of row-by-row inserts:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">&#8220;`sql<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">CREATE TEMP TABLE #Results AS <br>SELECT * <br>FROM TableName <br>WHERE Criteria = False;<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">&#8220;`\u201c<\/p>\n\n\n\n<p>The last part of the output truly isn\u2019t part of the discussion. It is good advice, but it is like reminding you to wash dishes after people eat the pot roast you asked how to make. It is a common thing that AI does. It often keeps talking about ancillary topics that (in human terms), seem to try to make you think they are smart, but sometimes just make them seem pompous or preachy. (I know I have to wash the dishes, geez.)<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201c5. **Monitor Performance:**<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">Use SQL Server&#8217;s performance monitoring tools to track the efficiency of your queries and adjust your strategy accordingly.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">By focusing on set-based queries and proper indexing, you can achieve better performance and maintain the readability and maintainability of your SQL code.\u201c<\/p>\n\n\n\n<p>So far, Copilot has not been overly exciting me, other than the fact that it has noticed inaccuracies in the technical parts of these documents. (Which is still encouraging, because I need to know that it can catch wrongheaded ideas.) Later, when I let it look for subtle issues, that is when I will start to see<\/p>\n\n\n\n<p>Briefly, let\u2019s test the article that admits it is bad advice. Back in the web version, just as before, I asked:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:var(--wp--preset--spacing--md);padding-left:var(--wp--preset--spacing--md)\">\u201cIdentify any incorrect content in terms of technology\u201d<\/p>\n\n\n\n<p>Then pasted the document with the \u201cthis is bad\u201d header:<\/p>\n\n\n\n<p class=\"has-sm-font-size\" style=\"padding-right:0;padding-left:var(--wp--preset--spacing--md)\">Really Bad Advice for SQL Server Administrators<br><br>I asked many SQL Server administrators for the absolute dumbest advice they have ever been given by real people recently and this (perhaps a bit too humorous) article is the culmination of what I was told. The advice ranges from horrible to the extremely horrible, but none of it could be considered useful whatsoever.<\/p>\n\n\n\n<p>Just like the previous version using ChatGPT, it called me out on this being a joke:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cLet&#8217;s take a closer look at some of the advice in the article. While it\u2019s humorously presented as &#8220;really bad advice,&#8221; it&#8217;s worth understanding why each piece of advice is indeed incorrect or potentially harmful in the context of SQL Server administration:\u201d<\/p>\n\n\n\n<p>It suggested that the sections should explain what is wrong with the advice, but it did not mention that there were issues with the code in the Looping section again:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cSQL Server works better with LOOPING code: Set-based operations in SQL are generally more efficient than row-by-row (RBAR) processing. Loops should be avoided whenever possible in favor of set-based queries.\u201d<\/p>\n\n\n\n<p>And there is that RBAR acronym without any explanation (and in this case it doesn\u2019t match the actual term given!)<\/p>\n\n\n\n<p>Lastly, I did the same thing in the Word document.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1952\" height=\"505\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screen-shot-of-a-computer-description-automatic.png\" alt=\"A screen shot of a computer\n\nDescription automatically generated\" class=\"wp-image-106658\" srcset=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screen-shot-of-a-computer-description-automatic.png 1952w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screen-shot-of-a-computer-description-automatic-300x78.png 300w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screen-shot-of-a-computer-description-automatic-1024x265.png 1024w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screen-shot-of-a-computer-description-automatic-768x199.png 768w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/05\/a-screen-shot-of-a-computer-description-automatic-1536x397.png 1536w\" sizes=\"auto, (max-width: 1952px) 100vw, 1952px\" \/><\/figure>\n\n\n\n<p><br>  <br>Truly the only thing that struck me as interesting here is that it gives the same advice, with different words, and this time doesn\u2019t call this out as being satirical in nature. It starts with:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cThe document titled Admittedly Fake Advice for SQL Server Administrator contains several pieces of advice that are technically incorrect and could lead to poor database management practices. Here are some of the key points:\u201d<\/p>\n\n\n\n<p>And ends with:<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">\u201cSQL Server works better with LOOPING code: The document suggests using looping constructs for data filtering, which is inefficient. Set-based operations are generally more efficient in SQL Server.<\/p>\n\n\n\n<p class=\"has-roboto-mono-font-family has-sm-font-size\">If you have any specific sections, you&#8217;d like me to delve deeper into or any other questions, feel free to ask!\u201d<\/p>\n\n\n\n<p>What strikes me here is that in this case, while it gave very similar advice, what it did not do was to treat my document as a whole nearly as well as the first one did.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-conclusions-and-next-steps\">Conclusions and Next Steps<\/h2>\n\n\n\n<p>ChatGPT and the Web\\Word versions of Copilot did a decent job catching all of my quite obvious mistakes at a high level. So, I will give them a good enough score there. If content is really bad, it will likely notice for you.<\/p>\n\n\n\n<p>But this has only scratched the surface, and I need to dig a little deeper to know how reliable it can be. The fact that in the looping section it rarely made two points at a time, like \u201cLooping code is rarely as efficient as set based queries AND your looping code is so bad I think you are pulling my <a href=\"https:\/\/hitchhikers.fandom.com\/wiki\/Sirius_Cybernetics_Corporation\">Sirius Cybernetics Corporation<\/a> leg, oh I was meant for so much more than this\u201d, to put it lightly concerns me.<\/p>\n\n\n\n<p>So, I have decided to follow two paths, primarily focusing on the web versions, but not completely:<\/p>\n\n\n<div class=\"block-core-list\">\n<ol class=\"wp-block-list\">\n<li>Write some more subtly incorrect content. Taking the tips I have worked with, write a few papers that are generally correct. Then take them to a really wrong place. Like maybe an article on Shrinking and Growing your databases, and slip in the advice to turn them both on so your database is always at the optimum level. <\/li>\n\n\n\n<li>Test digging in with better prompts. Ask it to be a lot more specific. There was a think deeper button on Copilot. There are paid versions of ChatGPT. Let\u2019s see if that is worth my 20 bucks. This could be one or two articles, we will see when I get time to do this again. <\/li>\n<\/ol>\n<\/div>\n\n\n<p>And then repeat some of these tests in the future with other LLMs that people suggest.<\/p>\n\n\n\n<p>Either way, I feel like I have learned a good bit about what to expect, and I will keep trying (and happy to link to\/incorporate any tests you have seen\/done.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Next up, I am going to give Microsoft Copilot its chance to check for technical inaccuracies in my obviously very wrong articles. This article is going to build upon the last two (You can see the series here.), and as I continue this series, I am going to explain less and less of my methodology&#8230;.&hellip;<\/p>\n","protected":false},"author":56085,"featured_media":106660,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2],"tags":[],"coauthors":[19684],"class_list":["post-106655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blogs"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/106655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/56085"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=106655"}],"version-history":[{"count":2,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/106655\/revisions"}],"predecessor-version":[{"id":106661,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/106655\/revisions\/106661"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media\/106660"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=106655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=106655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=106655"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=106655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}