{"id":111052,"date":"2026-07-06T12:00:00","date_gmt":"2026-07-06T12:00:00","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=111052"},"modified":"2026-06-15T09:12:56","modified_gmt":"2026-06-15T09:12:56","slug":"ai-llm-app-why-ai-still-cant-be-trusted-for-security","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/ai\/ai-llm-app-why-ai-still-cant-be-trusted-for-security\/","title":{"rendered":"AI can build the pipeline, but don’t trust it for security (building an app with an AI LLM, part three)"},"content":{"rendered":"\n

Building a database deployment pipeline with AI assistance sounds straightforward…until the AI starts treating temporary security shortcuts as permanent architecture. <\/strong><\/p>\n\n\n\n

In part three of his series on building an app and database almost entirely through an LLM<\/a>, Grant Fritchey walks through using GitHub Copilot and VSCode to configure Redgate Flyway for PostgreSQL on Azure Flexible Server, provision a self-hosted GitHub Actions runner inside a private VNet, and wire it all together into an automated CI\/CD pipeline. <\/strong><\/p>\n\n\n\n

The result is impressively functional \u2014 but the process surfaces a critical lesson: AI coding assistants are fast, capable, and dangerously confident about security decisions they have no business making alone.<\/strong><\/p>\n\n\n\n

Find the full repo, including every prompt, on GitHub<\/a>.<\/em>

Since I’m starting development on the database side of dbRosetta, and since I’m much more comfortable with databases than with code, I’m going to stay in the database sphere for a bit. <\/p>\n\n\n\n

My next step is to work with the AI to get a pipeline in place to deploy our database code to Azure Flexible Server<\/a>. I’m using Redgate Flyway<\/a> as the database automation tool because I’m comfortable with it (also, because it’s the best DB deployment tool out there \u2014 fight me).<\/p>\n\n\n\n

Setting up Flyway<\/h2>\n\n\n\n

I went straight to Microsoft Copilot<\/a> – my main AI companion for this experiment – and asked it to generate the prompt I’d need for VSCode<\/a>. However, I did a bad job on my own prompt. See if you can spot the problem:

Can you give me a prompt for VSCode, using the agreed prompt structure that you helped me build, to create a Flyway Enterprise baseline and initial set of migrations based on reverse engineering using Flyway Compare technology, to capture the existing database we created earlier through the Flyway CLI?
<\/em>
It might not be obvious why this prompt is bad. In fact, Copilot generated a perfectly reasonable VSCode prompt from that – versioned migration naming, non-destructive baseline, CI-friendly output, validation step. So, what’s wrong?<\/p>\n\n\n\n

Well, I didn’t specify what I wanted for development versus production environments – so it’s treating them all the same. This is something we’ll need to fix later. Isn’t it great that, even with AI, we can still introduce code debt!<\/p>\n\n\n\n

\n
\n
\n
\n

Deploy with confidence with Redgate Flyway
\r\n<\/h3>\n
\n The best-in-class database migration and version control tool – designed for for stable, governed database changes. <\/div>\n <\/div>\n Learn more & try for free<\/a>\n <\/div>\n <\/div>\n<\/section>\n\n\n

<\/p>\n\n\n\n

Giving the bad prompt to VSCode<\/h3>\n\n\n\n

I fed the prompt into VSCode. It went to work. It knows how to connect up to my Azure Flexible Server, so it ran `pg_dump` to get the SQL file for the baseline (I’d assumed it would use Flyway for this – it surprised me). And it then built out the `*.toml` file for controlling Flyway, and ran a baseline command against my database using the migrations it had created. <\/p>\n\n\n\n

It didn’t work flawlessly. It made a mistake in the configuration. Then, since it had provided multiple test steps, it fixed it. I’m not kidding. So, yeah, I’ve got a Flyway migration harness ready to go. <\/p>\n\n\n\n

There may be more that’s needed down the road to automate this fully, but I’m not going to lie – I’m surprised at what I’ve got so far. Pleasantly so.<\/p>\n\n\n\n

Creating a pipeline<\/h2>\n\n\n\n

When I’ve set these up myself, I get Flyway configured and functional first, then figure out how to get it to run inside the pipeline I create. Talking with Copilot, it wanted me to prompt it, to enable it to build the GitHub Actions<\/a>. <\/em>It is<\/em> weird talking to one to talk to the other, but it’s working gangbusters so far!<\/p>\n\n\n\n

I did have to remind it that, only earlier, it had told me we’d need to set up a VM (virtual machine)<\/a> inside my VNet (virtual network) to be a self-hosted GitHub runner…

I’ve noticed these things don’t do a flawless job of maintaining context. It falls back on us – at least in my experience so far – to be extremely thorough in our prompts in order to get the best results. However, it remembers again<\/em> that we’re behind a firewall. <\/p>\n\n\n\n

(And yes, before anyone asks, I’m using the paid tier for this adventure. I have to imagine the free tier may be even more forgetful.)<\/p>\n\n\n\n

What Copilot eventually produced<\/h3>\n\n\n\n

The prompt Copilot eventually produced was a self-hosted runner spec: cheapest viable Azure VM (Standard_B1s, ~$7.50\/month), no public IP, system-managed identity, secrets fetched from Key Vault at runtime via OpenID Connect (OIDC)<\/a>. Sensible. Over to VSCode…

This time it chose to create everything as bash scripts<\/a>. The first one ran cleanly and provisioned the VM, security group, network interface and VNet. On to the next script…<\/p>\n\n\n\n

This one involved a lot<\/em> of setup within the VM, where VSCode can’t help me as an agent \u2014 so I did a lot of typing from VSCode prompts. Now I’m the AI. Ha! <\/p>\n\n\n\n

The interesting thing was the troubleshooting. The AI didn’t get everything right the first time, although I may have introduced a typo or three. Troubleshooting, however, was actually pretty outstanding. I’d feed it errors and it would feed me solutions (or additional checks followed by solutions.)

Most of the rest of this was very interactive. Some work was readily done by GitHub Copilot acting as an agent inside VSCode. And some was me getting certificates and other things that couldn’t readily be retrieved through Copilot.<\/p>\n\n\n\n

When temporary becomes permanent (the security issues triggered by the AI)<\/h2>\n\n\n\n

It was interesting how much I had to remind the AI of things. For example, it wanted to test what we’d done, but only supported a `Development` or `Production` branch. Its first inclination was to add the branch we were working on, but that branch wouldn’t be permanent, so I had to remind it that we could do it for a test but it had to be easily undone. <\/p>\n\n\n\n

Then, we hit a security problem on our PostgreSQL<\/a> cluster.<\/h4>\n\n\n\n

It wanted to use the public IP address we had temporarily added to our VM. It was like, once a thing was done and available, that thing was permanently part of the project – despite<\/em> the stated purpose being a temporary thing for setup.

The conversation went something like this:

Me:<\/strong> Instead of the public IP address, shouldn’t we do something within Azure to let the resource access it since the IP address is temporary for the setup?

Copilot:<\/strong> Excellent point! You’re absolutely right. Since both the VM and PostgreSQL are in Azure, we should use VNet integration or Azure service endpoints instead of relying on the public IP.<\/em><\/p>\n\n\n\n

[Copilot then investigated and pivoted to suggesting the `0.0.0.0` “AllowAzureServices” firewall rule.]

Copilot: <\/strong>Perfect! The firewall rule `0.0.0.0` is a special Azure rule that allows all Azure services (including VMs in the same subscription) to connect to the PostgreSQL server. This is the Azure-native way to allow internal Azure resources to communicate.<\/em><\/p>\n\n\n\n

The latter is the kind of suggestion you want to pay close attention to – but why?<\/h4>\n\n\n\n

Well, the 0.0.0.0 rule<\/a> allows access from *any* Azure subscription, not just yours. It’s one of the more commonly cited misconfigurations in cloud security write-ups. <\/strong><\/p>\n\n\n\n

For a hobby project with no production data, the risk is small. For an engineer in their first job, however, copying this pattern into a real codebase because the AI called it the “Azure-native way,” the risk is most certainly not<\/em> small. <\/p>\n\n\n\n

In this instance, I pushed back, but only because I knew<\/em> to push back. We ended up with a sensible VNet-based path instead.

This was actually a common pattern across the whole step. Once the AI had something working – a temporary IP rule, a test branch in a deployment config, a one-off setup script – its instinct was to fold it into the architecture rather than flag it for removal. <\/p>\n\n\n\n

That’s simply because scaffolding and architecture look identical to an LLM (large language model). Both are *code that exists in the project*. The distinction lives in your head, not the model’s.<\/strong><\/p>\n\n\n\n

Summary: I’m very impressed – but security continues to be an issue<\/h2>\n\n\n\n

Overall, I’m blown away. Most snags were not<\/em> because the AI hallucinated<\/a>, or was horribly wrong, but instead from licensing, permissions, and stuff I didn’t define while I was writing prompts to both AIs. The silly thing was pretty darn helpful. <\/strong><\/p>\n\n\n\n

I let it do troubleshooting I was perfectly capable of doing myself, because I wanted to see it work things out. And it did. Quite well. <\/p>\n\n\n\n

AI is coming for your job… IF you’re not real good<\/em> at your job. I could have done every bit of this work, but using GitHub Copilot made it a lot faster and easier.<\/p>\n\n\n\n

However, I had to be on top of it. I had to track everything, because it didn’t always. It was more than ready to compromise security, multiple times. I couldn’t have just let it run.<\/strong> And this is a point I keep coming back to. The AI is a particularly brilliant, but extremely junior, developer on my team.<\/strong><\/p>\n\n\n\n

It’s lightning fast. It’s often correct. But it’s confidently wrong in exactly the places that matter most – and especially around the things that are “temporary for now”, but that no junior developer has the experience to flag. <\/p>\n\n\n\n

You don’t let a junior developer near production secrets without supervision. You don’t let them make the call on networking topology. The same goes here, with the AI. That’s why humans are still so valuable (at least for the moment). We’re not just employed – we’re necessary!<\/p>\n\n\n\n

In the next article, we’ll begin actually building the application.\u00a0See you there.<\/p>\n\n\n\n

\n
\n
\n
\n

Enjoying this article? Subscribe to the Simple Talk newsletter<\/h3>\n
\n Get selected articles, event information, podcasts and other industry content delivered straight to your inbox. <\/div>\n <\/div>\n Subscribe now<\/a>\n <\/div>\n <\/div>\n<\/section>\n\n\n
\n

FAQs: AI can build the pipeline, but don't trust it for security (building an app with an AI LLM, part three)<\/h2>\n\n

1. What is Redgate Flyway and why is it used for database deployments?<\/h3>\n
\n

Redgate Flyway<\/a> is a database migration and version control tool. It manages schema changes through numbered migration scripts, making deployments repeatable and auditable. In this project, Flyway Enterprise is used alongside Azure PostgreSQL Flexible Server to create a baseline from an existing database and apply controlled migrations through a CI\/CD pipeline.<\/p>\n <\/div>\n

2. Can GitHub Copilot set up a GitHub Actions pipeline automatically?<\/h3>\n
\n

Partially. Copilot can generate the YAML, bash scripts, and configuration files needed for a pipeline, and can troubleshoot errors when you feed them back. However, it requires careful human oversight \u2014 in this project it needed reminders about VNet constraints, self-hosted runner requirements, and security configurations it was getting wrong.<\/p>\n <\/div>\n

3. What security risks can AI assistants introduce in cloud infrastructure setup?<\/h3>\n
\n

A common pattern is treating temporary scaffolding as permanent architecture. In this case, Copilot suggested using the 0.0.0.0<\/code> PostgreSQL firewall rule \u2014 which grants access to all Azure subscriptions, not just your own \u2014 and described it as the “Azure-native” approach. A developer without prior cloud security experience could easily copy that pattern into production. The correct solution was a VNet-based private connection.<\/p>\n <\/div>\n

4. Do AI coding tools maintain context across a long conversation?<\/h3>\n
\n

Not reliably. In this project, Copilot would forget earlier constraints (such as the self-hosted runner requirement or the temporary nature of a public IP) and needed to be re-prompted. Writing thorough, explicit prompts and tracking decisions yourself remains essential.<\/p>\n <\/div>\n

5. Is AI going to replace database engineers?<\/h3>\n
\n

No – not experienced ones. AI functions like a fast, confident junior developer \u2014 useful for acceleration, but dangerous without supervision. It can’t distinguish scaffolding from production architecture, and it lacks the instinct to flag security or design decisions for review. Human expertise remains the necessary check.<\/p>\n <\/div>\n <\/section>\n","protected":false},"excerpt":{"rendered":"

Learn what happens when you let GitHub Copilot and VSCode set up database migrations and a GitHub Actions pipeline on Azure – featuring significant security risks.…<\/p>\n","protected":false},"author":221792,"featured_media":106674,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[159169,143526,53,46],"tags":[159075,4359,159395,5765],"coauthors":[6785],"class_list":["post-111052","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-development","category-featured","category-data-security-privacy-compliance","tag-ai","tag-development","tag-how-i-built-an-app-ai-llm","tag-security-and-compliance"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/111052","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/221792"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=111052"}],"version-history":[{"count":6,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/111052\/revisions"}],"predecessor-version":[{"id":111316,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/111052\/revisions\/111316"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media\/106674"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=111052"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=111052"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=111052"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=111052"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}