{"id":113860,"date":"2026-10-28T12:00:00","date_gmt":"2026-10-28T12:00:00","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=113860"},"modified":"2026-10-19T08:44:27","modified_gmt":"2026-10-19T08:44:27","slug":"stop-customer-data-leaking-into-chatgpt","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/ai\/stop-customer-data-leaking-into-chatgpt\/","title":{"rendered":"How personal data leaks into AI tools by accident (and how to stop it)"},"content":{"rendered":"\n

Personal data (like customer data) usually reaches an AI tool<\/strong> by accident, not by attack. Someone is busy and the chat window is one CTRL+V away, so it’s a quick and easy solution. <\/strong><\/p>\n\n\n\n

The most common ways personal data leaks into AI tools? Clean-up requests, BI assistant summaries, over-shared files read by copilots, coding assistants, call transcripts, and browser extensions. <\/strong><\/p>\n\n\n\n

Thankfully, you can prevent this happening without<\/em> ‘banning’ the use of AI by masking data at the source<\/strong>, fixing permissions before enabling AI ‘copilots’<\/strong>, providing an approved enterprise AI tool<\/strong>, and adding technical guardrails<\/strong> such as browser data loss prevention (DLP).<\/strong><\/p>\n\n\n\n

How (and why) does personal data leak into AI tools by accident?<\/h2>\n\n\n\n

A data analyst pastes a CSV into an AI tool like ChatGPT for \u201cclean up\u201d. A business intelligence (BI)<\/a> assistant then uses GPT to summarize a dashboard full of customer emails. Nobody intended<\/em> to share personal data, but they did.<\/p>\n\n\n\n

My previous article on anonymizing PII in LLM pipelines<\/a> covered the pipelines you design, whereas here I’m focusing on the pipelines not<\/em> designed to ingest PII (personally identifiable information)<\/a>. <\/p>\n\n\n\n

I’ll explain how personal data (or customer data<\/em> from an organization’s perspective) slips into AI tools by accident, and how to stop it without<\/em> banning the use of AI.<\/p>\n\n\n\n

From both a personal and professional perspective, the fastest way to get work done nowadays is via an LLM (large language model)<\/a> – and naturally, we’re all handling data of some sort every day. Data that, depending on the context, is usually full<\/em> of PII. <\/p>\n\n\n\n

That ends up being fed into the LLM – and this is often missed. Here are a few of the reasons this can happen within an organization\/business:<\/p>\n\n\n\n

Reason<\/strong><\/td>What happens<\/strong><\/td><\/tr><\/thead>
Convenience being over policy<\/strong><\/td>The ‘acceptable use policy’ is on the intranet. AI tools like ChatGPT are embedded within the browser and easier to access. Copy\/paste is also much easier.<\/td><\/tr>
PII becoming invisible<\/strong><\/td>After a while, you become numb to PII. ‘Email address’ and ‘phone number’ are just columns – as are health condition notes. They all look like the same plain text.<\/td><\/tr>
AI gains access to the data<\/strong><\/td>AI is now in business intelligence (BI) tools, integrated development environments (IDEs)<\/a>, emails, meeting apps, and more. And the ‘summarize with AI’ option is conveniently placed – and common.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

What are the most common ways customer data ends up in AI tools?<\/h2>\n\n\n\n
\n

The most common ways personal data ends up in AI tools like ChatGPT is through pasting data for clean-up, BI assistants summarizing dashboards, workspace ‘copilots’ reading over-shared files, coding assistants seeing result sets, emails, transcripts<\/strong> and screenshots<\/strong>, and browser extensions<\/strong>. <\/p>\n\n\n\n

Most look harmless at the time. <\/p>\n<\/blockquote>\n\n\n\n

Pasting data into ChatGPT to \u201cclean it up\u201d<\/h3>\n\n\n\n

Exporting and pasting a query into ChatGPT and prompting “dedupe this”<\/em>, “split the name column”<\/em>, etc. This happens when someone needs help with the data’s shape<\/em>, but actually pasted the data itself – triggering the accidental PII exposure.<\/p>\n\n\n\n

BI assistants summarizing dashboards<\/h3>\n\n\n\n

Most BI platforms now have an AI assistant summarizing report pages. The assistant will read whatever the report exports, including the tables with names and emails.

Enterprise-licensed assistants often keep data within the vendor’s boundary, but the summary can still end up in an email, a Slack channel, or a personal ChatGPT account.<\/p>\n\n\n\n

Workspace ‘copilots’ and permissions creep<\/h3>\n\n\n\n

AI ‘copilots’ in email and documents answer from everything a user can access. A customer spreadsheet shared with everyone is readable by the copilot too. In short: AI respects permissions, but everyone means everyone<\/em>.<\/p>\n\n\n\n

In fact, it’s how I once accidentally threw my social security (national ID) number into Claude Code<\/a>. I simply wasn’t paying attention to what I’d allowed the agent to see. Rookie mistake!<\/p>\n\n\n\n

Coding assistants and agents seeing result sets<\/h3>\n\n\n\n

Pasting a failing query is fine, but pasting error output or sample rows is not. The same goes for AI coding agents<\/a> given production database access to ‘debug’ something.<\/p>\n\n\n\n

\n
\n\n

You may also be interested in…<\/strong><\/p>\n\n\n\n

How to secure MCP servers connected to your database<\/a><\/p>\n\n<\/div>\n<\/div> \n\n\n

Emails, transcripts, and screenshots<\/h3>\n\n\n\n

Personal data lives outside of tables as well. Pasting an email thread to draft a reply, using a built-in AI notetaker\/assistant on a support call on Zoom or Microsoft Teams, or uploading a screenshot of a customer relationship manager (CRM)<\/a> record – these all give third-party software free access<\/em> to PII.<\/p>\n\n\n\n

Browser extensions and shared links<\/h3>\n\n\n\n

Any extension that can read page content can read your prompts. Be careful when discussing personal data in the browser, and remember that shared chat links can also expose pasted data to anyone<\/em> who has the URL<\/strong>.<\/p>\n\n\n\n

What happens to customer data after it\u2019s pasted into an AI tool?<\/h2>\n\n\n\n

The data is stored by the provider, on their terms. The data may be retained after deletion, and on consumer plans may be used for training. Depending on the account type:<\/p>\n\n\n\n

Account type<\/strong><\/td>Trains on your data by default?<\/strong><\/td><\/tr><\/thead>
Personal ChatGPT<\/td>Yes, unless the user opts out<\/td><\/tr>
ChatGPT Business \/ Enterprise \/ API<\/td>No<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Two caveats (even on business plans):<\/p>\n\n\n\n

It’s still a disclosure<\/strong>. Under the General Data Protection Regulation (GDPR)<\/a>, sending personal data to a processor needs a lawful basis and<\/em> a data processing agreement. A personal account does not fall under that.<\/p>\n\n\n\n

Deletion is hard to prove<\/strong>. Clearing a chat is not a guarantee of complete deletion.<\/p>\n\n\n\n

Be cautious – treat every prompt as an outbound data transfer.<\/strong><\/p>\n\n\n\n

How do you stop personal data leaking into AI tools?<\/h2>\n\n\n\n
\n

The goal is to make pasting data harmless, not to ban AI. A ban alone just pushes usage onto personal devices.<\/strong> <\/p>\n\n\n\n

In summary, here’s how to make pasting data harmless from the potential dangers of AI:<\/p>\n\n\n\n

How it leaks<\/th>Quickest fix<\/th><\/tr><\/thead>
Pasting data for clean-up<\/td>Masked or synthetic copies by default, plus an approved AI tool<\/td><\/tr>
BI (business intelligence) assistant summaries<\/td>Strip PII from semantic models; move customer-level drill-throughs into restricted reports<\/td><\/tr>
Workspace copilots<\/td>Fix \u201ceveryone\u201d sharing before rollout; test that row-level security holds<\/td><\/tr>
Coding assistants and agents<\/td>Masked dev data; no production access for agents<\/td><\/tr>
Emails, transcripts, screenshots<\/td>Approved AI notetaker; browser DLP (data loss prevention)<\/a> that also checks images<\/td><\/tr>
Browser extensions and shared links<\/td>Managed browser with an extension allowlist; disable public chat sharing<\/td><\/tr><\/tbody><\/table><\/figure>\n<\/blockquote>\n\n\n\n

Mask data at the source<\/h3>\n\n\n\n

Give analysts and developers masked or synthetic copies by default.<\/strong> Redgate Data Masker<\/a>, for example, keeps this consistent rather than a one-off script.<\/p>\n\n\n\n

Use views, column permissions or SQL Server’s Dynamic Data Masking<\/a><\/strong> so that reporting roles see region, not email. <\/p>\n\n\n\n

Classify PII columns first<\/strong>, then apply these anonymization techniques<\/a>.<\/strong><\/p>\n\n\n\n

Clean up BI tools and permissions before<\/em> enabling AI<\/h3>\n\n\n
\n