Personal data (like customer data) usually reaches an AI tool by accident, not by attack. Someone is busy and the chat window is one CTRL+V away, so it’s a quick and easy solution.
The most common ways personal data leaks into AI tools? Clean-up requests, BI assistant summaries, over-shared files read by copilots, coding assistants, call transcripts, and browser extensions.
Thankfully, you can prevent this happening without ‘banning’ the use of AI by masking data at the source, fixing permissions before enabling AI ‘copilots’, providing an approved enterprise AI tool, and adding technical guardrails such as browser data loss prevention (DLP).
How (and why) does personal data leak into AI tools by accident?
A data analyst pastes a CSV into an AI tool like ChatGPT for “clean up”. A business intelligence (BI) assistant then uses GPT to summarize a dashboard full of customer emails. Nobody intended to share personal data, but they did.
My previous article on anonymizing PII in LLM pipelines covered the pipelines you design, whereas here I’m focusing on the pipelines not designed to ingest PII (personally identifiable information).
I’ll explain how personal data (or customer data from an organization’s perspective) slips into AI tools by accident, and how to stop it without banning the use of AI.
From both a personal and professional perspective, the fastest way to get work done nowadays is via an LLM (large language model) – and naturally, we’re all handling data of some sort every day. Data that, depending on the context, is usually full of PII.
That ends up being fed into the LLM – and this is often missed. Here are a few of the reasons this can happen within an organization/business:
| Reason | What happens |
| Convenience being over policy | The ‘acceptable use policy’ is on the intranet. AI tools like ChatGPT are embedded within the browser and easier to access. Copy/paste is also much easier. |
| PII becoming invisible | After a while, you become numb to PII. ‘Email address’ and ‘phone number’ are just columns – as are health condition notes. They all look like the same plain text. |
| AI gains access to the data | AI is now in business intelligence (BI) tools, integrated development environments (IDEs), emails, meeting apps, and more. And the ‘summarize with AI’ option is conveniently placed – and common. |
What are the most common ways customer data ends up in AI tools?
The most common ways personal data ends up in AI tools like ChatGPT is through pasting data for clean-up, BI assistants summarizing dashboards, workspace ‘copilots’ reading over-shared files, coding assistants seeing result sets, emails, transcripts and screenshots, and browser extensions.
Most look harmless at the time.
Pasting data into ChatGPT to “clean it up”
BI assistants summarizing dashboards
Workspace ‘copilots’ and permissions creep
Coding assistants and agents seeing result sets
You may also be interested in…
Emails, transcripts, and screenshots
Browser extensions and shared links
What happens to customer data after it’s pasted into an AI tool?
The data is stored by the provider, on their terms. The data may be retained after deletion, and on consumer plans may be used for training. Depending on the account type:
| Account type | Trains on your data by default? |
| Personal ChatGPT | Yes, unless the user opts out |
| ChatGPT Business / Enterprise / API | No |
Two caveats (even on business plans):
It’s still a disclosure. Under the General Data Protection Regulation (GDPR), sending personal data to a processor needs a lawful basis and a data processing agreement. A personal account does not fall under that.
Deletion is hard to prove. Clearing a chat is not a guarantee of complete deletion.
Be cautious – treat every prompt as an outbound data transfer.
How do you stop personal data leaking into AI tools?
The goal is to make pasting data harmless, not to ban AI. A ban alone just pushes usage onto personal devices.
In summary, here’s how to make pasting data harmless from the potential dangers of AI:
How it leaks Quickest fix Pasting data for clean-up Masked or synthetic copies by default, plus an approved AI tool BI (business intelligence) assistant summaries Strip PII from semantic models; move customer-level drill-throughs into restricted reports Workspace copilots Fix “everyone” sharing before rollout; test that row-level security holds Coding assistants and agents Masked dev data; no production access for agents Emails, transcripts, screenshots Approved AI notetaker; browser DLP (data loss prevention) that also checks images Browser extensions and shared links Managed browser with an extension allowlist; disable public chat sharing
Mask data at the source
Give analysts and developers masked or synthetic copies by default. Redgate Data Masker, for example, keeps this consistent rather than a one-off script.
Use views, column permissions or SQL Server’s Dynamic Data Masking so that reporting roles see region, not email.
Classify PII columns first, then apply these anonymization techniques.
Clean up BI tools and permissions before enabling AI
- Strip PII from semantic models that only need counts and trends.
- Move customer-level drill-throughs into restricted reports.
- Fix files and sites shared with “everyone” before rolling out a copilot.
- Check which LLM processes requests and where, and test that labels and row-level security actually hold.
Give people an approved AI tool they’ll actually use
People use personal accounts because it’s easiest. Provide an enterprise AI plan with training off and SSO, make it the default everywhere, and write a one-line rule: no customer names, contact details, payment data or support notes in any AI tool except the approved one.
Add technical guardrails
- Add a browser DLP that warns on emails, card numbers and IDs pasted or uploaded to AI domains (including images).
- Manage browsers with an extension allowlist (whitelist).
- Disable public chat sharing wherever – and whenever – you can.
Checklist: what to check before you paste anything into an AI tool
If you can’t tick every box, strip the data first.
Done Question ☐ Am I using the approved AI tool on my work account? ☐ Have I replaced real values with masked or synthetic ones (since the model only needs the structure?) ☐ Have I removed names, contact details and IDs (including from free-text fields and screenshots?) ☐ Have I checked that any error output or result rows pasted alongside my query contain zero personal data? ☐ Is this chat staying private, with no plan to share it or its output more widely?
If you slip up, delete the conversation and tell your data protection officer (DPO) straight away.
Accidental AI data leaks are a design problem, not a people problem
Accidental AI data leaks are a design problem, not a people problem. Customer data reaches ChatGPT because it’s easy to reach and easy to overlook.
Mask it at the source, keep identities out of reports that don’t need them, fix permissions before copilots read them, and give people an approved tool they want to use.
Pair this with the five anonymization techniques for LLM pipelines to cover data sent on purpose and by accident.
And, to automate masking, why not try Redgate Data Masker free for 14 days?
Enhance your data security with an automated masking approach
FAQs: personal data and AI tools like ChatGPT
1. Does ChatGPT train on the data you paste into it?
On personal ChatGPT accounts, conversations may be used to train models unless the user opts out. ChatGPT Business, Enterprise and API usage isn’t used for training by default. Even then, pasting personal data is still a disclosure to a third party, so under GDPR you need a lawful basis and a data processing agreement.
2. Is pasting customer data into ChatGPT a GDPR breach?
It can be. Sending personal data to a processor needs a lawful basis and a data processing agreement, and a personal ChatGPT account has neither. Enterprise plans with a DPA reduce the risk but don’t remove your duty to minimize the data you share. Check with your DPO or legal team for your situation.
3. How do you stop employees pasting personal customer data into AI tools?
Mask or synthesize data at the source so anything pasted is harmless, provide an approved enterprise AI tool with training off, and add browser DLP that warns on emails, card numbers and IDs sent to AI domains. A policy alone isn’t enough, and a ban pushes people onto personal accounts.
4. Can AI assistants in BI tools and copilots see personal customer data?
Yes. BI assistants read whatever a report exports, and workspace copilots answer from everything a user can access, including files shared with everyone. Strip PII from semantic models, restrict personal customer-level reports, and fix over-shared files before enabling AI features.
5. What should you do if you’ve pasted personal customer data into an AI tool?
Delete the conversation and tell your DPO straight away. Deleting a chat isn’t proof of complete removal, so report it rather than relying on deletion alone. Then check whether the account was personal or approved, and whether the chat or its output was shared further.
This document contains proprietary information and is protected by copyright law.
Copyright © 2026 Red Gate Software Limited. All rights reserved
Load comments