In the spring of 2023, a Samsung engineer uploaded internal source code to ChatGPT to get help with a problem. Within weeks, Samsung had banned employees from using generative AI tools. Nobody was trying to leak anything. Someone was trying to do their job faster.
That’s how most AI data leaks happen. Not through hackers, but through ordinary people pasting the wrong thing into the right tool, or the right thing into the wrong one.
I’m a big believer in using AI at work, and I don’t think the answer is to ban it. The answer is to know where your data goes and build a few habits that make leaks unlikely. That’s what this guide covers. It’s practical guidance, not legal advice, and if your company already has an AI policy, that comes first.
Where your data actually goes
When you type or upload something into an AI tool, it can travel further than the chat window suggests. There are four places to think about.
The provider’s servers
Your conversations and files are usually stored by the provider, at least for a while, so you can see your history and so the provider can investigate abuse. How long they’re kept, and who can access them, depends on the provider and on the plan you’re using.
Model training
Some services may use conversations to improve future models unless you opt out. Business and enterprise plans typically commit not to train on customer data by default, while personal accounts often have different defaults. Don’t assume. Check the settings and terms for the exact plan you’re signed into.
Sharing features
Share links and published chats can expose far more than you meant to. In the summer of 2025, people discovered that thousands of shared ChatGPT conversations were showing up in Google search results. The cause was a “make this chat discoverable” checkbox that many users ticked without realizing what it meant. OpenAI removed the option, and its chief information security officer, Dane Stuckey, said it had “introduced too many opportunities for folks to accidentally share things they didn’t intend to.”
The lesson outlived the feature. Treat a share button with the same care as forwarding an email.
Integrations, extensions, and agents
The newest risk comes from AI that’s connected to your other tools: assistants that read your email and drive, meeting bots that record and transcribe calls, browser extensions that see every page you open, and agents that can take actions on your behalf. Every connection widens what the AI can see, and what could go wrong.
Sort your data into four buckets
A simple classification makes most decisions obvious.
| Bucket | Examples | In an AI tool? |
|---|---|---|
| Public | Published blog posts, product pages, press releases | Fine in any tool |
| Internal | Meeting notes, internal processes, early drafts without sensitive details | Only in company-approved tools |
| Confidential | Unreleased financials, strategy, source code, contracts, pricing, deal plans | Only in approved tools, and only if your policy explicitly allows it |
| Restricted | Passwords, API keys, customer personal data, health or payment information, anything under NDA or regulation | Keep it out, unless a specific approved system is built for exactly this |
When you’re unsure, treat information as one bucket more sensitive than you think it is. You’ll rarely regret it.
Check which version of the tool you’re using
The same product can handle data very differently depending on which account you’re signed into. A personal free account and a company-managed business workspace might share a logo and not much else.
Before using an AI tool for work, find out:
- Is this the company-approved version? Signing in with your work account through company single sign-on is a good sign. A personal account on a work task is not.
- Is training on your data off? Look in the privacy or data controls settings, and in the plan’s terms.
- How long are conversations kept? Some tools let you shorten retention or use a temporary chat that isn’t saved to your history.
- Who else can see what you create? Check workspace sharing defaults and who can open shared links.
- Where is the data processed? This matters if your company has rules about which countries data can be stored in.
If you can’t answer these, ask your IT or security team before using the tool for anything beyond public information. If you are the IT team, which at a small company is often the case, write the answers down once so nobody has to guess.
Habits that prevent most leaks
None of these take more than a few seconds.
- Use placeholders. Swap names and identifiers for stand-ins like “Customer A” or “Project X.” The advice you get back is almost always just as good.
- Describe instead of pasting. “Our churn went from about 4% to 6% after a price change” gives the model what it needs without the spreadsheet behind it.
- Share the minimum. Paste the paragraph that matters, not the whole contract. Upload twenty sample rows, not the whole database.
- Never paste secrets. Passwords, API keys, and access tokens have no business in a prompt. If you need help with a configuration file, strip the secrets out first.
- Look at screenshots before you send them. It’s easy to crop to the error message and miss the customer list in the sidebar or the tab names along the top.
- Check files for hidden content. Documents can carry comments, tracked changes, hidden sheets, and metadata you forgot were there.
- Think twice before sharing a chat. Assume a shared link can end up anywhere.
Agents and connectors raise the stakes
An assistant you paste text into only sees what you give it. An agent connected to your accounts can see everything those accounts can see, and it may act on what it reads.
That creates a new kind of risk. Agents read content written by other people: emails, web pages, shared documents. Some of that content can carry hidden instructions designed to hijack the agent. This is called indirect prompt injection, and the OWASP Gen AI Security Project lists prompt injection as the top security risk for applications built on large language models. In principle, a single malicious email could instruct an over-trusted agent to forward your files somewhere.
A few defenses go a long way:
- Grant the least access that works. Connect one folder, not the whole drive. Use read-only access wherever you can.
- Require approval for actions. Sending, sharing, deleting, and buying should all pause for your confirmation.
- Keep experiments away from important accounts. An experimental agent and your main inbox are a risky pair.
- Review your connections. Every few months, remove integrations and browser extensions you no longer use.
Our explainer on what an AI agent is covers how agents work and how to try one safely.
If you’ve already pasted something you shouldn’t have
It happens. What you do next matters more than the mistake.
- Delete the conversation and any shared links to it. That won’t necessarily erase every copy, but it limits further exposure.
- Rotate any exposed credentials right away. Assume a pasted password or key is compromised.
- Tell your IT or security team. They may need to contact the provider or meet reporting obligations, and early notice makes that much easier.
- Write down what happened while it’s fresh: what was shared, in which tool, under which account, and when.
Report it quickly. Speaking as someone who runs a company, I’d much rather hear about a mistake on the day it happens than discover it months later.
A one-page AI policy for small teams
If your organization doesn’t have a policy yet, a short one beats none. It should cover:
- Approved tools, including the specific plans and accounts to use.
- Data rules, using the four buckets above.
- Human review for anything AI-assisted that goes to customers or the public. A short routine like the one in our guide on how to fact-check an AI answer is enough for most teams.
- Connections and agents: who can connect AI tools to company systems, and who has to approve it. Clear ownership is also what gets agent projects out of the pilot stage.
- Incidents: who to tell, and how fast, when something goes wrong.
Revisit it every few months. AI tools change their features, and sometimes their defaults, faster than most software. Choosing the right tool is part of the job too, and our guide on how to choose an AI model includes the questions to ask about where your data can go.
Sources
- Forbes Australia, Samsung bans ChatGPT among employees after sensitive code leak, May 2, 2023
- Engadget, OpenAI is removing ChatGPT conversations from Google, August 1, 2025
- OWASP Gen AI Security Project, LLM01:2025 Prompt Injection
