Best Practices for Conversational Data Privacy

How To Handle Sensitive Information in your next AI Project It's crucial to handle sensitive user information with care. Whether it's personal data, financial details, or health information, understanding how to protect and manage it is essential to maintain trust and comply with privacy regulations. Here are 5 best practices to follow: 1. Identify and Classify Sensitive Data Start by identifying the types of sensitive data your application handles, such as personally identifiable information (PII), sensitive personal information (SPI), and confidential data. Understand the specific legal requirements and privacy regulations that apply, such as GDPR or the California Consumer Privacy Act. 2. Minimize Data Exposure Only share the necessary information with AI endpoints. For PII, such as names, addresses, or social security numbers, consider redacting this information before making API calls, especially if the data could be linked to sensitive applications, like healthcare or financial services. 3. Avoid Sharing Highly Sensitive Information Never pass sensitive personal information, such as credit card numbers, passwords, or bank account details, through AI endpoints. Instead, use secure, dedicated channels for handling and processing such data to avoid unintended exposure or misuse. 4. Implement Data Anonymization When dealing with confidential information, like health conditions or legal matters, ensure that the data cannot be traced back to an individual. Anonymize the data before using it with AI services to maintain user privacy and comply with legal standards. 5. Regularly Review and Update Privacy Practices Data privacy is a dynamic field with evolving laws and best practices. To ensure continued compliance and protection of user data, regularly review your data handling processes, stay updated on relevant regulations, and adjust your practices as needed. Remember, safeguarding sensitive information is not just about compliance — it's about earning and keeping the trust of your users.

This Stanford study examined how six major AI companies (Anthropic, OpenAI, Google, Meta, Microsoft, and Amazon) handle user data from chatbot conversations.  Here are the main privacy concerns. 👀 All six companies use chat data for training by default, though some allow opt-out 👀 Data retention is often indefinite, with personal information stored long-term 👀 Cross-platform data merging occurs at multi-product companies (Google, Meta, Microsoft, Amazon) 👀 Children's data is handled inconsistently, with most companies not adequately protecting minors 👀 Limited transparency in privacy policies, which are complex and hard to understand and often lack crucial details about actual practices Practical Takeaways for Acceptable Use Policy and Training for nonprofits in using generative AI: ✅ Assume anything you share will be used for training - sensitive information, uploaded files, health details, biometric data, etc. ✅ Opt out when possible - proactively disable data collection for training (Meta is the one where you cannot) ✅ Information cascades through ecosystems - your inputs can lead to inferences that affect ads, recommendations, and potentially insurance or other third parties ✅ Special concern for children's data - age verification and consent protections are inconsistent Some questions to consider in acceptable use policies and to incorporate in any training. ❓ What types of sensitive information might your nonprofit staff  share with generative AI?  ❓ Does your nonprofit currently specifically identify what is considered “sensitive information” (beyond PID) and should not be shared with GenerativeAI ? Is this incorporated into training? ❓ Are you working with children, people with health conditions, or others whose data could be particularly harmful if leaked or misused? ❓ What would be the consequences if sensitive information or strategic organizational data ended up being used to train AI models? How might this affect trust, compliance, or your mission? How is this communicated in training and policy? Across the board, the Stanford research points that developers’ privacy policies lack essential information about their practices. They recommend policymakers and developers address data privacy challenges posed by LLM-powered chatbots through comprehensive federal privacy regulation, affirmative opt-in for model training, and filtering personal information from chat inputs by default. “We need to promote innovation in privacy-preserving AI, so that user privacy isn’t an afterthought." How are you advocating for privacy-preserving AI? How are you educating your staff to navigate this challenge? https://lnkd.in/g3RmbEwD

"Be careful what you tell your chatbot." Padmini Soni and I unpacked the recent Stanford HAI study on how major AI platforms handle our chat data during our recent #AsianWomenAdvancingAI(AWAAI) roundtable.   Spoiler: it’s… a lot. 🤯   We talked about how people are quietly using chatbots for some of the most sensitive parts of their lives: 💬 Practicing hard conversations with family 🧪 Pasting lab results to “translate” medical jargon 💼 Uploading full resumes with phone, email, and location 🧠 Even leaning on chatbots as therapists   And yet, many of these platforms: -Store chats (sometimes for a long time, "temporary" ≠ "deleted") -May use them for training or profiling -May have human reviewers reading snippets of our conversations -Allow teens (and in practice, younger kids) to use tools that were never designed as mental health or safety-critical systems   A few big takeaways from the conversation: 🔐 Privacy ≠ just "no PII" Even if you remove your name, your location, health details, and relationship drama can make you highly identifiable, especially when combined with other data points. 🧸 Kids are using these tools… without meaningful consent Parents shared stories of 10–11 year olds asking chatbots how to argue with their parents (I know it sounds funny). Most platforms are not built with child emotional safety in mind. 🧠 Chatbots simulate empathy, they don’t feel it They're pattern machines, not people. The "sweet," validating tone can create emotional dependence, especially when human support is expensive, stigmatized, or hard to access. 🏢 Even "enterprise" and API use isn't always magically safe Founders in regulated industries are seeing big vendors suddenly support niche use cases that should rely on private data. That raises hard questions about what’s really being used under the hood. So what can we actually do right now? -Assume anything you paste into a general-purpose chatbot could be stored and reviewed -Strip or mask personal + client data whenever possible -Use tools with strong privacy / no-training guarantees for sensitive workflows (or self-hosted / on-device where feasible) -Talk to your kids, teams, and parents about what not to put in chat   At AWAAI, we're not anti-AI. We're pro-aware, pro-consent, and pro-human.   ICYM the recording, here it is: https://lnkd.in/eyWdzijg Zeba Karkhanawala Sri Ramaswamy Sandrine Mujinga Supriya Ramarao Prasanna Ramya Ganesh Megha Vithalani Amruta Ambre Evan Benjamin Cherie Lejeune Donna Rinck Rupa Shah Jill Stover Heinze Jasmine Schwarz Sai Aparna Mopuru Khusshboo Mehta Urvashi Batra Tami DeWeese Usha Jagannathan, PhD Seema Alexander Alan Zavala Nibha Prasad   #AI # #ResponsibleAI #AIethics #AWAAI #StanfordHAI #GenerativeAI  

Which AI Chatbot Protects Your Privacy Best? A Deep Dive Into Data Collection Practices Introduction AI chatbots are now mainstream, but their privacy practices vary widely. A review of leading chatbot apps reveals stark differences in how much data they collect, how they use it, and whether they share it with advertisers or use your prompts to train their models. Key Findings From Privacy Reports • Many chatbots are far more invasive than users expect, collecting device IDs, chat logs, keystrokes, browsing activity, and sometimes precise location. • Google’s Gemini is the most data-hungry, pulling browsing history, contacts, photos, emails, search history, videos, and more. • Qwen reports minimal data collection, but its privacy report conflicts with its policy and contains errors, raising trust concerns. • DeepSeek collects extensive user data and is based in China, where national law grants the government broad access to user information. Which Chatbot Has the Best Privacy Policy? • Microsoft Copilot stands out as the most privacy-protective option. – Collects minimal customer data – Does not share information with advertisers – Does not use your prompts or outputs to train foundation models – Complies with FedRAMP, HIPAA, and SOC standards • ChatGPT and others use your prompts for training unless you disable history or use enterprise tiers. • Google’s policy is thorough and transparent, but Gemini still collects the most data and allows human reviewers to read chats unless history is disabled. Why Data Sovereignty Matters • Foreign AI companies, especially in China, operate under laws granting government access to all user data. • DeepSeek’s rapid rise is sparking cybersecurity and privacy warnings from experts who see parallels to the TikTok concerns. • Even US-based tools gather more data than many users realize, reinforcing the need for stronger cross-industry privacy frameworks. How to Protect Yourself • Avoid mobile chatbot apps, which collect the most data. • Run models locally on your computer using tools like Ollama and open-source LLMs such as DeepSeek R1. • Use on-device AI features built into new PCs and GPUs, which process data without internet transmission. • Disable chat history wherever possible to reduce human review and limit training use. Why This Matters AI chatbots are now embedded in daily workflows, but privacy practices lag behind adoption. As global tensions rise and AI becomes central to personal and workplace productivity, data protection is no longer optional. Copilot’s approach shows a privacy-first model is achievable. The question is whether competitors will follow suit — or continue treating user data as fuel for training and advertising. I share daily insights with 34,000+ followers across defense, tech, and policy. If this topic resonates, I invite you to connect and continue the conversation. Keith King https://lnkd.in/gHPvUttw

The most dangerous adversary in the era of AI is… you. Before you dismiss that, hear me out. Security industry reports continue to agree on one thing year over year: the human element is the primary cause of breaches. Not because people are malicious, but because we are human. AI amplifies this problem in a very unique way that I find fascinating. Because it interact with us in human-like ways. That means our own cognition and biases become part of the attack surface. We see it in the news and industry reports. Employees paste sensitive data like financials, strategy docs, and customer records into unmanaged AI chatbots. Often through personal accounts. Completely invisible to corporate security. We know this is risky. So why do we still do it? In my opinion, it's not lack of security training. I think the answer is rooted in human psychology and how the human OS works. When a chatbot communicates in natural, conversational language, we instinctively respond as if it were trusted human. Psychologists call this anthropomorphism: our tendency to assign human qualities to non-human systems. That perception changes our judgment. Furthermore, under cognitive load or time pressure, the brain defaults to shortcuts, and an unmanaged AI chats without guardrails feels like the fastest solution. Social proof reinforces this. We see colleagues or the broader industry using unmanaged AI freely, so we fall into the "everyone is doing it" trap. The conversational tone encourages disclosure, and we share more than we intend, much like we would in a trusted dialogue. It starts with a quick summary. It ends with company secrets sitting in a chatbot.💀 Do this instead: 🔒Pause three seconds before you paste. Ask: Would I be okay if this data appeared outside the company? If not, don’t paste. 🔒Use approved company AI tools with proper guardrails. 🔒When in doubt, ask your security team. AI is a force multiplier for speed... and for mistakes. Treat the chat box like a public space, not a diary. Your best defense is one deliberate pause before you press Enter.

6 Ways Companies Can Make ChatGPT Usage Safer 💼🔐 There have been reported cases and studies where private or other sensitive data, such as emails, phone numbers, names, and other information, have been extracted from ChatGPT either by: 🛑 Prompt Injection Attacks: In this scenario, malicious prompts are used to bypass ChatGPT's internal safeguards, extracting sensitive information by manipulating the model into revealing data it was not meant to disclose. ⚠️ Data Leakage from Training: AI models like ChatGPT have been found to unintentionally reveal sensitive information that was part of their training datasets. This can occur when the model is prompted in ways that lead to the accidental output of memorized data. Despite these risks, ChatGPT offers significant advantages, and more and more people are using it in the work environment. It has become increasingly critical for companies to develop guidelines, frameworks, and train their staff on the responsible and secure use of ChatGPT. Here are six ways companies can make ChatGPT usage safer: 🔒 Implement Data Privacy and Protection Measures: Set clear rules preventing the entry of sensitive data, ensuring compliance with privacy regulations like GDPR and CCPA. 👀 Establish Human Oversight for High-Risk Outputs: Introduce a process for reviewing critical ChatGPT outputs before they are released to ensure accuracy. 📊 Use a Risk Assessment Framework: Categorize ChatGPT use cases based on risk levels, with higher-risk tasks requiring stricter controls. ⚖️ Address Ethical and Bias Concerns: Regularly audit ChatGPT outputs to detect and correct any biases or ethical issues. 🎓 Provide Employee Training on ChatGPT Use: Educate employees on acceptable and secure uses of ChatGPT, emphasizing both benefits and risks. 📈 Monitor and Report AI Usage: Track all ChatGPT interactions in real time to ensure compliance with internal policies and address misuse swiftly. By implementing these strategies, companies can reduce the risks associated with ChatGPT usage and ensure responsible AI deployment. What are your experiences with companies training their employees to use AI tools like ChatGPT? 🤔 Please share! #AIinBusiness #DataPrivacy #AIGovernance #ResponsibleAI #ChatGPTSafety

{Please} Think twice before prompting personal information The article by Nicole Nguyen highlights the risks of oversharing with AI chatbots like ChatGPT, especially when the tools feel conversational and trustworthy. Users often forget that once information is typed into a chatbot, control over it may be lost. Personal data—ranging from medical records to sensitive financial or company information—can end up being stored, reviewed, or even exposed in data breaches. Although some chatbots promise to minimize the collection of personal details, past incidents have shown that even well-known services are vulnerable to leaks. There are five key examples of what not to share with AI: first, identity information such as Social Security or passport numbers; second, medical results, which lack the special protections they would receive in clinical settings; third, financial account details that could lead to unauthorized access; fourth, corporate data that could unintentionally expose confidential material; and fifth, login credentials, which chatbots are not designed to securely store. https://lnkd.in/eZhxtbQX To protect privacy, the article recommends opting out of training data use in chatbot settings, using temporary chat modes like ChatGPT’s Incognito-style feature, deleting chat histories frequently, anonymizing questions through tools like Duck.ai, and avoiding platforms with unclear or unrestricted data policies, such as DeepSeek. https://lnkd.in/epgg-5TX

Reuters reported this week that OpenAI is fighting a court order to turn over about 20 million ChatGPT conversations tied to a lawsuit. Read it here: https://lnkd.in/eriJiKrC   It's so easy to treat GPTs like an assistant, friend, or colleague. It's also easy for us to forget that we are subscribers to these tools and instead of the owner, which means we don't keep control of our information. Unfortunately, this lawsuit is a huge reminder that not all versions of these tools work the same way and that people don't know (or don't care until it's too late) how their data is handled behind the scenes.   The consumer versions of ChatGPT, both free and paid, keep interaction data to train the model (yes, you can control this a little bit, but I'd be curious to see what data really is retained on the back end that may be turned over in this suit). Enterprise and API setups can be configured for zero data retention, which means the prompts and outputs are deleted after processing- but do your employees know that and are they using your model and not the publicly available one?   If you work with client information, case material, or anything operational, it should not run through a consumer account. Keep your exploratory testing there if you need it, but move real work to enterprise or API environments where you know how the data is handled. Keep your own record of what you did, when you did it, and which model you used so your process is clear and repeatable.   You also have the option to host your own AI models offline so you retain full control over your data. Running open source models on your own hardware gives you direct control over what is stored, deleted, or logged.   Be deliberate about where you put your data. I imagine it's not long before something becomes public about someone violating an NDA by putting protected information in a public system like this and training a model on it by accident. Be careful out there!   #OSINT #AI #DataPrivacy #Governance #Investigations #OpenSourceIntelligence #PangeaResearch

When I’m working with organizations on their AI strategy, I often get asked about the risks of using AI—especially when it comes to privacy and confidential data. And the concern is valid. A recent article reported that some AI tools are leaking users’ private chats onto the open web. Think HR notes, employee conversations, performance feedback—data that was never meant to be public. Here’s the takeaway: • AI tools can be incredibly powerful, but without clear governance, they’re risky. • What seems like a simple prompt could expose sensitive employee data. • And once trust is lost, it’s hard to get back. So what can you do? 1. Audit your tools—know what’s being used and how it stores data. 2. Create clear guidelines—make sure your team understands what not to share. 3. Train your people—especially in departments like HR, Legal, and Finance. 4. Partner with experts—companies like AixHR specialize in helping HR teams use AI securely and effectively (without the app sprawl or fear factor). You don’t have to hit pause on AI—you just need a smart strategy. One that elevates your work without compromising your values. If you’re thinking about your next move with AI, let’s talk. #AIinHR #DataPrivacy #ResponsibleAI #HRLeadership #AixHR #ElevateNotEliminate

✨Guess what is the most underrated LLM #promptinjection risk for enterprises in 2024? Hint: It's not arbitrary code exec, hallucination, model tampering, overspending, detrimental decision-making... 👇 It's data disclosure. Why? LLMs are too kind and helpful: they cannot refrain themselves from giving away what they have been trained to know (or what they have access to), and, unlike other risks, I haven't seen operational, precise and efficient #cyber countermeasures against LLM innocence yet. In 2023, we've seen countless examples of sibylline prompts leading to arbitrary prompt-engineered data disclosure (kudos to Peter Gostev for sharing masterful examples of such context-escaping throughout the year). So what should we do? My favorite mitigation is called the #IAM reflection pattern: a corporate LLM must only have access to what her consumers (machines or humans) are entitled to. In 2024, THIS pattern must be the main driver for: a) designing secure boundaries between a conversational #AI and backend applications b) breaking down LLMs instances not along business functionality axes, but along need-to-know axes In the worst case, the client will only get what she already knows (or ought to know). LLM will be bound to provide a reflection of the client's own knowledge. LLM will act as a (very) sophisticated mirror. No more escaping. #databreach #privacy #confidentiality #security Dor Sarig Itamar Golan Didier Girard Sean Poris Oliver Cronk