Understanding User Consent and Data Ownership

This new white paper by Stanford Institute for Human-Centered Artificial Intelligence (HAI) titled "Rethinking Privacy in the AI Era" addresses the intersection of data privacy and AI development, highlighting the challenges and proposing solutions for mitigating privacy risks. It outlines the current data protection landscape, including the Fair Information Practice Principles, GDPR, and U.S. state privacy laws, and discusses the distinction and regulatory implications between predictive and generative AI. The paper argues that AI's reliance on extensive data collection presents unique privacy risks at both individual and societal levels, noting that existing laws are inadequate for the emerging challenges posed by AI systems, because they don't fully tackle the shortcomings of the Fair Information Practice Principles (FIPs) framework or concentrate adequately on the comprehensive data governance measures necessary for regulating data used in AI development. According to the paper, FIPs are outdated and not well-suited for modern data and AI complexities, because: - They do not address the power imbalance between data collectors and individuals. - FIPs fail to enforce data minimization and purpose limitation effectively. - The framework places too much responsibility on individuals for privacy management. - Allows for data collection by default, putting the onus on individuals to opt out. - Focuses on procedural rather than substantive protections. - Struggles with the concepts of consent and legitimate interest, complicating privacy management. It emphasizes the need for new regulatory approaches that go beyond current privacy legislation to effectively manage the risks associated with AI-driven data acquisition and processing. The paper suggests three key strategies to mitigate the privacy harms of AI: 1.) Denormalize Data Collection by Default: Shift from opt-out to opt-in data collection models to facilitate true data minimization. This approach emphasizes "privacy by default" and the need for technical standards and infrastructure that enable meaningful consent mechanisms. 2.) Focus on the AI Data Supply Chain: Enhance privacy and data protection by ensuring dataset transparency and accountability throughout the entire lifecycle of data. This includes a call for regulatory frameworks that address data privacy comprehensively across the data supply chain. 3.) Flip the Script on Personal Data Management: Encourage the development of new governance mechanisms and technical infrastructures, such as data intermediaries and data permissioning systems, to automate and support the exercise of individual data rights and preferences. This strategy aims to empower individuals by facilitating easier management and control of their personal data in the context of AI. by Dr. Jennifer King Caroline Meinhardt Link: https://lnkd.in/dniktn3V

The Oregon Department of Justice released new guidance on legal requirements when using AI. Here are the key privacy considerations, and four steps for companies to stay in-line with Oregon privacy law. ⤵️ The guidance details the AG's views of how uses of personal data in connection with AI or training AI models triggers obligations under the Oregon Consumer Privacy Act, including: 🔸Privacy Notices. Companies must disclose in their privacy notices when personal data is used to train AI systems. 🔸Consent. Updated privacy policies disclosing uses of personal data for AI training cannot justify the use of previously collected personal data for AI training; affirmative consent must be obtained. 🔸Revoking Consent. Where consent is provided to use personal data for AI training, there must be a way to withdraw consent and processing of that personal data must end within 15 days. 🔸Sensitive Data. Explicit consent must be obtained before sensitive personal data is used to develop or train AI systems. 🔸Training Datasets. Developers purchasing or using third-party personal data sets for model training may be personal data controllers, with all the required obligations that data controllers have under the law. 🔸Opt-Out Rights. Consumers have the right to opt-out of AI uses for certain decisions like housing, education, or lending. 🔸Deletion. Consumer #PersonalData deletion rights need to be respected when using AI models. 🔸Assessments. Using personal data in connection with AI models, or processing it in connection with AI models that involve profiling or other activities with heightened risk of harm, trigger data protection assessment requirements. The guidance also highlights a number of scenarios where sales practices using AI or misrepresentations due to AI use can violate the Unlawful Trade Practices Act. Here's a few steps to help stay on top of #privacy requirements under Oregon law and this guidance: 1️⃣ Confirm whether your organization or its vendors train #ArtificialIntelligence solutions on personal data.  2️⃣ Validate your organization's privacy notice discloses AI training practices. 3️⃣ Make sure organizational individual rights processes are scoped for personal data used in AI training. 4️⃣ Set assessment protocols where required to conduct and document data protection assessments that address the requirements under Oregon and other states' laws, and that are maintained in a format that can be provided to regulators.

📸Meta’s request for camera roll access signals a critical inflection point in AI development—one that reveals the inadequacy of our current consent frameworks for both individuals and organizations. The core issue isn’t privacy alone. It’s the misalignment between how AI systems learn and how humans actually share. When we post a photo publicly, we’re making a deliberate choice—about context, audience, meaning. Camera roll access bypasses that intentionality entirely. Your unshared photos hold different signals: 📍 family moments 📍 screenshots of private conversations 📍 creative drafts 📍 work documents All of it becomes potential training data—without your explicit intent. For individuals, this shift creates three serious concerns: 1. Consent erosion — the boundary between “what I share” and “what gets analyzed” disappears 2. Context collapse — meaning is flattened when private data fuels generalized models 3. Invisible labor — your memories become unpaid inputs for commercial systems For organizations, the implications are just as pressing: 🔹 Data strategy: Companies must distinguish between available data and appropriate data. Consent isn’t binary—it’s contextual and evolving. 🔹 Long-term trust: The businesses that optimize for genuine user agency—not maximum data extraction—will be the ones that sustain real relationships and build better systems. Here’s a quick evaluation framework I use: ✅ Does this data improve the specific task the user requested? ✅ Could similar results be achieved with targeted, user-controlled input? ✅ Are we optimizing for system performance or user autonomy? The future of AI will be shaped by these choices. Not just what we can do with data—but what we choose to honor. We need systems that amplify human judgment, not bypass it. Design that aligns with consent, not convenience. The question isn’t just: can AI understand us? It’s: will it respect how we want to be understood? → How are you thinking about these trade-offs in your personal tech use? → And if you’re building AI—what frameworks are you using to balance capability with care? #AIethics #ConsentByDesign #RelationalAI #ResponsibleInnovation #MetaAI #DataGovernance #DigitalSovereignty #WeCareImpact

Today, a recruiter invited me to a call about a potential role I was very interested in learning more about. But, less than an hour before the meeting, I received a sudden calendar update: “Fred from Fireflies will join to record and transcribe the conversation.” - No prior request for consent. - No explanation of how the recording would be stored. - No clear details on how my data might be used. What should have been a straightforward conversation instantly shifted into a scramble to protect my privacy (voice, image, and data). Recording an interview, without clear, advance permission, erodes trust before the first question is even asked. Consent is a deliberate agreement that lets everyone show up prepared and comfortable. This is an ethical issue. No doubt, an AI note-taker could be valuable to this recruiter. But, they also raise questions about data retention, confidentiality, and intellectual property. A candidate discussing career history, research, or sensitive client details deserves to know exactly how those records will be used and who will have access. If you truly aim to build an inclusive hiring process, plan for ethical recording practices from the first email. - State your intentions. - Outline how the file will be stored and data retention policies. - Offer alternative accommodations. - Secure explicit consent well before the call. Anything less feels like surveillance disguised as efficiency. How are you making sure your use of AI tools in interviews respects privacy, consent, and accessibility? *Note, I am fortunate to be able to walk away from situations that violate my privacy, and I did exactly that in this case. I recognize that many candidates cannot afford to decline and must navigate similar scenarios without the option to stay no. If you are in that position, I see you and stand with you. #CyberSecurity #DataPrivacy #Consent

Today's AI contract tip is about the problems with using ownership when describing data rights and obligations. Ownership is a legal status. The person who owns something has the title, and with that title comes some rights. If the title is to intellectual property, the owner has the right to exclude others from doing things (patents and trade secrets) or the exclusive right to do things (copyright and trademark). Parties can also own other property, like goods, buildings, and land. Most AI contracts refer to data ownership, but the party with the data rarely meets the legal definition of ownership. Instead, that party has control. They collected, enhanced, and stored the information themselves or received that data from someone else who did. Subject to any legal or contractual restrictions, the party with the data can use and share that data with others. Talking casually about owning data isn’t that big of a deal in a lot of situations, but it is a big problem when we do that in contracts. Contracts define our relationship with our counterparties. If we have inaccurate ownership claims there, we may expose ourselves to implied warranties and an inability to enforce incorrect terms on their face. Here are four ways to draft better data provisions in your AI product contracts: 1. Focus on rights and usage, not ownership - You don't need to discuss ownership if there's no IP involved. Instead, create contractual provisions that address what each party can do with the data. Can they analyze it? Store it? Share it with third parties? The key is specificity, not broad claims of ownership. And be precise about how long the rights last. 2. List prohibited uses - Don't rely on generic restrictions. Get specific about what they can’t do with the data. Make sure you align with any legal restrictions that apply to your data, but you may want to go beyond those laws. For example, do you also want to prohibit using the data to develop competing products or selling data-derived insights to competitors? 3. Create accountability - Standard audit rights probably aren’t enough. Look at other ways to check, including compliance certificates or data logs that show how the data is being used. 4. Customize your remedies - Look at what you need if the counterparty violates the contractual restrictions. You may leave yourself exposed if you rely on typical termination rights and contract breach claims. Update your contract to provide a better path, especially when data misuse is a legal violation. Your contract should address what matters. Avoid the imprecise messiness of relying on ownership to do that. Instead, define exactly what rights each party has and what they can do with the data. What other advice would you add about data ownership in contracts? #AIContractTips #Contracts #HowToContract

During seed round due diligence, we found a red flag: the startup didn’t have rights to the dataset used to train its LLM and hadn’t set up a privacy policy for data collection or use. AI startups need to establish certain legal and operational frameworks to ensure they have and maintain the rights to the data they collect and use, especially for training their AI models. Here are the key elements for compliance: 1. Privacy Policy: A comprehensive privacy policy that clearly outlines data collection, usage, retention, and sharing practices. 2. Terms of Service/User Agreement: Agreements that users accept which should include clauses about data ownership, licensing, and how the data will be used. 3. Data Collection Consents: Explicit consents from users for the collection and use of their data, often obtained through clear opt-in mechanisms. 4. Data Processing Agreements (DPAs): If using third-party services or processors, DPAs are necessary to define the responsibilities and scope of data usage. 5. Intellectual Property Rights: Ensure that the startup has clear intellectual property rights over the collected data, through licenses, user agreements, or other legal means. 6. Compliance with Regulations: Adherence to relevant data protection regulations such as GDPR, CCPA, or HIPAA, which may dictate specific requirements for data rights and user privacy. 7. Data Anonymization and Security: Implementing data anonymization where necessary and ensuring robust security measures to protect data integrity and confidentiality. 8. Record Keeping: Maintain detailed records of data consents, privacy notices, and data usage to demonstrate compliance with laws and regulations. 9. Data Audits: Regular audits to ensure that data collection and usage align with stated policies and legal obligations. 10. Employee Training and Policies: Training for employees on data protection best practices and establishing internal policies for handling data. By having these elements in place, AI startups can help ensure they have the legal rights to use the data for training their AI models and can mitigate risks associated with data privacy and ownership. #startupfounder #aistartup #dataownership

The EDPB recently published a report on AI Privacy Risks and Mitigations in LLMs.   This is one of the most practical and detailed resources I've seen from the EDPB, with extensive guidance for developers and deployers. The report walks through privacy risks associated with LLMs across the AI lifecycle, from data collection and training to deployment and retirement, and offers practical tips for identifying, measuring, and mitigating risks.   Here's a quick summary of some of the key mitigations mentioned in the report:   For providers: • Fine-tune LLMs on curated, high-quality datasets and limit the scope of model outputs to relevant and up-to-date information. • Use robust anonymisation techniques and automated tools to detect and remove personal data from training data. • Apply input filters and user warnings during deployment to discourage users from entering personal data, as well as automated detection methods to flag or anonymise sensitive input data before it is processed. • Clearly inform users about how their data will be processed through privacy policies, instructions, warning or disclaimers in the user interface. • Encrypt user inputs and outputs during transmission and storage to protect data from unauthorized access. • Protect against prompt injection and jailbreaking by validating inputs, monitoring LLMs for abnormal input behaviour, and limiting the amount of text a user can input. • Apply content filtering and human review processes to flag sensitive or inappropriate outputs. • Limit data logging and provide configurable options to deployers regarding log retention. • Offer easy-to-use opt-in/opt-out options for users whose feedback data might be used for retraining.   For deployers: • Enforce strong authentication to restrict access to the input interface and protect session data. • Mitigate adversarial attacks by adding a layer for input sanitization and filtering, monitoring and logging user queries to detect unusual patterns. • Work with providers to ensure they do not retain or misuse sensitive input data. • Guide users to avoid sharing unnecessary personal data through clear instructions, training and warnings. • Educate employees and end users on proper usage, including the appropriate use of outputs and phishing techniques that could trick individuals into revealing sensitive information. • Ensure employees and end users avoid overreliance on LLMs for critical or high-stakes decisions without verification, and ensure outputs are reviewed by humans before implementation or dissemination. • Securely store outputs and restrict access to authorised personnel and systems.   This is a rare example where the EDPB strikes a good balance between practical safeguards and legal expectations. Link to the report included in the comments.   #AIprivacy #LLMs #dataprotection #AIgovernance #EDPB #privacybydesign #GDPR

If you're following AI, data licensing, and the closing off of the web to web crawlers/scrapers, this is a fresh study with hard data on the topic. Abstract: "General-purpose artificial intelligence (AI) systems are built on massive swathes of public web data, assembled into corpora such as C4, RefinedWeb, and Dolma. To our knowledge, we conduct the first, large-scale, longitudinal audit of the consent protocols for the web domains underlying AI training corpora. Our audit of 14,000 web domains provides an expansive view of crawlable web data and how codified data use preferences are changing over time. We observe a proliferation of AI-specific clauses to limit use, acute differences in restrictions on AI developers, as well as general inconsistencies between websites' expressed intentions in their Terms of Service and their robots.txt. We diagnose these as symptoms of ineffective web protocols, not designed to cope with the widespread re-purposing of the internet for AI. Our longitudinal analyses show that in a single year (2023-2024) there has been a rapid crescendo of data restrictions from web sources, rendering ~5%+ of all tokens in C4, or 28%+ of the most actively maintained, critical sources in C4, fully restricted from use. For Terms of Service crawling restrictions, a full 45% of C4 is now restricted. If respected or enforced, these restrictions are rapidly biasing the diversity, freshness, and scaling laws for general-purpose AI systems. We hope to illustrate the emerging crises in data consent, for both developers and creators. The foreclosure of much of the open web will impact not only commercial AI, but also non-commercial AI and academic research." https://lnkd.in/ebTfnCgZ?

As AI agent marketplaces emerge, a fundamental issue remains unresolved: who controls the knowledge they generate and benefits from the value they create? The prevailing approach treats data ownership as something that begins at the moment of digital capture, ignoring the reality that true ownership starts with human intent. Intent drives action, and action produces data—making sovereignty over that data the cornerstone of any AI-driven economy. Without a framework to uphold first-party ownership, AI agents risk becoming tools of extraction rather than enablers of autonomy. This article explores how embedding representation, governance, and enforceable rights at the data level creates an AI ecosystem where individuals and organizations retain control over their digital assets, ensuring that AI serves as an extension of human agency rather than a mechanism for centralized exploitation.

AI tools are improving rapidly—but so are the privacy risks. If you're using AI in your workflow, here are two things you should be doing to protect your data: 1) Read the terms of service. Many tools say “you own your content,” but also claim the right to use, remix, or even resell it. Licensing language matters. 2) Update your settings. Even after accepting terms, you often have to manually opt out of letting your data train their models. Tools like ChatGPT and Claude include this as a default unless you turn it off. These steps take five minutes—but they can save you from unintentionally sharing proprietary information or client data. In a world where AI is embedded in everything from email to creative software, data awareness is a competitive edge.