Implementing Data Subject Rights in AI/ML Models: Erasure and Rectification

The increasing use of artificial intelligence (AI) and machine learning (ML) models has led to growing concerns about data privacy and the protection of individual rights. Regulations like the General Data Protection Regulation (GDPR) grant individuals certain rights concerning their personal data, including the right to erasure (also known as the "right to be forgotten") and the right to rectification. These regulations aim to protect individuals from potential privacy catastrophes caused by the increasing use of AI, particularly in applications like large language models (LLMs) that continuously collect and process vast amounts of personal data (1). Implementing these rights in the context of AI/ML models presents unique challenges due to the complex ways these models learn and store information. In the United States, lawsuits have alleged violations of intellectual property, privacy, and property rights due to the use of massive datasets for training AI models, further highlighting the need for robust mechanisms to protect data subject rights (2). This article explores recent research on how to implement data subject rights, focusing on erasure and rectification, within AI/ML models.

Machine Unlearning: Enabling the Right to Erasure

Machine unlearning aims to remove the influence of specific data points from a trained model without requiring complete retraining. This is crucial for complying with data subject erasure requests, as retraining large models can be computationally expensive and time-consuming (3). Unlearning can be applied in two distinct ways: to remove the effects of targeted information from a model's parameters (e.g., removing personal data) or to prevent a model from generating targeted types of information in its outputs (5).

Current research categorizes unlearning methods into four main scenarios: centralized unlearning, distributed and irregular data unlearning, unlearning verification, and privacy and security issues in unlearning 6. Within centralized unlearning, the primary focus of this article, there are two main categories:

Exact and Approximate Unlearning Methods

Exact Unlearning

• Description: Guarantees complete removal of a data point's influence, replicating retraining without the erased data.

• Advantages: Strong guarantees, ensures complete data removal.

• Disadvantages: Can be computationally demanding, especially for large models.

Approximate Unlearning

• Description: Aims to efficiently remove data influence through limited parameter updates.

• Advantages: More efficient than exact unlearning.

• Disadvantages: May not be scalable.

Impact of Unlearning on Model Performance

While approximate unlearning methods offer computational efficiency, they often lack strong guarantees on unlearning effectiveness, leading to a performance gap compared to exact unlearning or retraining from scratch 7. Research suggests that model sparsity can significantly improve the performance of approximate unlearning methods, closing this gap while maintaining efficiency 7. Sparsity-aware unlearning techniques, which incorporate sparsity regularization during training, can further enhance unlearning efficacy 7. To evaluate unlearning performance, researchers often present the performance gap with "Retrain" (retraining the model without the erased data) in blue within results tables 9.

It's important to consider the potential side effects of unlearning on model performance, including impacts on model accuracy, generalization, and safety 8. Additionally, research has shown that machine unlearning can impact fairness, particularly under non-uniform data deletion, where certain types of data are more likely to be unlearned than others 4. This raises ethical considerations and highlights the need for unlearning methods that mitigate potential biases.

Challenges in Machine Unlearning

Several challenges arise in implementing machine unlearning:

• Stochasticity of Training: The inherent randomness in training AI/ML models can make it difficult to precisely quantify and remove the influence of specific data points 10. This randomness arises from factors like the initialization of model parameters, the order in which data is presented during training, and the use of stochastic optimization algorithms.

• Incremental Learning: Unlearning in models that are continuously updated with new data presents challenges in maintaining unlearned status and managing the accumulation of unlearning requests over time 11. As new data is incorporated, the model's parameters change, and the effects of previously unlearned data might resurface or become intertwined with new information.

• Privacy Concerns: Unlearning processes themselves might raise privacy risks, as attackers could potentially exploit unlearning mechanisms to infer information about the erased data 12. For example, by observing changes in the model's output after unlearning, an attacker might be able to deduce information about the removed data.

• Forgettability Sequence: Research indicates that different samples exhibit varying levels of difficulty in being forgotten, leading to the concept of a "forgettability sequence" 14. This suggests that the order in which data is unlearned can influence the overall effectiveness of the unlearning process.

• Hyperparameter Tuning: Approximate unlearning algorithms may fail to effectively unlearn data in situations where hyperparameter tuning methods, such as cross-validation, have been used to select models 15. This is because hyperparameter tuning can inadvertently leak information about the training data, making it harder to completely remove the influence of specific data points.

• Trusted Research Environments: Trusted Research Environments (TREs) provide secure environments for researchers to access and train on sensitive personal data 16. However, the disclosure of trained models from TREs raises concerns about potential data leakage, as models can inadvertently encode personal information. This highlights the need for careful disclosure control mechanisms within TREs to mitigate privacy risks.

The Right to Rectification in AI/ML Models

While the right to erasure focuses on removing data, the right to rectification addresses the need to correct or update inaccurate or outdated information. In the context of AI/ML models, this presents the challenge of modifying the model's internal representations to reflect the corrected data. Traditional methods like retraining the entire model with the updated data can be computationally expensive. Research is exploring more efficient techniques for implementing the right to rectification, such as:

• Incremental Updating: Developing methods to incrementally update the model's parameters with the corrected information without requiring full retraining. This could involve techniques like targeted fine-tuning or localized parameter adjustments.

• Data Augmentation: Augmenting the training data with synthetic samples that reflect the corrected information. This can help the model learn the updated patterns without directly accessing the original data.

• Model Editing: Developing techniques to directly edit the model's internal representations to reflect the corrected information. This could involve modifying specific neurons or connections within the model.

Limiting Personal Data Output from Generative AI Models

Generative AI models, capable of creating new content, raise concerns about the potential for unintended disclosure of personal data. Research explores various methods to limit such outputs:

Model Finetuning

Adjusting model parameters to reduce the likelihood of generating specific personal data 17. This can involve techniques like:

• Conditional Likelihood Optimization: Optimizing the model to maximize the likelihood of generating desired outputs while minimizing the likelihood of generating personal data 19.

• Data Augmentation: Training on random or similar unedited facts to encourage locality and prevent the model from overfitting to specific personal data 19.

• Fine-tuning after Prompting: Using prompts to guide the model's generation process and then fine-tuning the model to improve its ability to classify or generate desired outputs 20.

• Data Design for Fine-tuning: Designing the format of the fine-tuning data to improve the behavior of small language models, such as enhancing their reasoning and self-correction abilities 21.

Data Redaction

Removing or masking personal data from the training dataset before training the model 17. This can be challenging due to the difficulty of identifying and removing all instances of personal data, especially in large and complex datasets.

Output Modification

Implementing post-processing techniques to filter or modify the model's output to prevent the generation of personal data 22. This can involve techniques like:

• Differential Privacy: Adding noise to the model's output to make it harder to infer information about individual data points.

• Synthetic Data Generation: Generating synthetic data that mimics the statistical properties of the original data but does not contain any personal information.

However, it's important to acknowledge the limitations and potential biases in using generative AI for tasks like social media data analysis. The lack of transparency in some generative AI models and the potential for undetected algorithm biases can undermine the validity and replicability of findings 23.

Verification of Unlearning

Ensuring that unlearning requests have been effectively implemented is crucial for maintaining trust and transparency. Verification strategies allow data owners to confirm the removal of their data's influence from the model 24. However, research suggests that current verification methods are fragile and can be circumvented by malicious model providers 25. This highlights the need for more robust verification techniques to ensure the integrity of unlearning processes. Some potential approaches for improving verification include:

• Cryptographic Proofs: Using cryptographic techniques to generate verifiable proofs that the unlearning process has been correctly executed.

• Auditing Mechanisms: Developing independent auditing mechanisms to verify the unlearning process and ensure compliance with data protection regulations.

• Differential Privacy: Applying differential privacy techniques to the unlearning process to limit the information that can be inferred about the erased data.

What does this mean

Implementing data subject rights, particularly the right to erasure and rectification, in the context of AI/ML models is a complex and evolving field. Machine unlearning offers a promising approach to enabling data erasure, but challenges remain in terms of efficiency, accuracy, and privacy. The inherent stochasticity of training, the dynamic nature of incremental learning, and the potential for privacy risks require careful consideration and the development of robust unlearning techniques.

Limiting personal data output from generative AI models requires a combination of techniques, including model finetuning, data redaction, and output modification. However, striking a balance between leveraging the power of generative AI and protecting data subject rights is crucial. Ensuring the integrity of unlearning through robust verification methods is essential for building trust and ensuring compliance with data protection regulations. However, current verification methods face challenges due to their fragility and potential for circumvention, highlighting the need for ongoing research to develop more reliable techniques.

The interconnectedness of these challenges underscores the need for a holistic approach to implementing data subject rights in AI/ML models. Future research should focus on developing unlearning and rectification techniques that are not only effective and efficient but also address privacy concerns and ensure fairness. This requires interdisciplinary collaboration between computer scientists, legal experts, and ethicists to navigate the complex landscape of AI/ML development and data protection. Further investigation is needed to understand the long-term implications of unlearning and rectification on model performance, generalization, and societal impact. Ultimately, the goal is to create AI/ML systems that are both innovative and responsible, respecting individual rights while harnessing the transformative potential of these technologies.

Works cited

1. Machine Unlearning for Traditional Models and Large Language Models : A Short Survey - arXiv, accessed January 24, 2025, https://arxiv.org/html/2404.01206v1

2. Privacy of Personal Data in the Generative AI Data Lifecycle, accessed January 24, 2025, https://jipel.law.nyu.edu/privacy-of-personal-data-in-the-generative-ai-data-lifecycle/

3. Machine Unlearning: A Comprehensive Survey - arXiv, accessed January 24, 2025, https://arxiv.org/html/2405.07406v2

4. Unveiling Fairness Implications of Machine Unlearning Methods - arXiv, accessed January 24, 2025, https://arxiv.org/pdf/2302.03350

5. Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy, Research, and Practice - Google DeepMind, accessed January 24, 2025, https://deepmind.google/research/publications/101479/

6. Machine Unlearning: A Comprehensive Survey - arXiv, accessed January 24, 2025, https://arxiv.org/html/2405.07406v1

7. Model Sparsity Can Simplify Machine Unlearning - OpenReview, accessed January 24, 2025, https://openreview.net/pdf?id=0jZH883i34

8. On the Limitations and Prospects of Machine Unlearning for Generative AI - arXiv, accessed January 24, 2025, http://arxiv.org/pdf/2408.00376

9. Model Sparsity Can Simplify Machine Unlearning - OpenReview, accessed January 24, 2025, https://openreview.net/forum?id=0jZH883i34

10. Scalability Challenges in Privacy-Preserving Federated Learning | NIST, accessed January 24, 2025, https://www.nist.gov/blogs/cybersecurity-insights/scalability-challenges-privacy-preserving-federated-learning

11. arXiv:2307.02246v1 [cs.CV] 5 Jul 2023, accessed January 24, 2025, https://arxiv.org/pdf/2307.02246

12. Remember What You Want to Forget: Algorithms for Machine Unlearning - OpenReview, accessed January 24, 2025, https://openreview.net/pdf?id=pvCLqcsLJ1N

13. Data Protection Issues in Automated Decision-Making Systems Based on Machine Learning: Research Challenges - MDPI, accessed January 24, 2025, https://www.mdpi.com/2673-8732/4/1/5

14. Machine Unlearning in Forgettability Sequence - ResearchGate, accessed January 24, 2025, https://www.researchgate.net/publication/384770931_Machine_Unlearning_in_Forgettability_Sequence

15. Algorithms that Approximate Data Removal: New Results and Limitations - OpenReview, accessed January 24, 2025, https://openreview.net/pdf?id=G4VOQPYxBsI

16. Disclosure control of machine learning models from trusted research environments (TRE): New challenges and opportunities, accessed January 24, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC10130764/

17. Whispered Tuning: Data Privacy Preservation in Fine-Tuning LLMs through Differential Privacy - Scientific Research Publishing, accessed January 24, 2025, https://www.scirp.org/pdf/jsea_2024012215492080.pdf

18. Model Editing by Standard Fine-Tuning - arXiv, accessed January 24, 2025, https://arxiv.org/html/2402.11078v3

19. [2402.11078] Model Editing by Standard Fine-Tuning - arXiv, accessed January 24, 2025, https://arxiv.org/abs/2402.11078

20. Fine-tuning after Prompting: an Explainable Way for Classification - ResearchGate, accessed January 24, 2025, https://www.researchgate.net/publication/384204663_Fine-tuning_after_Prompting_an_Explainable_Way_for_Classification

21. Data Design For Fine-Tuning To Improve Small Language Model Behaviour, accessed January 24, 2025, https://cobusgreyling.medium.com/data-design-for-fine-tuning-to-improve-small-language-model-behaviour-8616cb1e78c0

22. Generative AI and Data Privacy: The Challenge of PII Use in Training Data Sets - Smarsh, accessed January 24, 2025, https://www.smarsh.com/blog/thought-leadership/generative-AI-and-data-privacy-the-challenge-of-PII-use-in-training-data-sets

23. Disclosure Standards for Social Media and Generative Artificial Intelligence Research: Toward Transparency and Replicability - PMC, accessed January 24, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC10795517/

24. Verification of Machine Unlearning is Fragile - arXiv, accessed January 24, 2025, https://arxiv.org/pdf/2408.00929

25. (PDF) Verification of Machine Unlearning is Fragile - ResearchGate, accessed January 24, 2025, https://www.researchgate.net/publication/382867578_Verification_of_Machine_Unlearning_is_Fragile

26. Verification of Machine Unlearning is Fragile (Conference Paper) | NSF PAGES, accessed January 24, 2025, https://par.nsf.gov/biblio/10538548-verification-machine-unlearning-fragile