Minor Project ReportCyber security Effects on AI: Challenges and Mitigation Strategies
- 1. A Project on “Cyber security Effects on AI: Challenges and Mitigation Strategies” Submitted by Srinjoy Paul (221001102046) Ayush Kumar Madeshiya (221001102166) Shaury (221001102487) Aakash Singh (221001102203) Aditya Chandra (221001102045) Rimsha Naz (221001102496) Techno India University Kolkata, West Bengal
- 2. Student Name: Srinjoy Paul Roll Number: 221001102046 Registration No: 1002211349 Year: 2022-2023 Department: BCA (H) Session: 2022-2025
- 3. Acknowledgement We express our heartfelt gratitude to our Project Guide, Prof. Rohan Mallick, for his invaluable guidance and support throughout this project. We also thank the Head of the Department, Dr. Anil Bikash Chowdhury, and our Invigilator for their encouragement and assistance. This project would not have been possible without the collaborative efforts of our peers and the resources provided by our institution.
- 4. Project Abstraction Artificial Intelligence (AI) has transformed numerous sectors, including cyber security. While AI enhances threat detection and response capabilities, it also introduces unique challenges such as adversarial attacks, data poisoning, and algorithmic vulnerabilities. This project explores the impact of cyber security threats on AI systems, emphasizing strategies to mitigate these risks and enhance AI resilience.
- 5. Project Report: Cyber Security Effects on AI Introduction Artificial Intelligence (AI) has revolutionized various industries, enhancing efficiency, automation, and decision-making processes. However, its integration into cyber security comes with significant challenges. AI systems are vulnerable to adversarial attacks, data poisoning, and security breaches that can compromise their integrity and reliability. This report explores the impact of cyber security threats on AI and discusses mitigation strategies to enhance AI resilience. The integration of AI in cyber security has revolutionized the ability to detect, predict, and respond to threats. However, as AI becomes a crucial asset, it also becomes a target for
- 6. sophisticated cyber attacks. This project investigates the dual-edged nature of AI in cyber security and evaluates the implications of various attack vectors on AI systems. Objectives The primary objectives of this project are: 1.To identify common cyber security threats targeting AI systems. 2.To analyze the impact of such threats on AI performance and reliability. 3.To propose mitigation techniques and best practices for securing AI systems. Theoretical Framework Cyber Security Threats in AI
- 7. 1.Adversarial Attacks: Small perturbations in input data that mislead AI models. 2.Data Poisoning: Introducing malicious data during training to corrupt the model. 3.Model Inversion Attacks: Extracting sensitive training data from AI models. Security Challenges Lack of standardized frameworks for AI security. Trade-offs between model complexity and robustness. Mitigation Strategies 1.Robust Training Techniques: Using adversarial training to enhance model resilience. 2.Secure Multi-Party Computation & Federated Learning: Enhancing data privacy and model security.
- 8. 3.Regular Audits and Penetration Testing: Ensuring continuous security evaluation of AI systems. Methodology The research follows a multi-step approach: 1.Literature Review: Examination of existing studies on cyber security risks in AI. 2.Evaluation of Current Mitigation Strategies: Assessing existing security measures and proposing improvements. 3.Case Studies & Real-world Examples: Analyzing past cyber security incidents involving AI systems. 4.Implementation of Mitigation Techniques: Experimenting with different approaches to enhance AI security. Case Studies and Real-World Examples Adversarial Attacks on Image Recognition AI: AI poses a new dimension of security threats to computer science as it changes how generative AI models are developed. An adversarial attack manipulates the input data
- 9. with perturbations for the model to predict or generate false outputs inaccurately. Have you ever wondered how hackers can trick AI systems into making mistakes? That’s where adversarial attacks come in. These sneaky attacks manipulate AI models to make incorrect predictions or decisions. According to the research, malicious attacks have been proven to reduce the performance of generative AI models by up to 80%. Understanding attacks on generative AI is necessary to ensure security and reliability. It was demonstrated that even slight perturbations in the input data heavily affect the performance of generative AI models. Adversarial attacks compromise numerous real-world applications, including self-driving cars, facial recognition systems, and medical image analysis. Types of Adversarial Attacks Targeted Attacks In targeted attacks, the attacker attempts to manipulate the model into classifying a particular instance incorrectly. This can often be done by adding perturbations to the input that are humanly unnoticeable yet have a significantly profound impact on the model’s decision-making process. Research has
- 10. illustrated that targeted attacks are very successful, with success rates in the range of 70% to 90% or higher, depending on the model and type of attack. Targeted attacks have been exploited in various real applications, including applications in image classification, malware detection, and self-driving cars. Non-Targeted Attacks In non-targeted attacks, the attacker aims to degrade the model’s general performance by falsely classifying multiple inputs. This may be achieved by adding random noise or other perturbations to the input. Non-targeted attacks could drastically degrade the accuracy and reliability of machine learning models. White-Box Attacks White-box attacks are a category in which an attacker is assumed to know the model’s architecture, parameters, and training data. This allows for a significantly more effective attack that exploits the model’s weakness. White-box attacks are more successful than black-box attacks because the attacker knows about the model. It is harder to defend against white-box attacks than black-box attacks since attackers can target vulnerable points of the model. Black-Box Attacks
- 11. In black-box attacks, the attacker can access only the model’s input and output. Hence, they cannot obtain any insights into what is happening inside the model, making it harder to craft an effective attack. Black-box attacks can be successful in different contexts. Combining them with advanced techniques such as gradient- based optimization and transferability can be powerful. Black- box attacks are relevant, especially in real-world applications, where attackers might not know the targeted model. Real World Examples Case Study 1: The Panda Attack on Image Net (Good fellow et al., 2015) The most famous example of such an attack is the work of Good fellow et al., where an arbitrary noise was added to an image of a panda that, before its addition, an existing model correctly classified but afterward, misled the model into categorizing it as a “gibbon.” This type of attack, called a Fast Gradient Sign Method (FGSM), proved that neural networks are vulnerable to adversarial examples. – Key Takeaways Small changes in input data can entirely deceive AI models. The attack revealed the vulnerability of deep neural networks and initiated research in robust defense.
- 12. Case Study 2: Adversarial Attacks on Tesla’s Self-Driving Cars In 2020, researchers from McAfee conducted an in-the-wild adversarial attack on self-driving Tesla cars. Tiny stickers pasted onto road signs were enough to make the AI system read an “85” speed limit sign because it saw a 35-mph speed limit sign. The distortion was so slight that a human barely noticed it, but the AI system was highly affected. – Key Insights: In other words, even advanced generative AI models, like those in autonomous vehicles, can be easily fooled by minor environmental modifications. In a real-world setting, physical adversarial attacks are one of the biggest threats to AI systems; the case shows this possibility. Case Study 3: Adversarial Attacks on Google Cloud Vision API (2019) Researchers from Tencent’s Keen Security Lab were able to attack Google Cloud Vision API – a widely used AI image recognition service – with a successful adversarial attack; in other words, they could cheat such AI by slightly manipulating input images and getting false labels. For example, by almost imperceptibly corrupting a picture of a cat, they made the API return it as guacamole.
- 13. – Key Take-Away: Those cloud-based APIs represent public AI services that are not immune to adversarial attacks. The attacks have targeted weaknesses in the models and cloud-based generative AI services that many other industries rely on. Data Poisoning in Financial AI Models: While specific real-world cases of data poisoning in financial AI models are often not publicly disclosed due to sensitive nature, potential scenarios and hypothetical examples include: manipulating loan approval algorithms to unfairly favor certain demographics, injecting false transaction data to disrupt fraud detection systems, or altering market data to influence investment recommendations; all of which could significantly impact financial decisions made by AI models based on poisoned training data. Key examples of how data poisoning could be used in financial AI: Biased Loan Approvals: An attacker could inject data into a loan approval model that disproportionately favors certain demographics like high- income individuals, leading to biased lending practices where
- 14. qualified applicants from lower income groups are unfairly denied loans. Fraud Detection Evasion: Malicious actors could intentionally insert false positive transactions into a fraud detection system to train the model to misclassify fraudulent activity as legitimate, allowing them to commit fraud without detection. Market Manipulation: By feeding a stock prediction model with artificially inflated or deflated market data, attackers could manipulate the model's predictions to influence market sentiment and drive stock prices in a desired direction. Algorithmic Trading Bias: If an algorithmic trading model is trained on data that is skewed towards a specific asset class or market condition, it could make biased trading decisions, potentially leading to significant financial losses. Potential real-world situations where data poisoning might occur: Insider Threat:
- 15. A disgruntled employee with access to financial data could intentionally inject false information into the training dataset to harm the company's financial models. Competitor Sabotage: A rival company might attempt to undermine a competitor's AI-powered financial platform by feeding it manipulated data to generate inaccurate predictions. Cybercriminal Attacks: Hackers could exploit vulnerabilities in a financial institution's data collection process to introduce poisoned data into the training set of their AI systems. Important considerations: Detection Challenges: Data poisoning attacks can be difficult to detect as they often appear subtle and blend seamlessly with legitimate data. Data Validation and Monitoring: Financial institutions need to implement robust data validation procedures and continuously monitor their AI models for signs of unexpected behavior that could indicate data poisoning. Regulatory Implications:
- 16. As AI becomes more prevalent in finance, regulatory bodies might develop guidelines to address the risks associated with data poisoning and ensure responsible AI development. Model Inversion in Healthcare AI: Cases where attackers extracted patient data from AI-driven medical models. AI Health Care Case Studies 1. University of Rochester Medical Center 2. Valley Medical Center University of Rochester Medical Center The University of Rochester Medical Center (URMC), the home to Strong Memorial Hospital and centerpiece of the university’s patient care and medical research, worked with Butterfly Network to improve patient access to imaging, according to a case study. After an initial assessment of enterprise-wide need and identifying medical groups to be the first adopters, URMC decided to provide incoming medical students in certain
- 17. disciplines with a personal Butterfly IQ probe, distributing 862 of the devices. The probes use AI and advanced imaging, sharp image quality, rapid data processing and user-centric ergonomics to improve the accuracy and speed of diagnoses of cholecystitis and bladder and other medical issues. URMC plans to have three times the number of Butterfly IQ probes in use by the end of 2026. Results 116% increase in ultrasound charge capture across health system 74% increase in scanning sessions 3 x increases in ultrasounds sent to electronic health record (EHR) system Valley Medical Center Xsolis’ Dragonfly Utilize platform, which provides AI-driven medical necessity scores, enabled Valley Medical Center to significantly increase its observation (OBS) rates to keep them more within Centers for Medicare and Medicaid Services
- 18. (CMS) and other local facilities’ averages, according to a case study. Within a month’s time of implementation, Valley Medical’s team became proficient at knowing when to review versus when to escalate to physician advisory services. This enabled Valley Medical to more easily keep an eye on their most important cases and make appropriate care and patient status decisions quickly as the clinical indications changed. The health care facility was also able to reallocate staff for more efficiency and job satisfaction. For example, one lead utilization management (UM) specialist was able to purely focus on appeals and denials resolution management. Results Increased case reviews from 60% to 100% Increased observation rate of discharged patients from 4% to 13% Improved extended observation rates by 25%
- 19. Findings and Analysis AI systems are increasingly targeted due to their critical role in security infrastructure. Adversarial attacks can significantly compromise AI performance, necessitating robust security measures. Existing security frameworks lack uniformity, requiring industry-wide standardization efforts. Regular updates and AI model monitoring are essential to mitigate emerging threats. Future Scope Development of AI-specific cyber security frameworks. Integration of AI with blockchain technology for enhanced security. Advancements in explainable AI to detect and prevent cyber attacks more effectively. Strengthening regulatory policies on AI security.
- 20. Conclusion Cyber security is both a challenge and an enabler for AI systems. Addressing vulnerabilities in AI requires a proactive approach, combining technological advancements with robust policies. By understanding and mitigating these risks, we can ensure the safe and effective deployment of AI in critical applications. This project highlights the importance of cyber security in safeguarding AI systems and provides insights into future research directions in this domain.