The Evolving Landscape of AI and Cybersecurity

Artificial intelligence (AI) is transforming businesses and society in profound ways. As companies rush to adopt AI to gain competitive advantage, we must also consider the impacts on cybersecurity. In this post, I will summarize key insights from a conference on AI and cybersecurity, organized by CESIN during its Eleventh Congress in Reims, France (a must-attend).

The speaker, Francoise Soulie-Fogelman , has extensive experience in AI and leads an industry group exploring its cybersecurity implications. She provides a historical overview of AI waves since the 1980s and explains why generative AI marks a seismic shift today. Francoise Soulie-Fogelman then examines AI's expanding attack surface, new threats, and countermeasures organizations can take.

The AI Landscape: Waves and Generations

AI has experienced successive "winters" and hype cycles over the past few decades. Francoise Soulie-Fogelman outlines two main approaches:

Symbolic AI: Tried fully encoding human knowledge and reasoning in expert system rules. This failed to match human capabilities.

Data-driven AI: Focuses on using data to accomplish tasks without explicit programming. More successful recently.

Key waves:

• 1980s-90s: Neural networks and machine learning developed but not yet practical

• 2012: AlexNet convincingly beat other techniques on image recognition, catalyzing deep learning

• 2012-2022: Widespread deployment of deep learning for perception tasks like image, speech, and language

• Late 2022: Generative AI arrives with systems like ChatGPT that can create novel content

Generative AI represents a seismic shift, with its ability to produce high-fidelity synthetic text, images, audio, and video. Adoption has skyrocketed exponentially faster than any prior technology. While exciting, it also expands the attack surface for bad actors.

"So AI is not going to replace the cyber team. If you think you're going on vacation, you're wrong. So no, it's not over, it's not, it continues. I'm not saying it continues the same way. I'm saying it continues with a lot more." - Francoise Soulie-Fogelman

Expanding Attack Surface with AI Growth

As companies rush to deploy AI before fully securing it, risks grow. Reasons include:

• More data and modalities: AI systems ingest diverse, multimodal data types. More inputs mean more potential vulnerabilities.

• Sheer scale: AI adoption is surging, exposing far more systems. Each one is a potential target.

• Constant evolution: Attackers and systems change rapidly. AI must continually adapt through learning.

• Lack of expertise: Most developers using AI toolkits lack cybersecurity knowledge. They wrongly assume the tools are secure by default.

This situation requires urgent attention to secure AI proactively rather than reactively. Next, we will explore key threats and countermeasures.

Emerging Threats and Countermeasures

Attackers are actively probing for ways to exploit AI, while defenders are just waking up to the risks. Known threat categories include:

• Data poisoning: Manipulating training data to degrade AI system performance.

• Model poisoning: Altering model parameters and logic to sabotage functionality.

• Evasion: Tricking deployed models with manipulated inputs.

• Extraction: Stealing confidential data or intellectual property within models.

Attack techniques will become more sophisticated over time as bad actors upskill.

However, organizations can take steps to defend their AI:

1. Monitor the expanding threat landscape and update defenses continuously.

2. Follow secure development best practices for the AI lifecycle.

3. Limit data access to reduce the attack surface.

4. Use multiple diverse defense tools and techniques.

5. Validate inputs thoroughly before feeding them into models.

6. Detect anomalies indicating potential attacks.

7. Control model autonomy instead of fully automating.

8. Build human oversight into deployment workflows.

No single solution will fully protect against AI threats. Ongoing Vigilance coupled with layered security is essential.

Generative AI Multiplies Risks

Generative AI warrants special concern given its rapid emergence. While limited today, its trajectory suggests wide-ranging risks including:

• Synthetic media for convincing phishing and fraud

• Impersonation of individuals via generated imagery and audio

• Automated mass production of tailored social engineering attacks

• Sophisticated bot-driven influence operations

• Counterfeit content undermining trust in institutions

• Difficulty differentiating what is real versus AI-fabricated

These threats make comprehensive defense even more challenging. Promising tools like media forensics and watermarking are still developing.

Opportunities to Use AI for Enhanced Defense

While AI increases risks, it also provides opportunities to strengthen cybersecurity. Potential benefits include:

• Automating repetitive manual processes for greater speed and scale.

• Detecting known threats early based on pattern recognition.

• Analyzing enormous volumes of threat data is not easily manageable by humans.

• Adaptively learning to detect novel anomalies and zero-day exploits.

• Generating synthetic datasets to augment training and testing.

• Providing predictive risk scoring to focus analyst attention.

• Orchestrating automated response playbooks to counter detected threats.

• Recommending remedial actions similar to past incidents.

These capabilities significantly expand the cybersecurity toolkit. However, AI cannot fully replace human judgment and oversight.

Recommendations for Organizations

Here are recommended focus areas for organizations seeking to improve their AI security posture:

1. Assign clear leadership responsibility for AI cyber risks.

2. Conduct an audit identifying current AI assets, data, and risks.

3. Build awareness and skills through education and training.

4. Develop policies and procedures governing AI utilization.

5. Perform ongoing threat assessments and penetration testing.

6. Institute secure design reviews within development workflows.

7. Implement controls limiting data access to the least privilege needed.

8. Continuously monitor systems and review logs for anomalies.

9. Simulate realistic attacks to evaluate and improve defenses.

10. Maintain comprehensive fallback plans in case defenses fail.

The Bottom Line

AI delivers tremendous value but also expands the cybersecurity challenge. By following prudent security practices, organizations can confidently pursue AI adoption while managing risks. Those ignoring AI threats or taking an overconfident “wait and see” attitude court disaster.

What steps is your organization taking to address AI cyber risks? What challenges do you face? I welcome your perspectives and recommendations in the comments.