SlideShare a Scribd company logo
Building Security at Scale
P R E S E N T E D B Y A l e x S t a m o s B l a c k H a t U S A 2 0 1 4 | A u g u s t 7 , 2 0 1 4
Agenda
▪ The Security Industry and Web Scale Problems
!
▪ Combating Security Nihilism
!
▪ What is Yahoo doing about it?
2
Theses
3
▪ The security industry has failed to consider the needs of
scale, including diversity of user base
!
▪ A post-Snowden nihilism is affecting our industry’s
approach to securing users
!
▪ Enterprise security teams need to evolve to proactively
gain trust
The Security Industry and Web
Scale Problems
5
What do I mean by scale?
Amount of
!
› Data
› Systems
› Users
Diversity of
!
› Users
› Threat Models
Who is the prototypical customer of security products?
6
7
Big Banks Web Scale
Customers x 10 x 10
Concurrent Users x 10 x 10
Front-End Servers x 10 x 10
Total Servers x 10 x 10
Customer Value $100’s $.01s
Cust Stickiness High Low-Medium
Meat-Space Identity Strong Weak
Post-Facto Action? Yes Rarely
8
Most security companies
are aiming for this
Our reality is more like this
[1]
[1] Flickr user Kevin Gebhart CC BY-NC-SA 2.0
[2] Flickr user Dan Buczynski CC BY-NC-ND 2.0
[2]
Things People Try to Sell Us
What they try to sell us:
10
Super smart pizza boxes
Software sensors with
centralized intelligence
What we would buy:
[1]
[1] Flickr user ms.akr CC BY 2.0
[2] Flickr user Mike Fleming CC BY SA 2.0
[2]
11
Arista 7508E
1152 x 10GbE
30Tbps backplane
5kW
Palo Alto 7050
120Gbps throughput
2.4kW
12
5kW
600kW
What they try to sell us:
Databased back SIEM
Hadoop based
anomaly detection
What we would buy:
[1]
[1] Flickr user Bob Mical CC BY 2.0
What they try to sell us:
14
Windows Anti-APT
Virtualization or
Kernel Firewall
Docker HIDS
What we would buy:
[1]
[1] Flickr user broterham CC BY NC 2.0
!
Free Business Ideas
15
▪ Freemium Key Management System
› Bootstrap via manual approval, trust in network, or remote attestation
› Create master cert, view into corporate keyspace with lazy security checks
!
▪ Freemium Overlay Network
› Goes great with key manager!
› Allow for easy IP management across public/private cloud
› Could be IPv6 only. Terminate inside of containers?
!
▪ Bug Bounty with Automatic Verification
› We’re building this ourselves with Selenium
More Free Ideas
16
▪ ARM CoreOS Servers with Lightweight Remote Attestation
› ARM is going to be big in Big Data environments
› At scale building systems remotely is currently terrifying
› Any scale organization does not have 100% physical control
!
▪ OpenSSL with Remotable Handshake
› Why are we putting private keys on the most exposed systems?
› Need to remote the handshake to an HSM or TXT backed key server
› Should get 20:1 ratio
Breaking through the excuses
Security Nihilism
18
Ve believe zat nothing…
ist secure enough vor ze real world, Lebowski!
Flickr user Joe Goldberg CC BY-SA 2.0
[1]
[1]
19
“Your system is not secure against this [advanced attack|unlikely
scenario] therefore it shouldn’t exist”
“That’s just security through obscurity!”
We need to build systems for all levels of user and adversary
Non-obvious protections can increase the chance of catching
an attacker in time, especially for interactive systems
20
“The [NSA|FSB|PLA] will just own up the user’s system and get the
data that way”
“Users are idiots and will screw this up”
Forcing an adversary to expend resources and
risk detection is a valid goal
A system is only secure if it is safe, by default,
for the 25th percentile user
What are we doing about it?
The New Yahoo Paranoids
22
CISO
Corp Security
Operations and
Monitoring
Product Security
Engineering
Paranoid Labs PenetrationTesting
Intelligence,
Investigations and
Response
Risk and
Vulnerability
Management
Privacy Engineering
Chief of Staff
New Yahoo Paranoids
23
Chris Rohlf Doug DePerry Yan Zhu
24
Transport Encryption
Complete
!
› TLS 1.2
› ECDH(E)
› AES-GCM
› RSA 2048
Next up
!
› HSTS and pre-load
› ECDSA certificates
› Certificate Transparency
› ChaCha20 and Poly1305
› STARTTLS Pinning
Backbone Encryption
25
Self-Service Security
26
▪ Our scaling challenges in providing app sec services:
› Breadth: 80+ products in 60+ countries
› Speed: multiple daily web pushes and weekly mobile
!
▪ Any large org needs to create self-service options
› Mobile libraries
• Authentication and device identity
• TLS with pinning
› Mobile code scanning portal
› CI/CD Scanner integration
• Open-source coming!
Bug Bounty
27
Bug Bankruptcy
28
▪ Important factors in getting bugs closed:
› Detailed descriptions and mitigation instructions
› Accurate prioritization
› Consistent follow-up and real-time reporting
› Executive visibility
› Convincing company that you are a madman
Works well for me
29
The Future is Bright
30
▪ Our profession has never been so impactful on…
› Individuals
› Nation-States
› History
!
▪ With great power…
› It is impossible to work in this field without being a moral actor
!
▪ Remember that trust is more than security!
!
▪ Take this opportunity to do something that you will remember with
pride
Thank you
!
stamos@yahoo-inc.com

More Related Content

PDF
AppSec is Eating Security
PDF
Shifting Security Left - The Innovation of DevSecOps - ValleyTechCon
PDF
Application Security Testing for a DevOps Mindset
PDF
DevSecOps and the New Path Forward
PDF
Pragmatic Pipeline Security
PPTX
451 AppSense Webinar - Why blame the user?
PDF
Shifting Security Left - The Innovation of DevSecOps - AgileDC
PDF
Collaborative security : Securing open source software
AppSec is Eating Security
Shifting Security Left - The Innovation of DevSecOps - ValleyTechCon
Application Security Testing for a DevOps Mindset
DevSecOps and the New Path Forward
Pragmatic Pipeline Security
451 AppSense Webinar - Why blame the user?
Shifting Security Left - The Innovation of DevSecOps - AgileDC
Collaborative security : Securing open source software

What's hot (20)

PPTX
The Journey to DevSecOps
PDF
DevSecOps at Agile 2019
PDF
Securing 100 products - How hard can it be?
PDF
Release Your Inner DevSecOp
PDF
Serverless Security: A How-to Guide @ SnowFROC 2019
PDF
DevSecOps - Building continuous security into it and app infrastructures
ODP
Building an Open Source AppSec Pipeline
PPTX
Open Source Defense for Edge 2017
PPTX
Elizabeth Lawler - Devops, security, and compliance working in unison
PPTX
Failure is inevitable but it isn't permanent
PPTX
Security and DevOps Overview
PDF
DevSecCon Asia 2017 Ofer Maor: AppSec DevOps automation – real world cases
PDF
Faster Secure Software Development with Continuous Deployment - PH Days 2013
PDF
The Emergent Cloud Security Toolchain for CI/CD
PDF
Attacking Pipelines--Security meets Continuous Delivery
PDF
AppSec California 2018: The Path of DevOps Enlightenment for InfoSec
PDF
The Seven Habits of the Highly Effective DevSecOp
PDF
How to transform developers into security people
PDF
CSA Raleigh application security and deception in the cloud
PPTX
Defining DevSecOps
The Journey to DevSecOps
DevSecOps at Agile 2019
Securing 100 products - How hard can it be?
Release Your Inner DevSecOp
Serverless Security: A How-to Guide @ SnowFROC 2019
DevSecOps - Building continuous security into it and app infrastructures
Building an Open Source AppSec Pipeline
Open Source Defense for Edge 2017
Elizabeth Lawler - Devops, security, and compliance working in unison
Failure is inevitable but it isn't permanent
Security and DevOps Overview
DevSecCon Asia 2017 Ofer Maor: AppSec DevOps automation – real world cases
Faster Secure Software Development with Continuous Deployment - PH Days 2013
The Emergent Cloud Security Toolchain for CI/CD
Attacking Pipelines--Security meets Continuous Delivery
AppSec California 2018: The Path of DevOps Enlightenment for InfoSec
The Seven Habits of the Highly Effective DevSecOp
How to transform developers into security people
CSA Raleigh application security and deception in the cloud
Defining DevSecOps
Ad

Similar to Security at Scale - Lessons from Six Months at Yahoo (20)

PPTX
Reduciendo riesgos a través de controles de acceso, manejo de privilegios y a...
PDF
Cloud Security and OSI layer 2 - the layer oft forgotten
PPTX
Securing Systems - Still Crazy After All These Years
PDF
Securing the Internet of Things
PDF
The cyber security hype cycle is upon us
PDF
GISEC 2015 Your Network in the Eyes of a Hacker - DTS Solution
PDF
What CISOs should know about SAP security
PDF
Cybersecurity
PPTX
Top Application Security Trends of 2012
PPTX
Protect your Database with Data Masking & Enforced Version Control
PPTX
Disruptionware-TRustedCISO103020v0.7.pptx
PDF
NVIS-Pitch Deck version 6 - 2022 MAR.pdf
PDF
Crush Common Cybersecurity Threats with Privilege Access Management
PPTX
The evolution of IT in a cloud world
PPTX
5 Ways To Fight A DDoS Attack
PDF
Security from the Start: Optimizing Your Acquia Experience with Acquia Cloud...
PPTX
Web 3.0 – From Buzzword to Security with Schellman
PPTX
4 Cyber Security KPIs
PPTX
GDPR Part 5: Better Together Quest & Cyberquest
PPTX
Security: more important than ever - Sophos Day Belux 2014
Reduciendo riesgos a través de controles de acceso, manejo de privilegios y a...
Cloud Security and OSI layer 2 - the layer oft forgotten
Securing Systems - Still Crazy After All These Years
Securing the Internet of Things
The cyber security hype cycle is upon us
GISEC 2015 Your Network in the Eyes of a Hacker - DTS Solution
What CISOs should know about SAP security
Cybersecurity
Top Application Security Trends of 2012
Protect your Database with Data Masking & Enforced Version Control
Disruptionware-TRustedCISO103020v0.7.pptx
NVIS-Pitch Deck version 6 - 2022 MAR.pdf
Crush Common Cybersecurity Threats with Privilege Access Management
The evolution of IT in a cloud world
5 Ways To Fight A DDoS Attack
Security from the Start: Optimizing Your Acquia Experience with Acquia Cloud...
Web 3.0 – From Buzzword to Security with Schellman
4 Cyber Security KPIs
GDPR Part 5: Better Together Quest & Cyberquest
Security: more important than ever - Sophos Day Belux 2014
Ad

Recently uploaded (20)

PDF
The New Creative Director: How AI Tools for Social Media Content Creation Are...
PDF
Testing WebRTC applications at scale.pdf
DOCX
Unit-3 cyber security network security of internet system
PPT
Design_with_Watersergyerge45hrbgre4top (1).ppt
PPTX
artificial intelligence overview of it and more
PDF
Cloud-Scale Log Monitoring _ Datadog.pdf
PPTX
Introuction about WHO-FIC in ICD-10.pptx
PDF
Triggering QUIC, presented by Geoff Huston at IETF 123
PDF
Sims 4 Historia para lo sims 4 para jugar
PDF
Decoding a Decade: 10 Years of Applied CTI Discipline
PDF
SASE Traffic Flow - ZTNA Connector-1.pdf
PDF
An introduction to the IFRS (ISSB) Stndards.pdf
PDF
RPKI Status Update, presented by Makito Lay at IDNOG 10
PPTX
522797556-Unit-2-Temperature-measurement-1-1.pptx
PPTX
Introduction to Information and Communication Technology
PDF
Slides PDF The World Game (s) Eco Economic Epochs.pdf
PDF
WebRTC in SignalWire - troubleshooting media negotiation
PDF
💰 𝐔𝐊𝐓𝐈 𝐊𝐄𝐌𝐄𝐍𝐀𝐍𝐆𝐀𝐍 𝐊𝐈𝐏𝐄𝐑𝟒𝐃 𝐇𝐀𝐑𝐈 𝐈𝐍𝐈 𝟐𝟎𝟐𝟓 💰
PPTX
presentation_pfe-universite-molay-seltan.pptx
PDF
Vigrab.top – Online Tool for Downloading and Converting Social Media Videos a...
The New Creative Director: How AI Tools for Social Media Content Creation Are...
Testing WebRTC applications at scale.pdf
Unit-3 cyber security network security of internet system
Design_with_Watersergyerge45hrbgre4top (1).ppt
artificial intelligence overview of it and more
Cloud-Scale Log Monitoring _ Datadog.pdf
Introuction about WHO-FIC in ICD-10.pptx
Triggering QUIC, presented by Geoff Huston at IETF 123
Sims 4 Historia para lo sims 4 para jugar
Decoding a Decade: 10 Years of Applied CTI Discipline
SASE Traffic Flow - ZTNA Connector-1.pdf
An introduction to the IFRS (ISSB) Stndards.pdf
RPKI Status Update, presented by Makito Lay at IDNOG 10
522797556-Unit-2-Temperature-measurement-1-1.pptx
Introduction to Information and Communication Technology
Slides PDF The World Game (s) Eco Economic Epochs.pdf
WebRTC in SignalWire - troubleshooting media negotiation
💰 𝐔𝐊𝐓𝐈 𝐊𝐄𝐌𝐄𝐍𝐀𝐍𝐆𝐀𝐍 𝐊𝐈𝐏𝐄𝐑𝟒𝐃 𝐇𝐀𝐑𝐈 𝐈𝐍𝐈 𝟐𝟎𝟐𝟓 💰
presentation_pfe-universite-molay-seltan.pptx
Vigrab.top – Online Tool for Downloading and Converting Social Media Videos a...

Security at Scale - Lessons from Six Months at Yahoo

  • 1. Building Security at Scale P R E S E N T E D B Y A l e x S t a m o s B l a c k H a t U S A 2 0 1 4 | A u g u s t 7 , 2 0 1 4
  • 2. Agenda ▪ The Security Industry and Web Scale Problems ! ▪ Combating Security Nihilism ! ▪ What is Yahoo doing about it? 2
  • 3. Theses 3 ▪ The security industry has failed to consider the needs of scale, including diversity of user base ! ▪ A post-Snowden nihilism is affecting our industry’s approach to securing users ! ▪ Enterprise security teams need to evolve to proactively gain trust
  • 4. The Security Industry and Web Scale Problems
  • 5. 5 What do I mean by scale? Amount of ! › Data › Systems › Users Diversity of ! › Users › Threat Models
  • 6. Who is the prototypical customer of security products? 6
  • 7. 7 Big Banks Web Scale Customers x 10 x 10 Concurrent Users x 10 x 10 Front-End Servers x 10 x 10 Total Servers x 10 x 10 Customer Value $100’s $.01s Cust Stickiness High Low-Medium Meat-Space Identity Strong Weak Post-Facto Action? Yes Rarely
  • 8. 8 Most security companies are aiming for this Our reality is more like this [1] [1] Flickr user Kevin Gebhart CC BY-NC-SA 2.0 [2] Flickr user Dan Buczynski CC BY-NC-ND 2.0 [2]
  • 9. Things People Try to Sell Us
  • 10. What they try to sell us: 10 Super smart pizza boxes Software sensors with centralized intelligence What we would buy: [1] [1] Flickr user ms.akr CC BY 2.0 [2] Flickr user Mike Fleming CC BY SA 2.0 [2]
  • 11. 11 Arista 7508E 1152 x 10GbE 30Tbps backplane 5kW Palo Alto 7050 120Gbps throughput 2.4kW
  • 13. What they try to sell us: Databased back SIEM Hadoop based anomaly detection What we would buy: [1] [1] Flickr user Bob Mical CC BY 2.0
  • 14. What they try to sell us: 14 Windows Anti-APT Virtualization or Kernel Firewall Docker HIDS What we would buy: [1] [1] Flickr user broterham CC BY NC 2.0 !
  • 15. Free Business Ideas 15 ▪ Freemium Key Management System › Bootstrap via manual approval, trust in network, or remote attestation › Create master cert, view into corporate keyspace with lazy security checks ! ▪ Freemium Overlay Network › Goes great with key manager! › Allow for easy IP management across public/private cloud › Could be IPv6 only. Terminate inside of containers? ! ▪ Bug Bounty with Automatic Verification › We’re building this ourselves with Selenium
  • 16. More Free Ideas 16 ▪ ARM CoreOS Servers with Lightweight Remote Attestation › ARM is going to be big in Big Data environments › At scale building systems remotely is currently terrifying › Any scale organization does not have 100% physical control ! ▪ OpenSSL with Remotable Handshake › Why are we putting private keys on the most exposed systems? › Need to remote the handshake to an HSM or TXT backed key server › Should get 20:1 ratio
  • 18. Security Nihilism 18 Ve believe zat nothing… ist secure enough vor ze real world, Lebowski! Flickr user Joe Goldberg CC BY-SA 2.0 [1] [1]
  • 19. 19 “Your system is not secure against this [advanced attack|unlikely scenario] therefore it shouldn’t exist” “That’s just security through obscurity!” We need to build systems for all levels of user and adversary Non-obvious protections can increase the chance of catching an attacker in time, especially for interactive systems
  • 20. 20 “The [NSA|FSB|PLA] will just own up the user’s system and get the data that way” “Users are idiots and will screw this up” Forcing an adversary to expend resources and risk detection is a valid goal A system is only secure if it is safe, by default, for the 25th percentile user
  • 21. What are we doing about it?
  • 22. The New Yahoo Paranoids 22 CISO Corp Security Operations and Monitoring Product Security Engineering Paranoid Labs PenetrationTesting Intelligence, Investigations and Response Risk and Vulnerability Management Privacy Engineering Chief of Staff
  • 23. New Yahoo Paranoids 23 Chris Rohlf Doug DePerry Yan Zhu
  • 24. 24 Transport Encryption Complete ! › TLS 1.2 › ECDH(E) › AES-GCM › RSA 2048 Next up ! › HSTS and pre-load › ECDSA certificates › Certificate Transparency › ChaCha20 and Poly1305 › STARTTLS Pinning
  • 26. Self-Service Security 26 ▪ Our scaling challenges in providing app sec services: › Breadth: 80+ products in 60+ countries › Speed: multiple daily web pushes and weekly mobile ! ▪ Any large org needs to create self-service options › Mobile libraries • Authentication and device identity • TLS with pinning › Mobile code scanning portal › CI/CD Scanner integration • Open-source coming!
  • 28. Bug Bankruptcy 28 ▪ Important factors in getting bugs closed: › Detailed descriptions and mitigation instructions › Accurate prioritization › Consistent follow-up and real-time reporting › Executive visibility › Convincing company that you are a madman Works well for me
  • 29. 29
  • 30. The Future is Bright 30 ▪ Our profession has never been so impactful on… › Individuals › Nation-States › History ! ▪ With great power… › It is impossible to work in this field without being a moral actor ! ▪ Remember that trust is more than security! ! ▪ Take this opportunity to do something that you will remember with pride