SlideShare a Scribd company logo
Secure Keystone Deployment:
Lessons Learned and Best Practices
Priti Desai
Sr. Software Engineer
Secure Keystone Deployment 1
The Symantec Team
• Cloud Platform Engineering
– We are building a consolidated cloud platform that provides infrastructure and
platform services for next generation Symantec products and services
• Me
– In Security for over 6 years
– Symantec Insight - Reputation Based Security
– Symantec Data Analytics Platform
– OpenStack Engineer - Keystone
– OpenStack Security Group
– Cop Open Source
Secure Keystone Deployment
2
OpenStack Security Group
Secure Keystone Deployment
3
security
notes
Retrieved from http://www.openstack.orgRetrieved from http://docs.openstack.org
Secure Keystone Deployment
Why is Keystone security critical?
What is Keystone?
How is Authentication process implemented in
Keystone?
How is Authorization mechanism implemented in
OpenStack?
AuthN Overview
Secure Keystone Deployment
5
Cloud User
Cloud User
Identity
(SQL/LDAP)
Keystone
Token (SQL)
Identity
(SQL/LDAP)
Keystone
Token (SQL)
Request sent with
Username and Password
Verify username and
password (hash of
password)
Successful verification
Request metadata for user
tenant relationship
Assignment
(SQL)
Assignment
(SQL)
User tenant relationship
information
Request to generate new
token
Response with new token
Response with token
AuthZ Overview
Secure Keystone Deployment
6
Cloud User
Cloud User Keystone
OpenStack
Service
Keystone
OpenStack
Service
Request sent with session
token
Verify session token
Successful verification
Is this token correct?
Does it allow the service
usage?
Service executes
the request
Response with success
Secure Keystone Deployment
Why is Keystone security critical?
Does it store/transmit any sensitive information?
What kind of cloud asset does it store?
Is any type of attack possible on Keystone? Can it
bring down the entire cloud?
Keystone Security is Critical
Secure Keystone Deployment
8
• Gatekeeper
• Access to OpenStack Cloud
• Assets
• Users
• Passwords
• Tokens
• Roles
• Catalog
• Vulnerable to DoS
Retrieved from http://internet.phillipmartin.info
Retrieved from http://blogs.citypages.com
Retrieved from http://assets.nydailynews.com
What was our approach to identifying
key vulnerabilities?
Secure Keystone Deployment
9
Security Risks
Secure Keystone Deployment
10
• Global Security Office
 Threat Model
 Penetration Tests
 Traceability Matrix
Retrieved from http://www.technetics.com.au
Threat Model
Secure Keystone Deployment
11
Secure Keystone Deployment
12
Spoofing
Tampering
Repudiation
Information Disclosure
Denial of Service
Elevation of Privileges
Threat Model
What kind of security deficiencies did
we discover?
Secure Keystone Deployment
13
Secure Keystone Deployment
14
Attack: Keystone user credential theft
Attack: Insecure file permissions on
Keystone.conf
Keystone.conf
Attack: Access to cloud admin
privileges for almost free
Attack: Leaking sensitive data in log
messages
Attack: DoS – Authentication chaining
- Havana
Attack: Unauthorized access to MySQL
database
Many more …
Traceability Matrix
Secure Keystone Deployment
15
✖
✖
✖
Keystone User Credential Theft
Secure Keystone Deployment
16
Mitigate: Secure Communication - SSL
Secure Keystone Deployment
17
Hardware Load Balancer Hardware Load Balancer
Keystone KeystoneKeystone
SSL Client
SSL Server
SSL Client
SSL Server
mod_ssl
35357/SSL 5000/SSL
mod_ssl
35357/SSL 5000/SSL
mod_ssl
35357/SSL 5000/SSL
Public API Admin API
Insecure file permissions on Keystone.conf
Secure Keystone Deployment
18
Mitigate:
• Restrict ownership to service user
- chown keystone:keystone /etc/keystone/keystone.conf
• Restrict to read and write by the owner
- chmod 640 /etc/keystone/keystone.conf
hostnameabc
hostnameabc
hostnameabcuser
user
user
Access to admin privileges is almost free
Secure Keystone Deployment
19
• Service Token
• Bootstrap Keystone
• Cloud admin privileges
• Register bad service/endpoints
Mitigate: Disable Service Token
• Comment out admin_token from /etc/keystone/keystone.conf:
admin_token=e2112effd3ff05b8c88ad14e096e6615
• Remove admin token auth middleware from
/etc/keystone/keystone-paste.ini:
[filter:admin_token_auth]
paste.filter_factory =
keystone.middleware:AdminTokenAuthMiddleware.factory
Secure Keystone Deployment
20
Who is the cloud admin now?
Secure Keystone Deployment
21
Create Cloud Admin
• Leveraging Keystone Domain
• Before disabling service token:
• Create a domain “cloud_admin_domain”
• Grant “admin” role to appropriate user “Bob Smith”
• Update keystone policy.json file:
• Replace:
"cloud_admin": [["rule:admin_required", "domain_id:admin_domain_id"]],
• With:
"cloud_admin”: [["rule:admin_required”,"domain_id:<cloud_admin_domain_id>"]],
Secure Keystone Deployment
22
Leaking Sensitive Information in Log Messages
• Debug mode include plaintext request logging
• Passwords
• Tokens
• Mitigate:
• Disable debug mode in keystone.conf with:
• With debug mode ON, upgrade keystone client:
• python-keystoneclient >= 0.10.1 (OSSN-0024)
Secure Keystone Deployment
23
[DEFAULT]
debug=False
Leaking Sensitive Information in Log Messages
Identity API V2 - INFO level logs contains auth tokens (OSSN-0023)
Mitigate:
• Set the log level to WARN in logging.conf:
Secure Keystone Deployment
24
[handler_file]
class = FileHandler
Level = WARN
Keystone DoS Attack
Identity API V3 – Authentication Chaining – CVE-2014-2828
Secure Keystone Deployment
25
Keystone DoS Attack
Mitigate:
• Impacted Versions: from 2013.1 to 2013.2.3
• Patch applied during IceHouse rc2
• Upgrade Keystone >= 2013.2.4
Secure Keystone Deployment
26
Q&A
Let’s talk…
Secure Keystone Deployment 27
Thank You
Priti Desai
Priti_Desai@symantec.com
@pritidesai8
References
• http://docs.openstack.org/developer/keystone/
• https://blog-nkinder.rhcloud.com/?p=7
• https://blueprints.launchpad.net/keystone/+spec/service-scoped-
tokens
• http://docs.openstack.org/sec/
• http://www.florentflament.com/blog/setting-keystone-v3-
domains.html
• https://wiki.openstack.org/wiki/Security_Notes
Secure Keystone Deployment
29
References (Images)
• Crime Identity Theft: http://internet.phillipmartin.info/crime_identity_theft.gif
• Computer Theft: http://blogs.citypages.com/blotter/Computer%20theft.gif
• Mickey Washington ID:
http://assets.nydailynews.com/polopoly_fs/1.1864391!/img/httpImage/image.jpg_gen/de
rivatives/article_970/mickey13n-1-web.jpg
• Threat, Asset, and Vulnerability:
http://www.technetics.com.au/images/easyblog_images/79/b2ap3_thumbnail_manage_y
our_risk_400_20140924-122014_1.jpg
• Openstack security Notes: http://www.openstack.org/assets/openstack-logo/openstack-
one-color-alt.pdf
• OpenStack security Guide: http://docs.openstack.org/common/images/openstack-
security-guide.jpg
Secure Keystone Deployment
30

More Related Content

PDF
Company and Market Overview
PDF
Managed It Services Pricing PowerPoint Presentation Slides
PDF
Athena & Step Function 으로 통계 파이프라인 구축하기 - 변규현 (당근마켓) :: AWS Community Day Onl...
PDF
Integrating Fiware Orion, Keyrock and Wilma
PPTX
How to migrate from Lotus Notes to SharePoint 2013 or SharePoint Online
PPTX
IT전략계획-04.보안 아키텍처
PPT
Lotus notes app migration process v1.2
PPTX
Proactive Performance Monitoring for Genesys Call Centers
Company and Market Overview
Managed It Services Pricing PowerPoint Presentation Slides
Athena & Step Function 으로 통계 파이프라인 구축하기 - 변규현 (당근마켓) :: AWS Community Day Onl...
Integrating Fiware Orion, Keyrock and Wilma
How to migrate from Lotus Notes to SharePoint 2013 or SharePoint Online
IT전략계획-04.보안 아키텍처
Lotus notes app migration process v1.2
Proactive Performance Monitoring for Genesys Call Centers

What's hot (8)

PDF
Migration Day - SAP en AWS: Estrategias de migración
PDF
Iis server 2003
PPTX
Migrating and modernizing your data estate to Azure with Data Migration Services
PDF
Event Dispatcher
PDF
DevOps with Chef
PDF
AWS 관리형 서비스를 중심으로 한 NCSOFT 와 Reality Reflection의 클라우드 사용기 - AWS Summit Seoul ...
PDF
Azure Monitoring Overview
PPTX
Migrate an Existing Application to Microsoft Azure
Migration Day - SAP en AWS: Estrategias de migración
Iis server 2003
Migrating and modernizing your data estate to Azure with Data Migration Services
Event Dispatcher
DevOps with Chef
AWS 관리형 서비스를 중심으로 한 NCSOFT 와 Reality Reflection의 클라우드 사용기 - AWS Summit Seoul ...
Azure Monitoring Overview
Migrate an Existing Application to Microsoft Azure
Ad

Similar to Secure Keystone Deployment (20)

PPT
Creating Secure Applications
PPT
OpenStack - Security Professionals Information Exchange
PDF
VMworld 2013: Introducing NSX Service Composer: The New Consumption Model for...
PPTX
Openstack security presentation 2013
PDF
Masterless Puppet Using AWS S3 Buckets and IAM Roles
PDF
Asia-14-Rios-Owning-A-Building-Exploiting-Access-Control-And-Facility-Managem...
PDF
Securing Microservices using Play and Akka HTTP
PPTX
DevSecOps: Key Controls to Modern Security Success
PDF
VMworld 2013: NSX PCI Reference Architecture Workshop Session 2 - Privileged ...
PDF
DCEU 18: Docker Enterprise Platform and Architecture
PPTX
Cloud_Security_Final
PPTX
Integrate Security into DevOps - SecDevOps
PPTX
Fiware cloud developers week brussels
PDF
Breaking The Cloud Kill Chain
PPTX
Cloud Security Architecture.pptx
PDF
Vietnam Global Azure Bootcamp 2019 - Security on Azure Kubernetes Services wi...
PPT
Securing Servers in Public and Hybrid Clouds
PPTX
Security for cloud native workloads
PDF
DevSecOps: Taking a DevOps Approach to Security
PPTX
How to Install and Configure your own Identity Manager GE
Creating Secure Applications
OpenStack - Security Professionals Information Exchange
VMworld 2013: Introducing NSX Service Composer: The New Consumption Model for...
Openstack security presentation 2013
Masterless Puppet Using AWS S3 Buckets and IAM Roles
Asia-14-Rios-Owning-A-Building-Exploiting-Access-Control-And-Facility-Managem...
Securing Microservices using Play and Akka HTTP
DevSecOps: Key Controls to Modern Security Success
VMworld 2013: NSX PCI Reference Architecture Workshop Session 2 - Privileged ...
DCEU 18: Docker Enterprise Platform and Architecture
Cloud_Security_Final
Integrate Security into DevOps - SecDevOps
Fiware cloud developers week brussels
Breaking The Cloud Kill Chain
Cloud Security Architecture.pptx
Vietnam Global Azure Bootcamp 2019 - Security on Azure Kubernetes Services wi...
Securing Servers in Public and Hybrid Clouds
Security for cloud native workloads
DevSecOps: Taking a DevOps Approach to Security
How to Install and Configure your own Identity Manager GE
Ad

Recently uploaded (20)

PPTX
web development for engineering and engineering
PPTX
Sustainable Sites - Green Building Construction
PDF
PPT on Performance Review to get promotions
PPT
Project quality management in manufacturing
PPTX
Geodesy 1.pptx...............................................
DOCX
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
PDF
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
PDF
composite construction of structures.pdf
PDF
Digital Logic Computer Design lecture notes
PPT
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
PDF
Operating System & Kernel Study Guide-1 - converted.pdf
PDF
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
PPTX
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
PPTX
Foundation to blockchain - A guide to Blockchain Tech
PDF
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
PPTX
Lecture Notes Electrical Wiring System Components
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
PPTX
UNIT 4 Total Quality Management .pptx
PPTX
CH1 Production IntroductoryConcepts.pptx
PPT
Mechanical Engineering MATERIALS Selection
web development for engineering and engineering
Sustainable Sites - Green Building Construction
PPT on Performance Review to get promotions
Project quality management in manufacturing
Geodesy 1.pptx...............................................
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
composite construction of structures.pdf
Digital Logic Computer Design lecture notes
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
Operating System & Kernel Study Guide-1 - converted.pdf
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
Foundation to blockchain - A guide to Blockchain Tech
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
Lecture Notes Electrical Wiring System Components
Embodied AI: Ushering in the Next Era of Intelligent Systems
UNIT 4 Total Quality Management .pptx
CH1 Production IntroductoryConcepts.pptx
Mechanical Engineering MATERIALS Selection

Secure Keystone Deployment

  • 1. Secure Keystone Deployment: Lessons Learned and Best Practices Priti Desai Sr. Software Engineer Secure Keystone Deployment 1
  • 2. The Symantec Team • Cloud Platform Engineering – We are building a consolidated cloud platform that provides infrastructure and platform services for next generation Symantec products and services • Me – In Security for over 6 years – Symantec Insight - Reputation Based Security – Symantec Data Analytics Platform – OpenStack Engineer - Keystone – OpenStack Security Group – Cop Open Source Secure Keystone Deployment 2
  • 3. OpenStack Security Group Secure Keystone Deployment 3 security notes Retrieved from http://www.openstack.orgRetrieved from http://docs.openstack.org
  • 4. Secure Keystone Deployment Why is Keystone security critical? What is Keystone? How is Authentication process implemented in Keystone? How is Authorization mechanism implemented in OpenStack?
  • 5. AuthN Overview Secure Keystone Deployment 5 Cloud User Cloud User Identity (SQL/LDAP) Keystone Token (SQL) Identity (SQL/LDAP) Keystone Token (SQL) Request sent with Username and Password Verify username and password (hash of password) Successful verification Request metadata for user tenant relationship Assignment (SQL) Assignment (SQL) User tenant relationship information Request to generate new token Response with new token Response with token
  • 6. AuthZ Overview Secure Keystone Deployment 6 Cloud User Cloud User Keystone OpenStack Service Keystone OpenStack Service Request sent with session token Verify session token Successful verification Is this token correct? Does it allow the service usage? Service executes the request Response with success
  • 7. Secure Keystone Deployment Why is Keystone security critical? Does it store/transmit any sensitive information? What kind of cloud asset does it store? Is any type of attack possible on Keystone? Can it bring down the entire cloud?
  • 8. Keystone Security is Critical Secure Keystone Deployment 8 • Gatekeeper • Access to OpenStack Cloud • Assets • Users • Passwords • Tokens • Roles • Catalog • Vulnerable to DoS Retrieved from http://internet.phillipmartin.info Retrieved from http://blogs.citypages.com Retrieved from http://assets.nydailynews.com
  • 9. What was our approach to identifying key vulnerabilities? Secure Keystone Deployment 9
  • 10. Security Risks Secure Keystone Deployment 10 • Global Security Office  Threat Model  Penetration Tests  Traceability Matrix Retrieved from http://www.technetics.com.au
  • 12. Secure Keystone Deployment 12 Spoofing Tampering Repudiation Information Disclosure Denial of Service Elevation of Privileges Threat Model
  • 13. What kind of security deficiencies did we discover? Secure Keystone Deployment 13
  • 14. Secure Keystone Deployment 14 Attack: Keystone user credential theft Attack: Insecure file permissions on Keystone.conf Keystone.conf Attack: Access to cloud admin privileges for almost free Attack: Leaking sensitive data in log messages Attack: DoS – Authentication chaining - Havana Attack: Unauthorized access to MySQL database Many more …
  • 15. Traceability Matrix Secure Keystone Deployment 15 ✖ ✖ ✖
  • 16. Keystone User Credential Theft Secure Keystone Deployment 16
  • 17. Mitigate: Secure Communication - SSL Secure Keystone Deployment 17 Hardware Load Balancer Hardware Load Balancer Keystone KeystoneKeystone SSL Client SSL Server SSL Client SSL Server mod_ssl 35357/SSL 5000/SSL mod_ssl 35357/SSL 5000/SSL mod_ssl 35357/SSL 5000/SSL Public API Admin API
  • 18. Insecure file permissions on Keystone.conf Secure Keystone Deployment 18 Mitigate: • Restrict ownership to service user - chown keystone:keystone /etc/keystone/keystone.conf • Restrict to read and write by the owner - chmod 640 /etc/keystone/keystone.conf hostnameabc hostnameabc hostnameabcuser user user
  • 19. Access to admin privileges is almost free Secure Keystone Deployment 19 • Service Token • Bootstrap Keystone • Cloud admin privileges • Register bad service/endpoints
  • 20. Mitigate: Disable Service Token • Comment out admin_token from /etc/keystone/keystone.conf: admin_token=e2112effd3ff05b8c88ad14e096e6615 • Remove admin token auth middleware from /etc/keystone/keystone-paste.ini: [filter:admin_token_auth] paste.filter_factory = keystone.middleware:AdminTokenAuthMiddleware.factory Secure Keystone Deployment 20
  • 21. Who is the cloud admin now? Secure Keystone Deployment 21
  • 22. Create Cloud Admin • Leveraging Keystone Domain • Before disabling service token: • Create a domain “cloud_admin_domain” • Grant “admin” role to appropriate user “Bob Smith” • Update keystone policy.json file: • Replace: "cloud_admin": [["rule:admin_required", "domain_id:admin_domain_id"]], • With: "cloud_admin”: [["rule:admin_required”,"domain_id:<cloud_admin_domain_id>"]], Secure Keystone Deployment 22
  • 23. Leaking Sensitive Information in Log Messages • Debug mode include plaintext request logging • Passwords • Tokens • Mitigate: • Disable debug mode in keystone.conf with: • With debug mode ON, upgrade keystone client: • python-keystoneclient >= 0.10.1 (OSSN-0024) Secure Keystone Deployment 23 [DEFAULT] debug=False
  • 24. Leaking Sensitive Information in Log Messages Identity API V2 - INFO level logs contains auth tokens (OSSN-0023) Mitigate: • Set the log level to WARN in logging.conf: Secure Keystone Deployment 24 [handler_file] class = FileHandler Level = WARN
  • 25. Keystone DoS Attack Identity API V3 – Authentication Chaining – CVE-2014-2828 Secure Keystone Deployment 25
  • 26. Keystone DoS Attack Mitigate: • Impacted Versions: from 2013.1 to 2013.2.3 • Patch applied during IceHouse rc2 • Upgrade Keystone >= 2013.2.4 Secure Keystone Deployment 26
  • 29. References • http://docs.openstack.org/developer/keystone/ • https://blog-nkinder.rhcloud.com/?p=7 • https://blueprints.launchpad.net/keystone/+spec/service-scoped- tokens • http://docs.openstack.org/sec/ • http://www.florentflament.com/blog/setting-keystone-v3- domains.html • https://wiki.openstack.org/wiki/Security_Notes Secure Keystone Deployment 29
  • 30. References (Images) • Crime Identity Theft: http://internet.phillipmartin.info/crime_identity_theft.gif • Computer Theft: http://blogs.citypages.com/blotter/Computer%20theft.gif • Mickey Washington ID: http://assets.nydailynews.com/polopoly_fs/1.1864391!/img/httpImage/image.jpg_gen/de rivatives/article_970/mickey13n-1-web.jpg • Threat, Asset, and Vulnerability: http://www.technetics.com.au/images/easyblog_images/79/b2ap3_thumbnail_manage_y our_risk_400_20140924-122014_1.jpg • Openstack security Notes: http://www.openstack.org/assets/openstack-logo/openstack- one-color-alt.pdf • OpenStack security Guide: http://docs.openstack.org/common/images/openstack- security-guide.jpg Secure Keystone Deployment 30