Secure Developer Access at Decisiv
0 likes433 views
The document discusses the implementation and security features of Teleport, an authentication and proxy solution that facilitates developer access while adhering to various security standards. It outlines the importance of designing secure and usable systems, user role management challenges, and the architecture of Teleport, which is designed for high availability and integrates with existing identity providers. Additionally, it emphasizes the significance of session recording and SIEM integration for monitoring user actions and ensuring security compliance.
![Use Labels and Metadata
Interpolation
commands:
- name: instance_id
command: ['/bin/curl', 'http://169.254.169.254/latest/meta-data/instance-id']
period: 24h0m0s
- name: account_id
command: ['/bin/sh', '-c', 'curl -s
http://169.254.169.254/latest/dynamic/instance-identity/document|jq -r .accountId']
period: 24h0m0s
- name: public_ip
command: ['/bin/curl', 'http://169.254.169.254/latest/meta-data/public-ipv4']
period: 24h0m0s
- name: app_owner
command: ['/opt/bin/get_tag’, ‘App_Owner’]
period: 24h0m0s](https://image.slidesharecdn.com/teleportcasestudywithdecisivadminhuntermadison-200902171809/85/Secure-Developer-Access-at-Decisiv-18-320.jpg)
![Use Labels and Metadata Interpolation
kind: role
metadata:
name: developers
spec:
allow:
logins:
- developers # '{{ external["ssh_user"] }}'
node_labels:
app_owner: '{{ external["team"] }}'
deny:
logins: null
node_labels:
app_owner: operations
options:
cert_format: standard
client_idle_timeout: 8h0m0s
enhanced_recording:
- command
- network
forward_agent: false
max_session_ttl: 8h0m0s
port_forwarding: true
version: v3](https://image.slidesharecdn.com/teleportcasestudywithdecisivadminhuntermadison-200902171809/85/Secure-Developer-Access-at-Decisiv-20-320.jpg)
Secure Developer Access at Decisiv
- 1. Teleport at Decisiv Hunter Madison
- 2. What we will cover Who is Hunter? What does Decisiv do? Securing Developer Access Implementation Decisions Q&A Session
- 4. Auditable Standards ● ISO 27002 ● ISO 9000 ● PCI DSS ● FedRamp ● NIST 500-292 ● GDPR ● CCPA
- 5. Two Core Questions ● Who can do what, when? ● Why did someone do something then?
- 6. Designing Secure Systems ● When Security Gets in the Way - Interactions, volume 16, issue 6: Norman, D. ○ “The audience, either not understanding the rationale or simply disagreeing with the necessity for the procedures imposed upon them, see these as impediments to accomplishing their jobs.” ● Music Software & Interface Design: Steinberg's Dorico - Tantacrul ○ “The reality of modern life is that we are now required to keep learning software all the time. It’s overwhelming, and a designer’s job should be to try and reduce that pain as much as possible.” ● It's not good enough to be secure ● It has to be usable
- 7. Users
- 8. Users Don’t Change... ● End user workflows are generally set in stone ○ Workflows don’t change as the company scales up ○ What is ok with 5 developers isn't ok with 60 ● Anything change made that an end user doesn’t see immediate benefit in is hard to sell
- 9. ...But The World Does ● Your company will grow in size and attack surface ○ Laptops will get stolen ○ You will need more cloud resources ● What works for five developers won’t work for sixty, six hundred, or six thousand
- 10. The User Role
- 11. The User Role ● Creates user accounts ● Adds ssh keys ● (Sometimes) tries to keep the UIDs consistent ● (Sometimes) sets up a .bash_profile ● (Sometimes) configures sudoers
- 12. The User Role Has Problems ● What happens when new people join and need access? ○ Hopefully, their key is provided to you the day they start ○ And they don’t need access immediately ○ Script needs to get run everywhere ● What happens when people leave? ○ Script needs to run everywhere again ○ Revocations don’t happen as fast as they should ● What happens when access is used to change application or server state improperly? ○ Installing apps onto boxes scheduled for decommissioning ○ App consoles ● What happens when one developer really wants to connect their blackberry to the vpn and ssh into boxes?
- 13. How We Fixed It
- 14. Teleport At A High Level ● It’s a highly available cluster of authentication and proxy servers which create an auditable and IDP secured SSH bastion host ● It’s also X.509 Certificate Authority ● It can store its state locally or in services like S3 and DynamoDB ○ For this talk, we are assuming that Teleport is configured to use S3 and DynamoDB ● It records end user actions into multiple auditable forms
- 15. IDP
- 16. Configure your IDP ● Teleport benefits from having a good ontology inside of your IDP ● Your IDP pushes... ○ Groups which become Teleport roles ○ Attributes which are interpolated when evaluating roles ● All of this data is accessible to you at login time ● Making good use of it cuts down on the administrative headache significantly
- 17. First Time Provisioning Workflow
- 18. Use Labels and Metadata Interpolation commands: - name: instance_id command: ['/bin/curl', 'http://169.254.169.254/latest/meta-data/instance-id'] period: 24h0m0s - name: account_id command: ['/bin/sh', '-c', 'curl -s http://169.254.169.254/latest/dynamic/instance-identity/document|jq -r .accountId'] period: 24h0m0s - name: public_ip command: ['/bin/curl', 'http://169.254.169.254/latest/meta-data/public-ipv4'] period: 24h0m0s - name: app_owner command: ['/opt/bin/get_tag’, ‘App_Owner’] period: 24h0m0s
- 19. Labels in the Teleport UI
- 20. Use Labels and Metadata Interpolation kind: role metadata: name: developers spec: allow: logins: - developers # '{{ external["ssh_user"] }}' node_labels: app_owner: '{{ external["team"] }}' deny: logins: null node_labels: app_owner: operations options: cert_format: standard client_idle_timeout: 8h0m0s enhanced_recording: - command - network forward_agent: false max_session_ttl: 8h0m0s port_forwarding: true version: v3
- 21. SSH
- 22. It’s Just SSH Host proxy.example.com StrictHostKeyChecking no UserKnownHostsFile /dev/null CheckHostIP no CertificateFile ~/.tsh/keys/example.com/%u@example.com-cert.pub IdentityFile ~/.tsh/keys/example.com/%u@example.com Port 3023 Host *.apps.example.com Port 3022 ProxyCommand ssh %r@proxy.example.com -s proxy:%h:%p StrictHostKeyChecking no
- 23. Really, Just SSH ● Your tools that use SSH and can read ~/.ssh/ssh_config will work with Teleport too! ○ tsh login and go! ○ Provided it supports certificate authentication (IDEA-216138) ● I’ve tested ○ Ansible (Parminko) ○ Inspec (Train) ○ Capistrano (Net::SSH) ○ OpenSSH ● Windows ○ Enable OpenSSH for Windows ○ tsh.exe only provides the signed certificates
- 25. /proc/<pid>/environ ● Teleport sets environment variables for each session ○ The two to know TELEPORT_SESSION and SSH_TELEPORT_USER ● If you are unsure who started a process (like a tmux or screen session) check the environ and find the TELEPORT_SESSION
- 26. Terminal Example
- 27. Looking at Sessions, Proactively
- 28. Session Recording ● Teleport records every session for playback ● These are great to watch, but hard to search through at scale
- 29. Events Table ● With “Enhanced Session Recording” enabled, a new “session.command” event becomes available. ● It’s worth getting this data into your SIEM/Logging solution ○ NEW_IMAGE ● This gives you a really quick and easy way to find and log “problem commands” ○ Screen ○ Tmux ○ psql
- 30. SIEM Integration
- 31. Not Just SSH
- 32. You Have A X509 CA ● Every time a user logs in with tsh, they get a newly refreshed X509 certificate ● You can use these client certs to authenticate with a lot of tools that don’t necessarily support SSO out of the box ○ OpenVPN ○ Postgres ○ Mariadb ● For server certs start with `tctl get --with-secrets cert_authority` ● Same session expiration rules apply ● Role support is application dependent ○ You can see groups (as organization), valid logins (as locality) and the username (as common name) in the subject ○ Your mileage will vary if the application supports parsing that data
- 33. Demo
- 36. “User” Tier ● End Users ○ Everywhere! ○ We are a global, remote company ○ Okta as an IDP ● Nodes ○ Lots of AWS accounts ○ We do account vending ○ Join tokens via cross account STS/SSM
- 37. Cluster Mode ● Tunneled ○ Single cluster ○ Nodes connect via the internet ○ Single SAML SP ● Trusted ○ Each account gets is own cluster ○ Single SAML SP shared via a primary cluster ○ We used this up to 4.0 ○ When it breaks, it *hurts* ○ Users need to be aware of cluster switches
- 38. Load Balancing Tier ● We run Teleport in a HA setup ● Application Load Balancers ○ Change your timeout to get the web console to work. ○ You can let teleport generate its own self signed SSL cert. ALBs don’t check SSL. ● Network Load Balancers ○ You will see constant errors in the logs because of the heartbeat.
- 39. Application Tier ● We run Teleport via an autoscaling group with one host per AZ ● We stack the Auth and Proxy components onto the same hosts ● Use SSM for your “break glass” mechanism
- 40. Database Tier ● We use S3 to store sessions and Dynamo to store state and events ● This makes our auth/proxy hosts stateless ○ Really nice for upgrades ● Events in Dynamo open up SIEM integrations ○ DyanmoDB streams with NEW_IMAGE
- 42. Any Questions?
- 43. Recommended Next Steps Download Teleport https://gravitational.com/teleport/download Join Teleport Community https://community.gravitational.com Read the Teleport Admin Guide https://gravitational.com/teleport/docs/
- 44. Teleport at Decisiv Hunter Madison