SlideShare a Scribd company logo

DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016

Chris Gates, profile picture
Chris Gates

The document presents insights from Chris Gates and Ken Johnson on security practices related to DevOps and applications, highlighting common flaws in tooling, employee monitoring, and proactive measures for web application security. It emphasizes hardening production environments in AWS, implementing multi-factor authentication, monitoring service usage, and responding to potential breaches through various AWS tools and services. Additionally, it offers recommendations for protecting employee workstations and securing GitHub repositories from accidental data leaks.

Internet
1 of 155
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
DevOops, Redux Chris Gates, Ken Johnson AppSec USA 2016
Background: KJ β€’ I’m NOT Kevin Johnson
Background: KJ β€’I’m NOT Ken Bone
Background: KJ β€’I AM Ken Johnson β€’CTO of nVisium - @cktricky β€’Former US Navy β€’Topics I’ve talked about: – Rails Security (Railsgoat) – Building an AppSec Program – DevOops: Common Flaws in DevOps Tooling – Exploitation of Web Applications
Background: KJ β€’ I run engineering (product) β€’ I work for a security company β€’ I have some concerns...same as you
Background: CG β€’ Chris Gates, Sr. Security Engineer - Uber β€’ Former Army β€’ Topics I’ve talked about: – Breaking into Oracle, Windows, lots of stuff – Phishing – Low to Pwned – Purple Teaming – DevOops – Common Flaws in DevOps Tooling
Background: CG β€’ Was a full time breaker β€’ Now full-ish time fixer β€’ Currently doing Blue Team stuff - <3 Python + REST APIs - Astonished at # of ppl who can’t Internet
About This Talk β€’ Original talk DevOops was about breaking stuff β€’ We were asked about β€œProactive” measures in DevOps/Agile/CI-CD environments – Quick Story β€’ We made a solution focused model based on β€œCommon” architecture and needs
Before We Begin β€’ Buckle up, lots of info coming your way β€’ Q&A will be reserved for hallway discussions β€’ Slides will have all the resources you need and will be available β€’ Sections are broken up between Human, Host, and Infrastructure
Employee Intelligence (Human) Making it difficult (for employees) to allow attackers to walk into our environment
Monitoring External Services β€’ Numerous ways for employees to accidently release data –Pastebin-like sites –Github β€’Gists β€’Code β€’ Examples: –Slack tokens in github –AWS configs in .dotfiles back ups –Tokens in logs/dumps/snippets
Monitoring GitHub β€’ How you could tackle the problem: –Use GitLab (internal) –Use gitolite (internal) –Use GitHub Enterprise (internal) –Use Phabricator (internal)
Monitoring GitHub β€’ But you won’t, you’ll set up a private GitHub for you org like everyone else. – Now you need to monitor when people post your private stuff on their personal repo – It happens. A lot.
Monitoring GitHub β€’ How you could tackle the problem: –Have employees join the your GitHub organization –Regularly crawl the list of members –Check out all their repos –Run regex against all files looking for known badness
Monitoring GitHub β€’ Gitrob –https://github.com/michenriksen/gitrob
Monitoring GitHub β€’ Gitrob
Monitoring GitHub
Monitoring GitHub
AWS Access Keys Example
AWS Access Keys Example
AWS Access Keys Example
Monitoring Pastebin* β€’ Pastebin* –https://github.com/jordan-wright/dumpmon –https://github.com/xme/pastemon –https://github.com/cvandeplas/pystemon
Monitoring Goals β€’ DumpMon https://github.com/jordan-wright/dumpmon
Monitoring Goals β€’ For Pay Services
Monitoring Goals β€’ For Pay Services
Monitoring Goals β€’For Pay Services - https://gitmonitor.com/
Monitoring Goals GitMonitor - Some options they provide
Workstation Protection (Host) Protecting and monitoring employees on their development workstations (and servers too)
Host Protections Developer Laptop Hardening β€’ osquery (OS X/Linux) β€’ Doorman β€’ BlockBlock β€’ Little Snitch β€’ CarbonBlack / Sysmon β€’ Splunk β€’ Simian
Host Protections β€’ osquery (https://osquery.io/) β€’ β€œosquery is an operating system instrumentation framework for OS X, Linux, and FreeBSD. The tools make low-level operating system analytics and monitoring both performant and intuitive.” β€’ β€œosquery exposes an operating system as a high-performance relational database. This allows you to write SQL queries to explore operating system data. With osquery, SQL tables represent abstract concepts such as running processes, loaded kernel modules, open network connections, browser plugins, hardware events or file hashes.”
Host Protections osquery β€’ Adhoc β€’ Scheduled β€’ Schedule query β€’ Collect logs β€’ Review change β€’ File Integrity Monitoring β€’ Yara rules β€’ Query packs
Host Protections osquery
Host Protections osquery
Host Protections β€’ Doorman (https://github.com/mwielgoszewski/doorman) β€’ β€œDoorman is an osquery fleet manager that allows administrators to remotely manage the osquery configurations retrieved by nodes.”
Host Protections
Host Protections β€’ BlockBlock (https://objective-see.com/products/blockblock.html) β€’ Kernel hook to identify any time software wants to persist β€’ Prompt to allow or deny β€’ β€œThe kernel extension tracks process creations, which are consumed by the daemon, which also monitors various persistence locations to detect any new items. Specifically the daemon (currently) watches for new kexts, launch daemon & agents, and new login items via the fsevents device (/dev/fsevents).”
Host Protections
Host Protections β€’ Little Snitch (https://www.obdev.at/products/littlesnitch/index.html) β€’ Host based firewall β€’ Prompt to allow or deny and for how long β€’ β€œLittle Snitch intercepts these unwanted connection attempts, and lets you decide how to proceed.”
Host Protections
Host Protections β€’ CarbonBlack (https://www.carbonblack.com/) β€’ Host based agent β€’ Monitor process create, writes, registry queries, net connections β€’ Create rules/watchlist for known bad behavior –Mimikatz-->company_name:*gentilkiwi* –FileVault Encryption Disabled -->process_name:fdesetup cmdline:disable –Unsigned JAR exec-->process_name:*.jar digsig_result: (digsig_result:"Unsigned") –OSX dump user hashes-->process_name:dscl cmdline:ShadowHashData
Host Protections
Host Protections
Host Protections β€’ Sysmon β€’ https://www.root9b.com/sites/default/files/whitepapers/R9B_blog_005_white paper_01.pdf β€’ https://jon.glass/tag/sysinternals/ β€’ http://www.darkoperator.com/blog/2014/8/8/sysinternals-sysmon β€’ https://www.bsk-consulting.de/2015/03/21/detect-system-file-manipulations- with-sysinternals-sysmon/ β€’ https://www.firemon.com/enhance-windows-anomaly-detection-sysmon/
Host Protections β€’ Splunk
Host Protections
Host Protections β€’ ELK
Host Protections β€’ ELK
Host Protections OSX Patch Management - Simian β€’ β€œSimian is an enterprise-class Mac OS X software deployment solution.” β€’ Allows you to push munki updates β€’ Free / OSS β€’ Runs on google cloud β€’ Project: https://github.com/google/simian
Host Protections Why do we bring this up? β€’ Some people aren’t aware you can perform free OSX patch management β€’ There are a lot of OSX developer shops without an β€œenterprise budget” β€’ Patch management is a no-brainer and security 101
Host Protections https://github.com/google/simian
Host Protections Simian Consists of 2 parts: β€’ Client – Private and Public SSL Keys used to authenticate – Configuration unique per OSX client β€’ Web Application/Server – Runs on Google Cloud – Keep in mind its free but… not for long (eventually costs a little for storage) Takes about a week to learn and get setup
Host Protections Web Application used to Manage Updates
Host Protections Client - DMG File
Host Protections Simian Recap: β€’ Learning curve is moderately difficult IMO β€’ Free-ish (eventually storage costs but still very minimal) β€’ Useful for patch updates and monitoring clients systems for low disk space, uptime, etc.
Production Protection (Infra) Protecting and monitoring production environments (AWS)
My AWS Goals β€’ Harden – Make it difficult to reach your AWS environment β€’ Monitor – If your AWS environment is breached, you need to know and alert yourselves β€’ Restore – Have the ability to reconstruct data/configs after a β€œhack”
AWS’s Plan β€’ Took the AWS Security Fundamentals Course and… – Fortunately, our strategy lines up with AWS recommendations – You are responsible for leveraging the tools AWS provides (financially) – Your configuration… that is on you – https://aws.amazon.com/training/course-descriptions/security- fundamentals/
AWS Hardening Basics Making it difficult (for attackers) to reach our environment
Hardening Checklist 1. Don’t Use The Root Account! 2. Disable Access Keys for Root Account 3. Multi-Factor Authentication 4. API + MFA 5. Strong Password Policy
Don’t Use Root Account β€’ Every AWS env has a root account, only necessary to use for very specific circumstances β€’ When these circumstances arise, notify your team that the account will be used β€’ We will discuss why this is important when we talk about CloudWatch metrics
Disable/Delete Root Account Access Keys β€’ Just delete them if they exist – Disable the access keys in the event you are unable to delete them completely for some reason β€’ Make sure your admins have a (verbal/written) policy that states β€œwe don’t create access keys for the root account”
MFA β€’ If credentials are stolen or guessed, we want a second layer of protection β€’ You can use apps or hardware to do this – Google Authenticator (Apps) – Gemalto (Hardware) β€’ Find the full list of MFA devices here: https://aws.amazon.com/iam/details/mfa/ β€’ This is so ridiculously easy to do, everyone should do it
MFA See the published slide deck for step by step instructions
MFA β€’ At this point, it's worth mentioning that non- administrators or those without IAM privileges cannot enable MFA on their own account β€’ Why is this a problem? Well, they need to be able to enable MFA on their own device… not the administrator’s β€’ Fortunately, we have a solution!
MFA
MFA β€’ Okay so that wasn’t the easiest to read, so here is the link: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_c redentials_delegate-permissions_examples.html#creds- policies-mfa-console β€’ Basically this IAM policy allows a user to manage their *OWN* MFA device
MFA (for Root Account) β€’ Need a shared MFA for root? TOTP! β€’ Recommend using something like 1password for teams, can share the TOTP code: https://support.1password.com/guides/mac/totp.html https://www.youtube.com/watch?v=eZyb-ArMK9g
API + MFA β€’ You have the ability to place a restriction where resources can only be interacted with if the user has authenticated with MFA β€’ This helps prevent (ab)use should someone steal access keys or credentials
API + MFA β€’ This entry enforces MFA for Web/API β€’ Do this for Admin & Power-User groups at a minimum
API + MFA β€’ Truth be told, doing this can be painful at first β€’ Things that used to work, might not (via the API) β€’ Fortunately, we have some answers for you β€’ Firstly, let’s discuss STS or SecurityToken Service
API + MFA β€’ Leverage STS in order to interact with the AWS API should this MFA restriction be placed on resources (and it should ☺ ) β€’ Example of using STS: https://gist.github.com/cktricky/127be4e431563a986f0f
API + MFA Output of script
API + MFA Use the creds to leverage tools like ec2-api-tools (-O <access key id>–W <secret> and –T <session token>)
API + MFA And in case you don’t like Ruby… https://github.com/jimbrowne/aws-sts-helpers
API + MFA β€’ ElasticBeanstalk does not work with STS. Le Terrible. β€’ However, there is a workaround, use CodePipeline β€’ Very simple process to setup but only works with: – GitHub – AWS CodeCommit – Amazon S3
Password Policy β€’ Password policies are important because historically people do not choose complex passwords β€’ MFA should help, but we’re talking about a layered approach β€’ Again, making our AWS environment harder to reach
Example Password Policy
Hardening Recap β€’ Make credentials hard to guess β€’ If guessed or stolen, we still have MFA β€’ Remember MFA only protects against the web and NOT the API… unless you change your policies and use STS β€’ Root account is King, protect your King
Hardening Recap β€’ Things we did not (and won’t discuss) – S3 bucket policies – Security Group configurations – SSH Key Management – Encrypting Data (Volumes, S3 buckets) β€’ Trusted Advisor – Use it, because it catches a lot of β€œlow hanging fruit” style issues
Hardening Recap β€’ Links to resources that discuss the items we’re not covering: – https://d0.awsstatic.com/whitepapers/compliance/AWS_Auditing_Securi ty_Checklist.pdf – http://aws-de-media.s3.amazonaws.com/images/Produktblaetter/AWS- Security-Check-List_eng.pdf – http://www.slideshare.net/AmazonWebServices/masterclass-advanced- security-best-practices β€’ Frankly you can’t throw a rock without hitting some basic info regarding AWS Security Checklists
AWS Monitoring Detecting malicious activity
AWS Monitoring β€’ Assuming hardening (prevention) has failed, how would we know? β€’ Luckily, AWS provides several services which alert to anomalies β€’ We will walk through examples of using these services, but ultimately decide what is right for you β€’ Fair warning, some of these services will provide a lot of noise
AWS Monitoring 4 important services: β€’ CloudTrail – Logs β€’ SNS – Notifications β€’ Config – Alerts for modifications & noncompliance β€’ CloudWatch – Alerts for specific types of behavior
AWS Monitoring
AWS Monitoring
AWS CloudTrail
AWS Monitoring (CloudTrail) β€’ CloudTrail is primarily used for log collection β€’ Other services like CloudWatch, for example, use those logs to filter relevant data
AWS Monitoring (CloudTrail) Pretty easy, first turn it on..
AWS Monitoring (CloudTrail) Configure the log group
AWS Monitoring (CloudTrail) Allow the creation of an IAM role by CloudTrail
AWS Monitoring (CloudTrail) β€’ At this point you have cloudtrail enabled β€’ Next step, BEFORE moving to CloudWatch or Config, is configuring SNS topics
AWS SNS
AWS Monitoring (SNS) β€’ Fantastic offering, <3 it – Examples of ways to be notified by SNS β€’ SMS β€’ Email β€’ JSON Post to your Application’s API endpoint
AWS Monitoring (SNS) β€’ Receive SMS/Email/Slack notifications for important events β€’ ^ This is so you get immediate notifications β€’ You can have multiple subscribers, I’d suggest you use that functionality β€’ Basic gist? Receive immediate updates for things you want to see… immediately ☺
AWS Monitoring (SNS) Create a topic
AWS Monitoring (SNS) Create Subscription
AWS Monitoring (SNS) Create SMS (or whatever, but in this case, SMS)
AWS Monitoring (SNS) Example of creating email subscription… bottom line you can have multiple ways of notifying people
AWS Config
AWS Monitoring (Config) β€’ Config: – AWS resource inventory, configuration history, and configuration change notifications – Can either design custom Config rules or use managed (pre-packaged) AWS Config rules – Discovery -Change Management – Compliance -Incident Response
AWS Monitoring (Config) β€’ Pre-packaged β€œManaged” AWS Rules – CLOUD_TRAIL_ENABLED – EIP_ATTACHED – ENCRYPTED_VOLUMES – INCOMING_SSH_DISABLED – INSTANCES_IN_VPC – REQUIRED_TAGS – RESTRICTED_INCOMING_TRAFFIC https://docs.aws.amazon.com/config/latest/developerguide/evaluate-config_use-managed-rules.html
AWS Monitoring (Config) β€’ Examples of things you can have alerts set for: – Change in Firewall (Security Group) ports – Changes in VPC – Any change… at all
AWS Monitoring (Config) Go to the Config service and choose resources to track
AWS Monitoring (Config) Or choose to track everything
AWS Monitoring (Config) Create a bucket, create an SNS topic (…we’ll discuss next)
AWS Monitoring (Config) Allow the role to be created and you’re all set!
AWS CloudWatch
AWS Monitoring (CloudWatch) β€’ We can be very particular here about what it is we want to see β€’ Some very interesting things you can monitor β€’ Some examples: – Billing Alerts (Important for detection of abuse or mistakes) – Track Root Account Usage – Failed login attempts
Billing Alarm
AWS Monitoring (CloudWatch - Billing) β€’ Used to prevent abuse or mistakes from costing your organization money β€’ Analyze and approximate your monthly spend β€’ Configure via CloudWatch β€’ Use SNS for instantaneous alerting
AWS Monitoring (CloudWatch - Billing) Navigate to billing & cost management; enable billing alerts
AWS Monitoring (CloudWatch - Billing) Create an SNS topic
AWS Monitoring (CloudWatch - Billing) Subscribe to Topic
AWS Monitoring (CloudWatch - Billing) Navigate to CloudWatch -> Metrics -> Billing
AWS Monitoring (CloudWatch - Billing) Choose USD/Estimate Charges -> Create Alarm
AWS Monitoring (CloudWatch - Billing) Set price point, SNS topic, and create alarm
AWS Monitoring (CloudWatch - Billing) Exact steps to enable can be found here: http://docs.aws.amazon.com/awsaccountbilling/latest/about v2/free-tier-alarms.html
Root Login Alarm
AWS Monitoring (CloudWatch – Root Login) β€’ Remember how I said don’t use the Root account routinely? β€’ BUT… if this account is used, you should know about it β€’ This is the reason you’ll want to notify others (who receive SNS alerts) of the fact you are about to use the account
AWS Monitoring (CloudWatch – Root Login) Choose log group, create metric
AWS Monitoring (CloudWatch – Root Login) Define Logs Metric Filter
AWS Monitoring (CloudWatch – Root Login) Assign/Create Filter
AWS Monitoring (CloudWatch – Root Login) Click β€œCreate Alarm”
AWS Monitoring (CloudWatch – Root Login) Define Alarm and you’re good…
AWS Monitoring (CloudWatch – Root Login) Exact steps (with pics) exist here: https://blogs.aws.amazon.com/security/post/Tx3PSPQSN8 374D/How-to-Receive-Notifications-When-Your-AWS- Account-s-Root-Access-Keys-Are-Used
Failed Login Alarm
AWS Monitoring (CloudWatch – Failed Logins) β€’ In the event someone is trying to break in, let’s alert ourselves to this! β€’ Failed logins typically suggest either someone forgot their password or… someone is trying to guess yours
AWS Monitoring (CloudWatch – Failed Logins) β€’ In the interest of time… the steps are pretty much the same as the root login alarm β€’ The Regex Filter however, is different
AWS Monitoring (CloudWatch – Failed Logins) Relevant filter pattern
AWS Monitoring (CloudWatch – Failed Logins) β€’ Exact steps exist here: http://docs.aws.amazon.com/awscloudtrail/la test/userguide/cloudwatch-alarms-for- cloudtrail.html#cloudwatch-alarms-for- cloudtrail-signin
Unauthorized Activity Alarm
AWS Monitoring (Unauthorized Activity) β€’ Remember the aws-interrogate tool? β€’ This alarm is the antidote β€’ Alerts us when someone is trying to access something in AWS, and does not have permissions
AWS Monitoring (Unauthorized Activity) β€’ Again, in the interest of time, steps are same as root login β€’ Regex is of course, different
AWS Monitoring (Unauthorized Activity) Set up regular expression
AWS Monitoring (Unauthorized Activity) What happens when we run interrogate
AWS Monitoring (Unauthorized Activity) The result of doing that is a nice nifty email to the engineering & security team
AWS Monitoring (CloudWatch) – Filter Patterns β€’ If you’d like to create your own custom filter patterns, here is a resource for that: http://docs.aws.amazon.com/AmazonCloudWatch/latest/De veloperGuide/FilterAndPatternSyntax.html
AWS + Splunk
AWS + Splunk β€’ Splunk is a pretty great resource for monitoring activity – Two separate plugins: β€’ Splunk App for AWS – https://splunkbase.splunk.com/app/1274/ β€’ Splunk Add-On – https://splunkbase.splunk.com/app/1876/
AWS + Splunk β€’ Examples of things you can view: – Billing – Topology – Usage – IAM Activity – SSH Key Pair Activity – User Activity – Network ACL(s) – VPC Activity – and a lot more…
AWS + Splunk
AWS + Splunk
AWS + Splunk
AWS + Splunk β€’ Splunk will need an AWS account in order to retrieve data β€’ Create account(s) for Splunk, grab the necessary permission policy from here: http://docs.splunk.com/Documentation/AddOns/released/A WS/ConfigureAWSpermissions
AWS + Splunk Configure AWS App for Splunk, add account(s), configure each input accordingly:
AWS + Splunk β€’To view things like IAM Activity… –Subscribe to a cloudtrail log via SNS –Utilize SQS and subscribe SQS to an SNS Topic
Monitoring Recap β€’ Alert yourself when things change β€’ This will get noisy, find a way to filter that which is important – If it’s a high risk event, send an SMS/Slack/Email blast β€’ At a minimum, alert yourself when odd things occur… like: – Billing increases past your normal spend – When somebody authenticates as Root – When someone has a login failure
Monitoring Recap β€’ Interesting Quora thread: – https://www.quora.com/My-AWS-account-was-hacked-and-I-have-a-50-000-bill-how-can-I- reduce-the-amount-I-need-to-pay β€’ Highlights from the article: – AWS has β€œa review board of sorts” to determine if you should be refunded – Bots are scouring GitHub searching for exposed access keys – One of the more AWS-seasoned responders mentioned doing part of what we discussed here today to avoid it – A decent number of the people posting on this thread said β€œYes, happened to me too”
AWS Restoration & Recovery Plan to fail, just don’t fail to plan
AWS Restoration & Recovery β€’ Do not USE AWS TO BACKUP YOUR AWS β€’ Offsite backups (meaning, off AWS site) β€’ Common things to back-up: – Databases/ Snapshots – S3 Buckets – EBS Volumes – CloudFormation Templates
AWS Restoration & Recovery β€’ Resources: – http://stackoverflow.com/questions/17087542/backup- solutions-for-aws-ec2-instances – https://github.com/Scalr/installer-ng – http://www.n2ws.com/blog/3-ways-ec2-windows- backup-and-recovery.html
AWS Incident Response Plan to fail, just don’t fail to plan
AWS Incident Response β€’ Could be its own talk β€’ Scout 2 -- https://github.com/nccgroup/Scout2 β€’Andrew Krug & Alex McCormack – Hardening AWS Environments and Automating Incident Response – https://www.youtube.com/watch?v=cmEUxxYFjK8
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
Contact Chris Gates Twitter: @carnal0wnage Blog: https://carnal0wnage.attackresearch.com Ken Johnson Twitter: @cktricky

More Related Content

PDF
Containerizing your Security Operations Center
Jimmy Mesta
Β 
PDF
Lares from LOW to PWNED
Chris Gates
Β 
PDF
DevOops & How I hacked you DevopsDays DC June 2015
Chris Gates
Β 
PPTX
DevOOPS: Attacks and Defenses for DevOps Toolchains
Chris Gates
Β 
PDF
Big Bang Theory: The Evolution of Pentesting High Security Enviroments IT Def...
Chris Gates
Β 
PDF
Appsec DC - wXf -2010
Chris Gates
Β 
PPTX
AWS Survival Guide
Ken Johnson
Β 
PDF
Exploiting XPC in AntiVirus
Csaba Fitzl
Β 
Containerizing your Security Operations Center
Jimmy Mesta
Β 
Lares from LOW to PWNED
Chris Gates
Β 
DevOops & How I hacked you DevopsDays DC June 2015
Chris Gates
Β 
DevOOPS: Attacks and Defenses for DevOps Toolchains
Chris Gates
Β 
Big Bang Theory: The Evolution of Pentesting High Security Enviroments IT Def...
Chris Gates
Β 
Appsec DC - wXf -2010
Chris Gates
Β 
AWS Survival Guide
Ken Johnson
Β 
Exploiting XPC in AntiVirus
Csaba Fitzl
Β 

What's hot (20)

PPTX
Devoops: DoJ Annual Cybersecurity Training Symposium Edition 2015
Chris Gates
Β 
PDF
LasCon 2014 DevOoops
Chris Gates
Β 
PDF
DevSecCon London 2017: Hands-on secure software development from design to de...
DevSecCon
Β 
PDF
Open Canary - novahackers
Chris Gates
Β 
PDF
Mitigating Exploits Using Apple's Endpoint Security
Csaba Fitzl
Β 
PDF
DevSecCon Singapore 2019: Workshop - Burp extension writing workshop
DevSecCon
Β 
PDF
From 0 to 0xdeadbeef - security mistakes that will haunt your startup
Diogo MΓ³nica
Β 
PDF
Security in serverless world (get.net)
Yan Cui
Β 
PDF
OWASP SF - Reviewing Modern JavaScript Applications
Lewis Ardern
Β 
PDF
DevSecCon London 2017 - MacOS security, hardening and forensics 101 by Ben Hu...
DevSecCon
Β 
PPTX
InSpec Workshop DevSecCon 2017
Mandi Walls
Β 
PDF
Csaba fitzl - Mount(ain) of Bugs
Csaba Fitzl
Β 
PDF
20+ ways to bypass your mac os privacy mechanisms
Csaba Fitzl
Β 
PDF
Tw noche geek quito webappsec
Thoughtworks
Β 
PDF
[Wroclaw #7] Why So Serial?
OWASP
Β 
PPTX
Web & Cloud Security in the real world
Madhu Akula
Β 
PDF
Attacking AWS: the full cyber kill chain
SecuRing
Β 
PPTX
Security Testing with Zap
Soluto
Β 
PDF
My tryst with sourcecode review
Anant Shrivastava
Β 
PDF
You wouldn't build a toast, would you?
Yan Cui
Β 
Devoops: DoJ Annual Cybersecurity Training Symposium Edition 2015
Chris Gates
Β 
LasCon 2014 DevOoops
Chris Gates
Β 
DevSecCon London 2017: Hands-on secure software development from design to de...
DevSecCon
Β 
Open Canary - novahackers
Chris Gates
Β 
Mitigating Exploits Using Apple's Endpoint Security
Csaba Fitzl
Β 
DevSecCon Singapore 2019: Workshop - Burp extension writing workshop
DevSecCon
Β 
From 0 to 0xdeadbeef - security mistakes that will haunt your startup
Diogo MΓ³nica
Β 
Security in serverless world (get.net)
Yan Cui
Β 
OWASP SF - Reviewing Modern JavaScript Applications
Lewis Ardern
Β 
DevSecCon London 2017 - MacOS security, hardening and forensics 101 by Ben Hu...
DevSecCon
Β 
InSpec Workshop DevSecCon 2017
Mandi Walls
Β 
Csaba fitzl - Mount(ain) of Bugs
Csaba Fitzl
Β 
20+ ways to bypass your mac os privacy mechanisms
Csaba Fitzl
Β 
Tw noche geek quito webappsec
Thoughtworks
Β 
[Wroclaw #7] Why So Serial?
OWASP
Β 
Web & Cloud Security in the real world
Madhu Akula
Β 
Attacking AWS: the full cyber kill chain
SecuRing
Β 
Security Testing with Zap
Soluto
Β 
My tryst with sourcecode review
Anant Shrivastava
Β 
You wouldn't build a toast, would you?
Yan Cui
Β 
Ad

Viewers also liked (12)

PDF
Going Purple : From full time breaker to part time fixer: 1 year later
Chris Gates
Β 
PPTX
Running Splunk on AWS
Alan Williams
Β 
PPTX
Travis Perkins: Building a 'Lean SOC' over 'Legacy SOC'
Splunk
Β 
PDF
hackcon2013-Dirty Little Secrets They Didn't Teach You In Pentesting Class v2
Chris Gates
Β 
PDF
Windows attacks - AT is the new black
Chris Gates
Β 
PDF
Top Security Challenges Facing Credit Unions Today
Chris Gates
Β 
PPTX
Dirty Little Secrets They Didn't Teach You In Pentest Class v2
Chris Gates
Β 
PDF
MSF Auxiliary Modules
Chris Gates
Β 
PDF
Building a Successful Internal Adversarial Simulation Team - Chris Gates & Ch...
Chris Gates
Β 
PPTX
Home Arcade setup (NoVA Hackers)
Chris Gates
Β 
PDF
ColdFusion for Penetration Testers
Chris Gates
Β 
PDF
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Chris Gates
Β 
Going Purple : From full time breaker to part time fixer: 1 year later
Chris Gates
Β 
Running Splunk on AWS
Alan Williams
Β 
Travis Perkins: Building a 'Lean SOC' over 'Legacy SOC'
Splunk
Β 
hackcon2013-Dirty Little Secrets They Didn't Teach You In Pentesting Class v2
Chris Gates
Β 
Windows attacks - AT is the new black
Chris Gates
Β 
Top Security Challenges Facing Credit Unions Today
Chris Gates
Β 
Dirty Little Secrets They Didn't Teach You In Pentest Class v2
Chris Gates
Β 
MSF Auxiliary Modules
Chris Gates
Β 
Building a Successful Internal Adversarial Simulation Team - Chris Gates & Ch...
Chris Gates
Β 
Home Arcade setup (NoVA Hackers)
Chris Gates
Β 
ColdFusion for Penetration Testers
Chris Gates
Β 
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Chris Gates
Β 
Ad

Similar to DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016 (20)

PPTX
System hardening - OS and Application
edavid2685
Β 
PPTX
Rapid Android Application Security Testing
Nutan Kumar Panda
Β 
PPTX
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
edm00se
Β 
PDF
Zane lackey. security at scale. web application security in a continuous depl...
Yury Chemerkin
Β 
PDF
Abusing bleeding edge web standards for appsec glory
Priyanka Aash
Β 
PPTX
Aleksei Dremin - Application Security Pipeline - phdays9
Alexey Dremin
Β 
PPTX
Hacking Tizen : The OS of Everything - Nullcon Goa 2015
Ajin Abraham
Β 
PDF
Road to Opscon (Pisa '15) - DevOoops
Gianluca Varisco
Β 
PPTX
Securing Your MongoDB Deployment
MongoDB
Β 
PDF
Owasp tds
snyff
Β 
PPTX
Csa container-security-in-aws-dw
Cloud Security Alliance, UK chapter
Β 
PDF
CMS Hacking Tricks - DerbyCon 4 - 2014
Greg Foss
Β 
PDF
Agile Secure Cloud Application Development Management
Adam Getchell
Β 
PPTX
Cloud Platforms for Java
3Pillar Global
Β 
PPTX
CollabSphere SC 103 : Domino on the Web : Yes, It's (Probably) Hackable
Darren Duke
Β 
PPTX
10 tips for Cloud Native Security
Karthik Gaekwad
Β 
PPTX
PowerShell - Be A Cool Blue Kid
Matthew Johnson
Β 
PPT
iOS Application Pentesting
n|u - The Open Security Community
Β 
PPTX
WordPress Security and Best Practices
Robert Vidal
Β 
PPTX
Creating Havoc using Human Interface Device
Positive Hack Days
Β 
System hardening - OS and Application
edavid2685
Β 
Rapid Android Application Security Testing
Nutan Kumar Panda
Β 
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
edm00se
Β 
Zane lackey. security at scale. web application security in a continuous depl...
Yury Chemerkin
Β 
Abusing bleeding edge web standards for appsec glory
Priyanka Aash
Β 
Aleksei Dremin - Application Security Pipeline - phdays9
Alexey Dremin
Β 
Hacking Tizen : The OS of Everything - Nullcon Goa 2015
Ajin Abraham
Β 
Road to Opscon (Pisa '15) - DevOoops
Gianluca Varisco
Β 
Securing Your MongoDB Deployment
MongoDB
Β 
Owasp tds
snyff
Β 
Csa container-security-in-aws-dw
Cloud Security Alliance, UK chapter
Β 
CMS Hacking Tricks - DerbyCon 4 - 2014
Greg Foss
Β 
Agile Secure Cloud Application Development Management
Adam Getchell
Β 
Cloud Platforms for Java
3Pillar Global
Β 
CollabSphere SC 103 : Domino on the Web : Yes, It's (Probably) Hackable
Darren Duke
Β 
10 tips for Cloud Native Security
Karthik Gaekwad
Β 
PowerShell - Be A Cool Blue Kid
Matthew Johnson
Β 
iOS Application Pentesting
n|u - The Open Security Community
Β 
WordPress Security and Best Practices
Robert Vidal
Β 
Creating Havoc using Human Interface Device
Positive Hack Days
Β 

More from Chris Gates (12)

PDF
Reiki 101 - Defcon29 MHHV
Chris Gates
Β 
PDF
WeirdAAL (Awesome Attack Library) CactusCon 2018
Chris Gates
Β 
PPTX
WeirdAAL (AWS Attack Library)
Chris Gates
Β 
PDF
PENETRATION TESTING FROM A HOT TUB TIME MACHINE
Chris Gates
Β 
PDF
Adversarial Simulation Nickerson/Gates Wild West Hacking Fest Oct 2017
Chris Gates
Β 
PDF
Open Source Information Gathering Brucon Edition
Chris Gates
Β 
PDF
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
Chris Gates
Β 
PDF
Big Bang Theory: The Evolution of Pentesting High Security Environments
Chris Gates
Β 
PDF
SOURCE Boston --Attacking Oracle Web Applications with Metasploit & wXf
Chris Gates
Β 
PDF
Hacking Oracle Web Applications With Metasploit
Chris Gates
Β 
PDF
Attacking Oracle with the Metasploit Framework
Chris Gates
Β 
PDF
Client-Side Penetration Testing Presentation
Chris Gates
Β 
Reiki 101 - Defcon29 MHHV
Chris Gates
Β 
WeirdAAL (Awesome Attack Library) CactusCon 2018
Chris Gates
Β 
WeirdAAL (AWS Attack Library)
Chris Gates
Β 
PENETRATION TESTING FROM A HOT TUB TIME MACHINE
Chris Gates
Β 
Adversarial Simulation Nickerson/Gates Wild West Hacking Fest Oct 2017
Chris Gates
Β 
Open Source Information Gathering Brucon Edition
Chris Gates
Β 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
Chris Gates
Β 
Big Bang Theory: The Evolution of Pentesting High Security Environments
Chris Gates
Β 
SOURCE Boston --Attacking Oracle Web Applications with Metasploit & wXf
Chris Gates
Β 
Hacking Oracle Web Applications With Metasploit
Chris Gates
Β 
Attacking Oracle with the Metasploit Framework
Chris Gates
Β 
Client-Side Penetration Testing Presentation
Chris Gates
Β 

Recently uploaded (20)

PPTX
Job_Card_System_Styled_lorem_ipsum_.pptx
Jeffkigen
Β 
PPTX
CHE NAA, , b,mn,mblblblbljb jb jlb ,j , ,C PPT.pptx
sumodmjohn3
Β 
PDF
Automated vs Manual WooCommerce to Shopify Migration_ Pros & Cons.pdf
CartCoders
Β 
PDF
Cloud-Scale Log Monitoring _ Datadog.pdf
MuhammadDhomanhuriMa
Β 
PDF
πŸ’° π”πŠπ“πˆ πŠπ„πŒπ„ππ€ππ†π€π πŠπˆππ„π‘πŸ’πƒ π‡π€π‘πˆ 𝐈𝐍𝐈 πŸπŸŽπŸπŸ“ πŸ’°
GRAB
Β 
PPTX
Introuction about WHO-FIC in ICD-10.pptx
naveenithkrishnan
Β 
PPTX
Digital Literacy And Online Safety on internet
riksa rifqi
Β 
PPTX
Slides PPTX World Game (s) Eco Economic Epochs.pptx
Steven McGee
Β 
PPTX
Introuction about ICD -10 and ICD-11 PPT.pptx
naveenithkrishnan
Β 
PPTX
Introduction about ICD -10 and ICD11 on 5.8.25.pptx
naveenithkrishnan
Β 
PPTX
Module 1 - Cyber Law and Ethics 101.pptx
anantyakhrisna1
Β 
PDF
Slides PDF The World Game (s) Eco Economic Epochs.pdf
Steven McGee
Β 
PDF
Testing WebRTC applications at scale.pdf
Giacomo Vacca
Β 
PDF
An introduction to the IFRS (ISSB) Stndards.pdf
farhankhan13694
Β 
PDF
Sims 4 Historia para lo sims 4 para jugar
lolpopy34
Β 
PPTX
SAP Ariba Sourcing PPT for learning material
NKKumar16
Β 
PDF
Unit-1 introduction to cyber security discuss about how to secure a system
shivangisharma636228
Β 
DOCX
Unit-3 cyber security network security of internet system
shivangisharma636228
Β 
PDF
Tenda Login Guide: Access Your Router in 5 Easy Steps
tendawifi16
Β 
PPTX
international classification of diseases ICD-10 review PPT.pptx
naveenithkrishnan
Β 
Job_Card_System_Styled_lorem_ipsum_.pptx
Jeffkigen
Β 
CHE NAA, , b,mn,mblblblbljb jb jlb ,j , ,C PPT.pptx
sumodmjohn3
Β 
Automated vs Manual WooCommerce to Shopify Migration_ Pros & Cons.pdf
CartCoders
Β 
Cloud-Scale Log Monitoring _ Datadog.pdf
MuhammadDhomanhuriMa
Β 
πŸ’° π”πŠπ“πˆ πŠπ„πŒπ„ππ€ππ†π€π πŠπˆππ„π‘πŸ’πƒ π‡π€π‘πˆ 𝐈𝐍𝐈 πŸπŸŽπŸπŸ“ πŸ’°
GRAB
Β 
Introuction about WHO-FIC in ICD-10.pptx
naveenithkrishnan
Β 
Digital Literacy And Online Safety on internet
riksa rifqi
Β 
Slides PPTX World Game (s) Eco Economic Epochs.pptx
Steven McGee
Β 
Introuction about ICD -10 and ICD-11 PPT.pptx
naveenithkrishnan
Β 
Introduction about ICD -10 and ICD11 on 5.8.25.pptx
naveenithkrishnan
Β 
Module 1 - Cyber Law and Ethics 101.pptx
anantyakhrisna1
Β 
Slides PDF The World Game (s) Eco Economic Epochs.pdf
Steven McGee
Β 
Testing WebRTC applications at scale.pdf
Giacomo Vacca
Β 
An introduction to the IFRS (ISSB) Stndards.pdf
farhankhan13694
Β 
Sims 4 Historia para lo sims 4 para jugar
lolpopy34
Β 
SAP Ariba Sourcing PPT for learning material
NKKumar16
Β 
Unit-1 introduction to cyber security discuss about how to secure a system
shivangisharma636228
Β 
Unit-3 cyber security network security of internet system
shivangisharma636228
Β 
Tenda Login Guide: Access Your Router in 5 Easy Steps
tendawifi16
Β 
international classification of diseases ICD-10 review PPT.pptx
naveenithkrishnan
Β 

DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016

  • 1. DevOops, Redux Chris Gates, Ken Johnson AppSec USA 2016
  • 4. Background: KJ β€’I AM Ken Johnson β€’CTO of nVisium - @cktricky β€’Former US Navy β€’Topics I’ve talked about: – Rails Security (Railsgoat) – Building an AppSec Program – DevOops: Common Flaws in DevOps Tooling – Exploitation of Web Applications
  • 5. Background: KJ β€’ I run engineering (product) β€’ I work for a security company β€’ I have some concerns...same as you
  • 6. Background: CG β€’ Chris Gates, Sr. Security Engineer - Uber β€’ Former Army β€’ Topics I’ve talked about: – Breaking into Oracle, Windows, lots of stuff – Phishing – Low to Pwned – Purple Teaming – DevOops – Common Flaws in DevOps Tooling
  • 7. Background: CG β€’ Was a full time breaker β€’ Now full-ish time fixer β€’ Currently doing Blue Team stuff - <3 Python + REST APIs - Astonished at # of ppl who can’t Internet
  • 8. About This Talk β€’ Original talk DevOops was about breaking stuff β€’ We were asked about β€œProactive” measures in DevOps/Agile/CI-CD environments – Quick Story β€’ We made a solution focused model based on β€œCommon” architecture and needs
  • 9. Before We Begin β€’ Buckle up, lots of info coming your way β€’ Q&A will be reserved for hallway discussions β€’ Slides will have all the resources you need and will be available β€’ Sections are broken up between Human, Host, and Infrastructure
  • 10. Employee Intelligence (Human) Making it difficult (for employees) to allow attackers to walk into our environment
  • 11. Monitoring External Services β€’ Numerous ways for employees to accidently release data –Pastebin-like sites –Github β€’Gists β€’Code β€’ Examples: –Slack tokens in github –AWS configs in .dotfiles back ups –Tokens in logs/dumps/snippets
  • 12. Monitoring GitHub β€’ How you could tackle the problem: –Use GitLab (internal) –Use gitolite (internal) –Use GitHub Enterprise (internal) –Use Phabricator (internal)
  • 13. Monitoring GitHub β€’ But you won’t, you’ll set up a private GitHub for you org like everyone else. – Now you need to monitor when people post your private stuff on their personal repo – It happens. A lot.
  • 14. Monitoring GitHub β€’ How you could tackle the problem: –Have employees join the your GitHub organization –Regularly crawl the list of members –Check out all their repos –Run regex against all files looking for known badness
  • 19. AWS Access Keys Example
  • 20. AWS Access Keys Example
  • 21. AWS Access Keys Example
  • 23. Monitoring Goals β€’ DumpMon https://github.com/jordan-wright/dumpmon
  • 26. Monitoring Goals β€’For Pay Services - https://gitmonitor.com/
  • 27. Monitoring Goals GitMonitor - Some options they provide
  • 28. Workstation Protection (Host) Protecting and monitoring employees on their development workstations (and servers too)
  • 29. Host Protections Developer Laptop Hardening β€’ osquery (OS X/Linux) β€’ Doorman β€’ BlockBlock β€’ Little Snitch β€’ CarbonBlack / Sysmon β€’ Splunk β€’ Simian
  • 30. Host Protections β€’ osquery (https://osquery.io/) β€’ β€œosquery is an operating system instrumentation framework for OS X, Linux, and FreeBSD. The tools make low-level operating system analytics and monitoring both performant and intuitive.” β€’ β€œosquery exposes an operating system as a high-performance relational database. This allows you to write SQL queries to explore operating system data. With osquery, SQL tables represent abstract concepts such as running processes, loaded kernel modules, open network connections, browser plugins, hardware events or file hashes.”
  • 31. Host Protections osquery β€’ Adhoc β€’ Scheduled β€’ Schedule query β€’ Collect logs β€’ Review change β€’ File Integrity Monitoring β€’ Yara rules β€’ Query packs
  • 34. Host Protections β€’ Doorman (https://github.com/mwielgoszewski/doorman) β€’ β€œDoorman is an osquery fleet manager that allows administrators to remotely manage the osquery configurations retrieved by nodes.”
  • 36. Host Protections β€’ BlockBlock (https://objective-see.com/products/blockblock.html) β€’ Kernel hook to identify any time software wants to persist β€’ Prompt to allow or deny β€’ β€œThe kernel extension tracks process creations, which are consumed by the daemon, which also monitors various persistence locations to detect any new items. Specifically the daemon (currently) watches for new kexts, launch daemon & agents, and new login items via the fsevents device (/dev/fsevents).”
  • 38. Host Protections β€’ Little Snitch (https://www.obdev.at/products/littlesnitch/index.html) β€’ Host based firewall β€’ Prompt to allow or deny and for how long β€’ β€œLittle Snitch intercepts these unwanted connection attempts, and lets you decide how to proceed.”
  • 40. Host Protections β€’ CarbonBlack (https://www.carbonblack.com/) β€’ Host based agent β€’ Monitor process create, writes, registry queries, net connections β€’ Create rules/watchlist for known bad behavior –Mimikatz-->company_name:*gentilkiwi* –FileVault Encryption Disabled -->process_name:fdesetup cmdline:disable –Unsigned JAR exec-->process_name:*.jar digsig_result: (digsig_result:"Unsigned") –OSX dump user hashes-->process_name:dscl cmdline:ShadowHashData
  • 43. Host Protections β€’ Sysmon β€’ https://www.root9b.com/sites/default/files/whitepapers/R9B_blog_005_white paper_01.pdf β€’ https://jon.glass/tag/sysinternals/ β€’ http://www.darkoperator.com/blog/2014/8/8/sysinternals-sysmon β€’ https://www.bsk-consulting.de/2015/03/21/detect-system-file-manipulations- with-sysinternals-sysmon/ β€’ https://www.firemon.com/enhance-windows-anomaly-detection-sysmon/
  • 48. Host Protections OSX Patch Management - Simian β€’ β€œSimian is an enterprise-class Mac OS X software deployment solution.” β€’ Allows you to push munki updates β€’ Free / OSS β€’ Runs on google cloud β€’ Project: https://github.com/google/simian
  • 49. Host Protections Why do we bring this up? β€’ Some people aren’t aware you can perform free OSX patch management β€’ There are a lot of OSX developer shops without an β€œenterprise budget” β€’ Patch management is a no-brainer and security 101
  • 51. Host Protections Simian Consists of 2 parts: β€’ Client – Private and Public SSL Keys used to authenticate – Configuration unique per OSX client β€’ Web Application/Server – Runs on Google Cloud – Keep in mind its free but… not for long (eventually costs a little for storage) Takes about a week to learn and get setup
  • 52. Host Protections Web Application used to Manage Updates
  • 54. Host Protections Simian Recap: β€’ Learning curve is moderately difficult IMO β€’ Free-ish (eventually storage costs but still very minimal) β€’ Useful for patch updates and monitoring clients systems for low disk space, uptime, etc.
  • 55. Production Protection (Infra) Protecting and monitoring production environments (AWS)
  • 56. My AWS Goals β€’ Harden – Make it difficult to reach your AWS environment β€’ Monitor – If your AWS environment is breached, you need to know and alert yourselves β€’ Restore – Have the ability to reconstruct data/configs after a β€œhack”
  • 57. AWS’s Plan β€’ Took the AWS Security Fundamentals Course and… – Fortunately, our strategy lines up with AWS recommendations – You are responsible for leveraging the tools AWS provides (financially) – Your configuration… that is on you – https://aws.amazon.com/training/course-descriptions/security- fundamentals/
  • 58. AWS Hardening Basics Making it difficult (for attackers) to reach our environment
  • 59. Hardening Checklist 1. Don’t Use The Root Account! 2. Disable Access Keys for Root Account 3. Multi-Factor Authentication 4. API + MFA 5. Strong Password Policy
  • 60. Don’t Use Root Account β€’ Every AWS env has a root account, only necessary to use for very specific circumstances β€’ When these circumstances arise, notify your team that the account will be used β€’ We will discuss why this is important when we talk about CloudWatch metrics
  • 61. Disable/Delete Root Account Access Keys β€’ Just delete them if they exist – Disable the access keys in the event you are unable to delete them completely for some reason β€’ Make sure your admins have a (verbal/written) policy that states β€œwe don’t create access keys for the root account”
  • 62. MFA β€’ If credentials are stolen or guessed, we want a second layer of protection β€’ You can use apps or hardware to do this – Google Authenticator (Apps) – Gemalto (Hardware) β€’ Find the full list of MFA devices here: https://aws.amazon.com/iam/details/mfa/ β€’ This is so ridiculously easy to do, everyone should do it
  • 63. MFA See the published slide deck for step by step instructions
  • 64. MFA β€’ At this point, it's worth mentioning that non- administrators or those without IAM privileges cannot enable MFA on their own account β€’ Why is this a problem? Well, they need to be able to enable MFA on their own device… not the administrator’s β€’ Fortunately, we have a solution!
  • 65. MFA
  • 66. MFA β€’ Okay so that wasn’t the easiest to read, so here is the link: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_c redentials_delegate-permissions_examples.html#creds- policies-mfa-console β€’ Basically this IAM policy allows a user to manage their *OWN* MFA device
  • 67. MFA (for Root Account) β€’ Need a shared MFA for root? TOTP! β€’ Recommend using something like 1password for teams, can share the TOTP code: https://support.1password.com/guides/mac/totp.html https://www.youtube.com/watch?v=eZyb-ArMK9g
  • 68. API + MFA β€’ You have the ability to place a restriction where resources can only be interacted with if the user has authenticated with MFA β€’ This helps prevent (ab)use should someone steal access keys or credentials
  • 69. API + MFA β€’ This entry enforces MFA for Web/API β€’ Do this for Admin & Power-User groups at a minimum
  • 70. API + MFA β€’ Truth be told, doing this can be painful at first β€’ Things that used to work, might not (via the API) β€’ Fortunately, we have some answers for you β€’ Firstly, let’s discuss STS or SecurityToken Service
  • 71. API + MFA β€’ Leverage STS in order to interact with the AWS API should this MFA restriction be placed on resources (and it should ☺ ) β€’ Example of using STS: https://gist.github.com/cktricky/127be4e431563a986f0f
  • 72. API + MFA Output of script
  • 73. API + MFA Use the creds to leverage tools like ec2-api-tools (-O <access key id>–W <secret> and –T <session token>)
  • 74. API + MFA And in case you don’t like Ruby… https://github.com/jimbrowne/aws-sts-helpers
  • 75. API + MFA β€’ ElasticBeanstalk does not work with STS. Le Terrible. β€’ However, there is a workaround, use CodePipeline β€’ Very simple process to setup but only works with: – GitHub – AWS CodeCommit – Amazon S3
  • 76. Password Policy β€’ Password policies are important because historically people do not choose complex passwords β€’ MFA should help, but we’re talking about a layered approach β€’ Again, making our AWS environment harder to reach
  • 78. Hardening Recap β€’ Make credentials hard to guess β€’ If guessed or stolen, we still have MFA β€’ Remember MFA only protects against the web and NOT the API… unless you change your policies and use STS β€’ Root account is King, protect your King
  • 79. Hardening Recap β€’ Things we did not (and won’t discuss) – S3 bucket policies – Security Group configurations – SSH Key Management – Encrypting Data (Volumes, S3 buckets) β€’ Trusted Advisor – Use it, because it catches a lot of β€œlow hanging fruit” style issues
  • 80. Hardening Recap β€’ Links to resources that discuss the items we’re not covering: – https://d0.awsstatic.com/whitepapers/compliance/AWS_Auditing_Securi ty_Checklist.pdf – http://aws-de-media.s3.amazonaws.com/images/Produktblaetter/AWS- Security-Check-List_eng.pdf – http://www.slideshare.net/AmazonWebServices/masterclass-advanced- security-best-practices β€’ Frankly you can’t throw a rock without hitting some basic info regarding AWS Security Checklists
  • 82. AWS Monitoring β€’ Assuming hardening (prevention) has failed, how would we know? β€’ Luckily, AWS provides several services which alert to anomalies β€’ We will walk through examples of using these services, but ultimately decide what is right for you β€’ Fair warning, some of these services will provide a lot of noise
  • 83. AWS Monitoring 4 important services: β€’ CloudTrail – Logs β€’ SNS – Notifications β€’ Config – Alerts for modifications & noncompliance β€’ CloudWatch – Alerts for specific types of behavior
  • 87. AWS Monitoring (CloudTrail) β€’ CloudTrail is primarily used for log collection β€’ Other services like CloudWatch, for example, use those logs to filter relevant data
  • 88. AWS Monitoring (CloudTrail) Pretty easy, first turn it on..
  • 90. AWS Monitoring (CloudTrail) Allow the creation of an IAM role by CloudTrail
  • 91. AWS Monitoring (CloudTrail) β€’ At this point you have cloudtrail enabled β€’ Next step, BEFORE moving to CloudWatch or Config, is configuring SNS topics
  • 93. AWS Monitoring (SNS) β€’ Fantastic offering, <3 it – Examples of ways to be notified by SNS β€’ SMS β€’ Email β€’ JSON Post to your Application’s API endpoint
  • 94. AWS Monitoring (SNS) β€’ Receive SMS/Email/Slack notifications for important events β€’ ^ This is so you get immediate notifications β€’ You can have multiple subscribers, I’d suggest you use that functionality β€’ Basic gist? Receive immediate updates for things you want to see… immediately ☺
  • 97. AWS Monitoring (SNS) Create SMS (or whatever, but in this case, SMS)
  • 98. AWS Monitoring (SNS) Example of creating email subscription… bottom line you can have multiple ways of notifying people
  • 100. AWS Monitoring (Config) β€’ Config: – AWS resource inventory, configuration history, and configuration change notifications – Can either design custom Config rules or use managed (pre-packaged) AWS Config rules – Discovery -Change Management – Compliance -Incident Response
  • 101. AWS Monitoring (Config) β€’ Pre-packaged β€œManaged” AWS Rules – CLOUD_TRAIL_ENABLED – EIP_ATTACHED – ENCRYPTED_VOLUMES – INCOMING_SSH_DISABLED – INSTANCES_IN_VPC – REQUIRED_TAGS – RESTRICTED_INCOMING_TRAFFIC https://docs.aws.amazon.com/config/latest/developerguide/evaluate-config_use-managed-rules.html
  • 102. AWS Monitoring (Config) β€’ Examples of things you can have alerts set for: – Change in Firewall (Security Group) ports – Changes in VPC – Any change… at all
  • 103. AWS Monitoring (Config) Go to the Config service and choose resources to track
  • 104. AWS Monitoring (Config) Or choose to track everything
  • 105. AWS Monitoring (Config) Create a bucket, create an SNS topic (…we’ll discuss next)
  • 106. AWS Monitoring (Config) Allow the role to be created and you’re all set!
  • 108. AWS Monitoring (CloudWatch) β€’ We can be very particular here about what it is we want to see β€’ Some very interesting things you can monitor β€’ Some examples: – Billing Alerts (Important for detection of abuse or mistakes) – Track Root Account Usage – Failed login attempts
  • 110. AWS Monitoring (CloudWatch - Billing) β€’ Used to prevent abuse or mistakes from costing your organization money β€’ Analyze and approximate your monthly spend β€’ Configure via CloudWatch β€’ Use SNS for instantaneous alerting
  • 111. AWS Monitoring (CloudWatch - Billing) Navigate to billing & cost management; enable billing alerts
  • 112. AWS Monitoring (CloudWatch - Billing) Create an SNS topic
  • 113. AWS Monitoring (CloudWatch - Billing) Subscribe to Topic
  • 114. AWS Monitoring (CloudWatch - Billing) Navigate to CloudWatch -> Metrics -> Billing
  • 115. AWS Monitoring (CloudWatch - Billing) Choose USD/Estimate Charges -> Create Alarm
  • 116. AWS Monitoring (CloudWatch - Billing) Set price point, SNS topic, and create alarm
  • 117. AWS Monitoring (CloudWatch - Billing) Exact steps to enable can be found here: http://docs.aws.amazon.com/awsaccountbilling/latest/about v2/free-tier-alarms.html
  • 119. AWS Monitoring (CloudWatch – Root Login) β€’ Remember how I said don’t use the Root account routinely? β€’ BUT… if this account is used, you should know about it β€’ This is the reason you’ll want to notify others (who receive SNS alerts) of the fact you are about to use the account
  • 120. AWS Monitoring (CloudWatch – Root Login) Choose log group, create metric
  • 121. AWS Monitoring (CloudWatch – Root Login) Define Logs Metric Filter
  • 122. AWS Monitoring (CloudWatch – Root Login) Assign/Create Filter
  • 123. AWS Monitoring (CloudWatch – Root Login) Click β€œCreate Alarm”
  • 124. AWS Monitoring (CloudWatch – Root Login) Define Alarm and you’re good…
  • 125. AWS Monitoring (CloudWatch – Root Login) Exact steps (with pics) exist here: https://blogs.aws.amazon.com/security/post/Tx3PSPQSN8 374D/How-to-Receive-Notifications-When-Your-AWS- Account-s-Root-Access-Keys-Are-Used
  • 127. AWS Monitoring (CloudWatch – Failed Logins) β€’ In the event someone is trying to break in, let’s alert ourselves to this! β€’ Failed logins typically suggest either someone forgot their password or… someone is trying to guess yours
  • 128. AWS Monitoring (CloudWatch – Failed Logins) β€’ In the interest of time… the steps are pretty much the same as the root login alarm β€’ The Regex Filter however, is different
  • 129. AWS Monitoring (CloudWatch – Failed Logins) Relevant filter pattern
  • 130. AWS Monitoring (CloudWatch – Failed Logins) β€’ Exact steps exist here: http://docs.aws.amazon.com/awscloudtrail/la test/userguide/cloudwatch-alarms-for- cloudtrail.html#cloudwatch-alarms-for- cloudtrail-signin
  • 132. AWS Monitoring (Unauthorized Activity) β€’ Remember the aws-interrogate tool? β€’ This alarm is the antidote β€’ Alerts us when someone is trying to access something in AWS, and does not have permissions
  • 133. AWS Monitoring (Unauthorized Activity) β€’ Again, in the interest of time, steps are same as root login β€’ Regex is of course, different
  • 134. AWS Monitoring (Unauthorized Activity) Set up regular expression
  • 135. AWS Monitoring (Unauthorized Activity) What happens when we run interrogate
  • 136. AWS Monitoring (Unauthorized Activity) The result of doing that is a nice nifty email to the engineering & security team
  • 137. AWS Monitoring (CloudWatch) – Filter Patterns β€’ If you’d like to create your own custom filter patterns, here is a resource for that: http://docs.aws.amazon.com/AmazonCloudWatch/latest/De veloperGuide/FilterAndPatternSyntax.html
  • 139. AWS + Splunk β€’ Splunk is a pretty great resource for monitoring activity – Two separate plugins: β€’ Splunk App for AWS – https://splunkbase.splunk.com/app/1274/ β€’ Splunk Add-On – https://splunkbase.splunk.com/app/1876/
  • 140. AWS + Splunk β€’ Examples of things you can view: – Billing – Topology – Usage – IAM Activity – SSH Key Pair Activity – User Activity – Network ACL(s) – VPC Activity – and a lot more…
  • 144. AWS + Splunk β€’ Splunk will need an AWS account in order to retrieve data β€’ Create account(s) for Splunk, grab the necessary permission policy from here: http://docs.splunk.com/Documentation/AddOns/released/A WS/ConfigureAWSpermissions
  • 145. AWS + Splunk Configure AWS App for Splunk, add account(s), configure each input accordingly:
  • 146. AWS + Splunk β€’To view things like IAM Activity… –Subscribe to a cloudtrail log via SNS –Utilize SQS and subscribe SQS to an SNS Topic
  • 147. Monitoring Recap β€’ Alert yourself when things change β€’ This will get noisy, find a way to filter that which is important – If it’s a high risk event, send an SMS/Slack/Email blast β€’ At a minimum, alert yourself when odd things occur… like: – Billing increases past your normal spend – When somebody authenticates as Root – When someone has a login failure
  • 148. Monitoring Recap β€’ Interesting Quora thread: – https://www.quora.com/My-AWS-account-was-hacked-and-I-have-a-50-000-bill-how-can-I- reduce-the-amount-I-need-to-pay β€’ Highlights from the article: – AWS has β€œa review board of sorts” to determine if you should be refunded – Bots are scouring GitHub searching for exposed access keys – One of the more AWS-seasoned responders mentioned doing part of what we discussed here today to avoid it – A decent number of the people posting on this thread said β€œYes, happened to me too”
  • 149. AWS Restoration & Recovery Plan to fail, just don’t fail to plan
  • 150. AWS Restoration & Recovery β€’ Do not USE AWS TO BACKUP YOUR AWS β€’ Offsite backups (meaning, off AWS site) β€’ Common things to back-up: – Databases/ Snapshots – S3 Buckets – EBS Volumes – CloudFormation Templates
  • 151. AWS Restoration & Recovery β€’ Resources: – http://stackoverflow.com/questions/17087542/backup- solutions-for-aws-ec2-instances – https://github.com/Scalr/installer-ng – http://www.n2ws.com/blog/3-ways-ec2-windows- backup-and-recovery.html
  • 152. AWS Incident Response Plan to fail, just don’t fail to plan
  • 153. AWS Incident Response β€’ Could be its own talk β€’ Scout 2 -- https://github.com/nccgroup/Scout2 β€’Andrew Krug & Alex McCormack – Hardening AWS Environments and Automating Incident Response – https://www.youtube.com/watch?v=cmEUxxYFjK8