Microservices Security: dos and don'ts

Microservices Security:
Dos and Don'ts
Stefano Di Paola, CTO & Chief Scientist @ Minded Security
July 2018 Summit

$ whoami
• Research
– Bug Hunter & Sec Research (Pdf
UXSS, Flash Security, HPP, JS
Security DOMinator/BlueClosure)
– Software Security Since ~'99
– CTO @ Minded Security
– Chief Scientist @ Minded Security
2

Agenda
• Microservices Vs Monolithic Apps
• Security in Microservice Architectures
• Common Services: Security Exposed
– Authorization Flow
– Microservice Events
– Requestor Microservice
– REST Aggregator/Forwarder
– Serverless services.
– PDF Generator
3

Goal
• Talk about Security in Microservices
Architectures
• Give insights about some of the most
interesting issues found in the last years
while testing the security of Multilayered
Microservices Architectures and how they
were fxed
• Will not talk about AWS misconfgurations
(too much to tell:)
4

Monolithic vs Microservices
•
5

Why Microservices?
• Scalability
• Maintainability
• Easy Refactor (No language constraint)
• Agile SDLC
• Fast+Continuous Deploy
6

Monolithic vs Microservices
•
7

Rings a bell?
Procedural vs Object Oriented
8

Scalability
9
●
X-axis scaling: scaling an application by running
clones behind a load balancer.
●
Y-axis scaling: splits the application into multiple,
different services.
●
The microservice architecture is an application of Y-
axis.
– Each service is responsible for one or more
closely related functions.
●
Two ways of decomposing the application into services.
– By Action: Verb-based decomposition and
define services that implement a single use case
such as checkout.
– By Context: decompose the application by noun
and create services responsible for all operations
related to a particular entity such as customer
management.
●
Mixed Action+Context works too.

Maintainability+Easy Refactor
• A component is a unit of software that is
independently replaceable and upgradeable.
• Services as components
– because services are independently deployable
• A service could be deployed on
– a fully controlled server on a container
– serverless (AWS Lambda, Google/Azure cloud
functions…)
– .. or in house of course
1
0

Communication between MS
• REST/Queues
• Remote calls are more
expensive than in-process
calls.
• Remote APIs need to be
coarser-grained
• Change the allocation of
responsibilities between
components.
1
1

Asynchronicity
• No more monolithic app means no more
single thread.
• Each Microservice can be considered as
a separate process
• If a process takes too much to fullfll its
duty it’ll block every one in the stack.
• Microservices must be Asynchronous as
much as possible
1
2

Common Pattern: Multi-API GW
15

17
A Case Study of Microservice Security

Agenda
– PDF Generator
1
8

Auth and Authz in MS
1
9
REST REQUEST
'http://localhost:8003/products/the_odyssey'
REST REQUEST
'http://localhost:8003/rest/login'

2
0
REST REQUEST
REST REQUEST
'http://localhost:8003/rest/login'
RESPONSE:
JWT:
{“user”:”stefano”,”id”: 22}.[CRYPTOGRAFICALLYSIGNED]

2
1
REST REQUEST
REST REQUEST
'http://localhost:8003/rest/order/54252'
JWT: {“user”:”stefano”,”id”: 22}.
[CRYPTOGRAFICALLYSIGNED]
Is Logged In?

2
2
Is Logged In?YES!
REST REQUEST

2
3
OKAY! GO ON!REST REQUEST

2
4
REST REQUEST
REST REQUEST
OKAY! GO ON!

Authorization != Authentication
• Microservices must be aware at some point
who can do/has access to what.
• Design decisions must be made and
implemented.
• Defense in Depth is the most appreciated:
– Implement a Identity Management System
– Each MS will request if token X is allowed to
execute the service.
• Each MS is responsible for the data it manages
2
5

The Fix
2
6
Each MS Should ask if User is allowed
to use the service

The Fix
2
7
Protect data from
Indirect Object Reference!

Agenda
– PDF Generator
2
8

Order Request
2
9
REST REQUEST
REST REQUEST
'http://localhost:8003/rest/products/the_odyssey'

Order Request
3
0
REST REQUEST
INTERNAL REST
'http://localhost:8003/product/the_odyssey'

Order Request
3
1
REST REQUEST
INTERNAL REST

Order Request
3
2
REST REQUEST
INTERNAL REST
FINAL RESPONSE
{
"id": "the_odyssey",
"title": "The Odyssey",
"passenger_capacity": 101,
"maximum_speed": 5,
"in_stock": 10
}

Order Request
3
3
REST REQUEST
LET’S PLACE AN ORDER!
POST http://localhost:8003/orders
{
"order_details": [
{
"product_id": "the_odyssey",
"quantity": 1
}
]
}

Order Request
3
4
REST REQUEST
{
"order_details": [
{
"quantity": 1
}
]
}
DO WE HAVE ANY
the_odyssey?

Order Request
3
5
REST REQUEST
{
"order_details": [
{
"quantity": 1
}
]
}
DO WE HAVE ANY
the_odyssey?
YES!

Order Request
3
6
REST REQUEST
{
"order_details": [
{
"quantity": 1
}
]
}
OKAY!
-1 FOR the_odyssey!

Order Request
3
7
REST REQUEST
{
"order_details": [
{
"quantity": 1
}
]
}
RESPONSE : {"order_id": 2131}

39
Race Condition!
Orders MS
Products
MS

40
Race Condition Schematics
Orders MS Products MSClient Request
1
1
Is the_odyssey in?
Yes!
1
1
-1 the_odyssey
The_odyssey: #1
The_odyssey: #01
Order Placed!

41
Race Condition Schematics
Orders MS Products MSClient Request
The_odyssey: #1
The_odyssey: #0
1
1
Is the_odyssey in?
Yes!
1
1
-1 the_odyssey
1
Order Placed!
2
2
Is the_odyssey in?
Yes!
2
2
-1 the_odyssey
2
Order Placed!
The_odyssey: #-1

The Fix
4
2
THE REST ENDPOINT MUST PERFORM
ATOMIC OPERATION WHEN MULTIPLE
ASYNCRONOUS MS ARE INVOLVED

Agenda
– PDF Generator
4
3

Requestor MicroService
4
4
REST REQUEST
PUT 'http://localhost:8003/rest/products/the_odyssey'
{“image”:”http://www.imghost.com/image.png”}
Http Request
To retrieve the URL

Requestor MicroService
4
5
REST REQUEST
PUT 'http://localhost:8003/rest/products/the_odyssey'
{“image”:”http://www.imghost.com/image.png”}
Http Request
To retrieve the URL

Requesting Arbitrary URL?
• Feature found several times during the years
• Sometimes correctly implemented.
• Sometimes not.
• Problems: Arbitrary requests to any internal
node.
• It might be called
SSRF By design
4
6

The Fix
• Containerize the service
• Deploy the container outside the other
sensitive services network
• Hardenize the container!
• Do not rely on DNS/IP black lists. Easy
to bypass! (at least keep the 1st
resolution!)
• Block requests to 127.0.0.1/8!!
4
9

Agenda
– PDF Generator
5
0

Rest Gateway
5
1
/user/:id
We discover that PRIVATE SERVER has a
undocumented endpoint

Rest Gateway
5
2
REST REQUEST
http://localhost:8003/rest/user/23
/user/:id

Rest Gateway
5
3
REST REQUEST
http://localhost:8003/rest/user/23
/user/:id
I knew it!
Not Externally Mapped!
You’re not going to pown me!

Rest Gateway
5
4
REST REQUEST
http://localhost:8003/rest/products/%252e%252e%252fuser%252f1
/user/:id

The Attack
• Double Encoding
• %2e => .
• %2f => /
• %25 => %
%252e => %2e => .
%252f => %2f => /
5
5
http://hostname:8003/rest/products/%252e%252e%252fuser%252f1

The Attack
●
API GATEWAY SEES
id= “%2e%2e%2fuser%2f1”
and sends http://privateserver/products/%2e%2e%2fuser
%2f1
But HttpClient/private server normalizes the URL to :
●
http://privateserver/user/1
5
6
PRIVATE SERVER REST APP SEES: /user/1

The Fix
• Apply Defense in depth
– Each MS should validate input data
– Each MS should encode data according to
the context when it’s sent to another layer
– Separate services based on endpoints
sematic groups.
5
7

Agenda
– Serverless services
– PDF Generator
5
8

The Threat
• In recent months, several articles and blog posts
exposed how malicious attackers are abusing
cloud environments in order to infect them with
crypto-mining malware.
• February 2018, cybersecurity firm Redlock
reported that hackers had secretly infiltrated
public cloud environments and were using the
compute instances to mine cryptocurrencies.
• Cloud Functions fit very well here! (AWS
Lamba, Google Cloud Functions etc…)
• Hardened Containers as well!!
6
0

The Threat
6
1
• One vulnerability is
enough RCE/Code
Injection.
• When you realize you’ve
been attacked it’s
probably too late:
Several $$ have
already been billed.
Eg:
AWS Lambda Scaling:
AWS Lambda will dynamically scale capacity in
response to increased trafic, subject to your
account's Account Level Concurrent Execution Limit.
To handle any burst in trafic, Lambda will
immediately increase your concurrently executing
functions by a predetermined amount, dependent on
which region it's executed
By default, the concurrent execution limit is enforced
against the sum of the concurrent executions of all
functions. The shared concurrent execution pool is
referred to as the unreserved concurrency allocation.
The default is set to 1,000.

Agenda
– PDF Generator
6
2

The Feature
• Export orders in PDF.
• Application is a Single Page Application using JS
Framework.
• The idea is to use
– WebKitToPDF
– a headless Chrome
– Custom Electron with Webview
• To export the rendered html as pdf.
EASY PEASY!
6
4

The Feature
• Create a local web page using USER data
• Save it as PDF
• Send it back to the USER
• Problem:
– How to build the page?
– How user data is imported?
6
5

The Feature
• POST /create/pdf
htmlData=<body>...</body>
●
Is that a Cross Site Scripting?
●
Yeah but it’s a self XSS, no impact right?
6
6

A Simple Attack
<iframe src=”http://internalHost/”></iframe>
6
7

The Attack
• It’s like having physical access to a browser on a
machine in hosts private network!
• In some case attacker might have access to
Filesystem (I.e read files in host FS)
• Attacker could also execute JavaScript
• ..and even implant a(nother) Cryptominer!
6
8

The Fix
• From a Browser perspective there’s no easy fix
but there’s a set of mitigations, too long to explain
but:
– Set browser to Offline (partially bypasssable)
– Disable JavaScript (bypassable)
– Intercept and block all request (partially
bypassable)
– Close process as soon as possible
• Mostly hardenize the container!!
6
9

Last question:
• How do we deploy?
• Infrastructure as code
• In the repository with
the rest of the code.
• Where are the access
keys and passwords?
• Is the repository
private? To whom?
7
0

Conclusions
• Microservices introduce new (old)
unexpected security scenarios
• Developers and System architects must
work together to generate ad hoc
containers to mitigate by design dangerous
features
• Complexity of the fows requires careful
design in grouping microservices together
• Never underestimate attackers
• Apply defense in depth!!!!
7
1

Questions?
Mail:
stefano.dipaola@mindedsecurity.com
Mobile: +39 3209495590
Global Corporate Site:
http://www.mindedsecurity.com
Blog: http://blog.mindedsecurity.com
Twitter: http://www.twitter.com/mindedsecurity
YouTube:
http://www.youtube.com/user/mindedsecurity
Thanks!

Microservices Security: dos and don'ts

More Related Content

What's hot (20)

Similar to Microservices Security: dos and don'ts (20)

More from Minded Security (15)

Recently uploaded (20)

Microservices Security: dos and don'ts