F5 ASM v12 DDoS best practices

BIG IP ASM V12
DDOS PROFILE
Lior Rotkovitch, NPI
ASM, L7 DDoS & Analytics
Global Service Tech Summit, Seattle
Sep, 2015, v3
lior@f5.com

© F5 Networks, Inc 2
ASM – DDoS Profile
DDOS - HTTP FLOODS ATTACKS
• From single IP to single URL
• From multiple IP’s to single fixed URL
• From multiple IP’s to multiple fixed URL’s
• From multiple IP’s to multiple random URL’s
• From multiple IP’s from a specific country
• Fine Tune your Thresholds & Reporting
DDOS - BOTS
• Simple bots
• Impersonating Bots
• Bots with cookies & JS capabilities
• Bots acting as full browser
• Reporting

HTTP Floods facts:
Servers Database
Hacktivism
Google Web Bot
Unidentified User
User
Source IP‘s
Users Or Bots
Web Site
Web Bot
• Legitimate Layer 7 requests
• Asking a web page thousands of time instead of one (few) times
• Exhausting backend servers resources: memory, CPU, Disk etc
• Relatively easy to execute with simple tools
• Not easy to detect the offending source nor to prevent it
• Wrong identification will prevent valid users from accessing the site (false positive )

HTTP Floods types
Hacktivism
Google Web Bot
Unidentified User
User
Source IP‘s
Users Or Bots
Web Bot
Requests increase from or/and to URL’s inside the web site
• From single IP to single URL
• From multiple IP’s to single fixed URL
• From multiple IP’s originating from a specific country
Web Site
Servers Database

ASM Detection & Mitigation concept - HTTP Floods
Latency App URL’s &
objects
Hacktivism
Google Web Bot
Unidentified User
User
RPSSource IP‘s
Users Or Bots
Web Bot
Source IP’s
ASM process:
1. Monitoring entities: RPS Latency IP’s URL’s
2. Detecting Increase
3. Activating Mitigation
Web Site
Servers Database

ASM Detection & Mitigation concept – DoS Profile
Location: Security ›› DoS Protection ›› DoS Profiles ›› dos

TPS Based Detection: Transaction Per Second based detection
and mitigation
Client: Hey server, can I get the web page ?
ASM: no, you are sending too many requests.
You will have to
• Answer CSID
• Answer CAPTCHA
• Be Rate Limited / Blocked
Server

TPS Based Detection
Monitoring Request Per Second increase form source IP, Geo, URL, Site Wide.
Then apply one of the mitigation policies: CSID, CAPTCHA, Rate limit

TPS Based Detection
1. BY Source IP (Detection & Mitigation Polices)
2. Mitigation polices:
a) Client Side Integrity Defense
b) CAPTCHA challenge
c) Request Blocking
3. By Geolocation (Detection & Mitigation Polices)
4. By URL (Detection & Mitigation Polices)
5. By Site Wide (Detection & Mitigation Polices)
6. Prevention Duration
1
a
b
c
3
4
5
2
6

By Source IP: Detection Criteria
Detection
Detection: thresholds for determining DDoS attack - by source IP increase
Mitigation: which mitigation will apply on the offending source IP
Mitigation

Ratio thresholds - measuring ratio with two time interval:
• Long (History Interval): Measure the last 1 hour RPS averages every 10 seconds
• Short (Detection Interval): Measure the last 10 seconds RPS averages every 10 seconds
Detection – Ratio

TPS increased by: ((370 - 50) /50)*100 = 640%
640% > 500% = True
Detection – Ratio
Long (History Interval): 50 TPS
Short (Detection Interval): 370 TPS
Example:

TPS increased by % AND minimum fix number of transactions
640% AND 40 = True
Detection – Ratio
Example:
At least X Transactions:
A minimum condition to
prevent false positive
increase (source IP
starts browsing the site
and goes from 0 to 30
RPS)

(TPS increased by % AND minimum fix number of transactions) OR TPS reached
640% AND 40 OR 200 = True
Detection – Ratio
Fixed
Example:
TPS reached:
Ratio thresholds
OR’ed with fixed TPS
or

TPS Based Detection
c) Request Blocking
1
a
b
c
3
4
5
2
6

Client Side Integrity Defense – Concept
User
Web Bot
ASM: no, you are sending too many
requests. Are you a browser ?
if a browser:
Yes, I’m a browser
If a bot:
*^lkjdfg@#$
ASM: ok, you are allowed. Here is the web
page you asked for.
ASM: Bye Bye – Blocked
Server

• Checking JavaScript capabilities
• A client is considered legitimate if it meets the following criteria:
• The client support JavaScript
• The client support HTTP cookies
• The client should calculate a challenge inside the JS
• If satisfied = legitimate client that can access the site
Client Side Integrity Defense – Concept

Client Side Integrity Defense - Flow
User Browser DoS Profile App
First main page access
HTTP Request (no cookie)
Computational challenge
Solve challenge/
set cookie with time stamp
HTTP Request (cookie) Reconstruct request
Original HTTP Request
HTTP Response (main page)
More object requests (cookie)
Validate cookie: format & time stamp
More object requests
More responses
More responsesDeliver page
• This is the flow and timeline
of events.
• Transparent to the user,
done under the hood
• Note that request is held at
the ASM and not arriving the
app until checks are
satisfied
• Not all checks are described
here, some are internal IP.
Send JS test

Client Side Integrity Defense –JavaScript sample
• The JS is obfuscated
• From user perceptive this
is transparent action.

• If no reply – No problem for us
• If didn’t solve the challenge but still sending request – Block (RST)
• If did solve the challenge but:
• Cookie is wrong format – Block (RST)
• Time stamp expired – Block (RST)
• If client access a resource (image) without getting the cookie first – Block (RST)
Client Side Integrity Defense – Mitigation summary

TPS Based Detection
c) Request Blocking
1
a
b
c
3
4
5
2
6

CAPTCHA Challenge - Concept
User
Web Bot
ASM: no, you are sending too many
requests. Please answer this CAPTCHA
challenge, show me your human !:
If a user:
OK, I answered
If none user:
Ha ? *^lkjdfg@#$
ASM: ok, you are allowed. Here is the web
page you asked for.
ASM: Bye Bye – Block him dood !
Server

CAPTCHA Challenge
Ultimate solution for identifying human or bot
Send challenge to every IP that reached IP detection criteria thresholds
To CAPTCHA or not to CAPTCHA ?
Some argues that CAPTCHA is not a good usability because an innocent user gets
CAPTCHA and he will not know why. So, remember that a valid user should pass
browser tests. i.e. if a user is blocked (or gets a CAPTCHA) there is a reason and
maybe he is not innocent (infected ?) .
“Completely Automated Public Turing test to tell
Computers and Humans Apart”

CAPTCHA – customize response
• Can be customize to the web site look and feel colors via css
• Failure Response page is served if the first attempted fails

CAPTCHA Challenge - Flow
Request mypage.php
GET /mypage.php (no cookie)
CAPTCHA HTML +JS response
Cookie with time stamp
Solve CAPTCHA
CAPTCHA rendered
Submit CAPTCHA
solution
GET /mypage.php + CAPTCHA
cookie
Verify CAPTCHA solution
Validate cookie
GET /mypage.php
HTML of mypage.phpHTML of mypage.php
mypage.php
rendered
Send CAPTCHA
• While the system is still in a
state of attack the offending
source will be presented with
another CAPTCHA every 5 min.
• Same as CSID, request is held
at the ASM until CAPTCHA is
solved

• If didn’t submit the challenge - no request DOSing us
• If didn’t solve the challenge but still sending us attacks – Blocked
• If did solve the challenge but:
• Cookie is wrong format – RST
• Time stamp expired 5 min– RST
CAPTCHA – mitigation summary

TPS Based Detection
c) Request Blocking
1
a
b
c
3
4
5
2
6

Request Blocking / Rate limit
ASM: no, you are sending too many requests.
I’m limiting your requests sending rate
While CSID and CAPTCHA try to understand who is the offending source (bots or browsers)
request limiting is indifferent to the “identity” and limits the offending sources.

Request Blocking
Request Blocking:
• Blocking: block all IP’s from the offending source – if a source IP reached thresholds I don’t
want him on my site at this point
• Rate Limit: limit the amount of allowed request from the offending source – if reached
thresholds I can sustain only some of the traffic at this point

Request Blocking – Mitigation Summary
• Block all – blocking all traffic from the offending
source (i.e. I don’t want to see any more traffic from
this source)
• Rate Limit – rate limit the offending source
Example
If long was 50 TPS And increase in short is 150 TPS
Rate limit to 50 TPS
Rate limit will limit to long (history) TPS rate

TPS based: by source IP – Summary
Client Side Integrity Check
CAPTCHA Challenge
Request Blocking
• Measuring source IP increase
• All source IP’s that reached the thresholds will be presented with the enabled mitigation
• If still increasing , fall back according to the order in the GUI (switching mitigation)

TPS Based Detection
c) Request Blocking
1
a
b
c
3
4
5
2
6

HTTP Floods – Geolocation detection and Mitigation
Hacktivism
Google Web Bot
Unidentified User
User
Source IP‘s
Users Or Bots
Web Bot
Servers Database
Web Site
http floods type: From multiple source
IP’s originating from a specific country

• Geolocation – Relative to the whole traffic of the site:
500 % request increase of the whole site from a specific country
AND
At least 10 % of the whole site traffic
Geolocation - Detection

Geolocation – Mitigation
• Client Side Integrity Check
• CAPTCHA Challenge
• Request Blocking
All clients requests arriving from the specific country will be presented with mitigation:
(note that blocking will block
all users from this country)

Geolocation – Black n White listing
• Allows access to the web site regardless of
geolocation detection criteria thresholds only
i.e. other thresholds still apply
• Specifies the countries that the system always blocks
whenever the system is in a state of DDoS detection.
• Done regardless of the thresholds set in the DDoS
profile

TPS Based Detection
c) Request Blocking
1
a
b
c
3
4
5
2
6

HTTP Floods – URL Detection and Mitigation
App URL’s &
objects
Hacktivism
Google Web Bot
Unidentified User
User
RPSSource IP‘s
Users Or Bots
Web Bot
Source IP’s
http://site.com/sell.php
Servers Database
Web Site
• Measuring requests increase on a URL
• Floods types:

TPS increase by* AND at least X TPS **
OR TPS reached
URL Detection Criteria
Collecting RPS on URL’s
Calculation:
*Ration of long and short
**Minimum TPS thresholds for detection

URL Detection Criteria– Mitigation
• Request Blocking – Rate limit (No block all)
All clients that access the URL:

TPS Based Detection
c) Request Blocking
1
a
b
c
3
4
5
2
6

HTTP Floods – Site Wide Detection and Mitigation
App URL’s &
objects
Hacktivism
Google Web Bot
Unidentified User
User
RPSSource IP‘s
Users Or Bots
Web Bot
Source IP’s
Monitoring: all entities
Servers
Database
Web Site
Floods types:
• Cases where DDoS attack is under the radar

*TPS increase by AND Minimum TPS thresholds for detection
OR TPS reached
Site-Wide Detection Criteria
• Collecting RPS on the entire website (all entities – URL’s, IP’s)
• In some cases the floods will avoid thresholds for IP based or URL based.
• Site wide provide another layer of detection and prevention
Detection: Ratio
Fixed

Site-Wide Detection Criteria – Mitigation
• Request Blocking - only rate limit no blocking
All clients that access the site:
Prevention polices

TPS Based Detection
c) Request Blocking
1
a
b
c
3
4
5
2
6

Prevention duration
• Request Blocking
De escalate - start from the top
Every 7200 seconds
Escalate top down every 120 second if thresholds are still increasing

Stress Based detection
• Predictive Latency – predict how long it will take to
serve a new incoming request
Server: I’m fine, keep on sending them
ASM: Hey server, how many more requests
can you handle ?
I’m the server

Stress Based Detection and prevention concept
Client: Hey server , can I get the web page ?
ASM: mmm let me check. The Server can take
additional incoming requests. you are allowed
ASM: no, my backend latency is now too high
and you are sending too many request.
You will have to:
• Answer CSID or
• Answer CAPTCHA or
• Be rate limit
Client: Hey server, can I get web pages again now ?
I‘m the server
…….
…….
After a while

Stress Based – GUI
• Same concept as TPS
based: source IP, Geo, URL,
Site wide and their
mitigation policies.
• Addition condition of
backend latency. i.e. only
when the two conditions
reach thresholds, then apply
mitigation policy.
Note: Can work together (operate in
parallel) with TPS based and act as
layers of protection (e.g. TPS based
does only CSID in alert mode and
Stress based does request blocking in
case of latency increase)

Stress Based Detection & Mitigation
• Similar to TPS based,
Quiz yourself, what each item
means ?
1. By Source IP
a) CSID
b) CAPTCHA
c) Request Blocking
2. By Geolocation
3. By URL
4. Site Wide

Stress Based Detection – thresholds condition
Latency
threshold
exceeded?
TPS
threshold
exceeded?
AND
Then:
Activate
Mitigation Policy
• Mitigation Is activated when
two types of thresholds are
reached :
Latency thresholds
AND
TPS thresholds

Stress Based Detection – thresholds condition
• in order to apply a prevention policy, both TPS and
Latency thresholds must be exceeded, then the enabled
prevention policy is activated.
• Latency thresholds are not visible in the GUI, they are part
of automatic detection.
Example:
Automatic stress detection enters a
state of exceeding thresholds. This by
itself will not active the prevention.
Only when the TPS thresholds will
exceed then the prevention policy is
activated.
prevention
TPS thresholds
stress detection

TPS based VS Stress based
• Quick way to protect against DDoS. I’m in trouble
and I want to block now !
• Fixed number on the TPS reached is very easy
and useful. Also easy to detect offending sources
• Allows the option to activate the Mitigation only
when the backed experiencing latency AND RPS
increase (I only want to block when the attack is
causing backend latency)
• Provide Layers of defense and notify about backend
issues (not just DDoS)
Conclusion:
TPS based is quick while latency based
allows more granular approach

Heavy URL’s
Not all URL’s are equal
Some are more attractive than others

Heavy URL’s
• Heavy URL’s are URL’s that consume more processing resources
from the server
• Are good application DoS point - Even few requests can DoS the app
• Typical heavy URLs are search box, product ID’s
Heavy URL
Servers Database
http://site.com/serach.php?q=a
Ho wow, this will take a while…
……
Searching …
…..
hold on…
…..
Almost there….

Heavy URL’s concept
• Automatically measures latency on URL’s for 48 hours
and decide who is heavy
• When any URL based mitigation is active, the heavy URL’s
that was detected as heavy will also “get” the active
mitigation

Heavy URL’s concept
Heavy URL is another detection capability. Once it is reached the thresholds AND one of the By URL detection
thresholds are reached Then the URL’s that consider heavy URL’s will be applied with the active mitigation policy

Heavy URL’s configuration
Example: By URL TPS reached 1000 TPS and is currently applying CSID mitigation.
Heavy URL is enabled

Heavy URL’s Configuration
1. Automatic Detection - Automatically add URL
that will be considered as heavy
2. Manual Heavy URLs – Manually add URL that
will be considered as heavy
3. Ignored URL – Exclude those URL from
“heaviness”
4. Latency Threshold – Above this threshold ->
heavy URL
1
2
3
4

Heavy URL – Reporting
If search.php is defined as heavy and if index URL is currently being mitigated with CSID because it exceeded the
thresholds of URL reached (or fixed) then every source IP that is accessing search.php will also get the CSID check.
Security ›› Reporting ›› DoS ›› Application ›› URL Latencies
Example:

• Reporting first
• Fine Tune your thresholds
BeforeDDoS Attack
During DDoS Attack
Remember, security is a process

First rule of detection - AVR Reporting
• AVR graphs help you understand the site metrics:
Statistics ›› Analytics ›› Overview
Statistics ›› Analytics ›› Transactions ›› View by
• AVR graphs inside ASM tab:
Security ›› Reporting ›› DoS ›› Overview
Security ›› Reporting ›› DoS ›› Application ›› Transaction outcome
Know your web site metrics
• Sources
• IP’s
• URL’s
• Site Wide
• Geolocation
• RPS
• TPS
• Latency
L7 DDoS measurements

• Out of the box thresholds are good for most web sites
• Depending on the web site traffic fine tuning thresholds might be needed.
• Fine tuning thresholds can be divided into:
• Before DDoS Attack
• During DDoS Attack
Why Fine Tune Thresholds ?
Good for me ??

Process:
Pre requisite: Enable DDoS Profile on the desired virtual
1) White list IP’s, geolocations countries, URL’s (admin) etc
2) Get visibility with transparent mode – write down metrics*
3) Test and decide which prevention will apply thresholds exceeds (TPS bases/
Latency based , heavy URL config etc)
4) Fine tune thresholds for fixed and ratio based
5) Switch to blocking – When needed
Fine Tune Thresholds Before attack
*good list for L7 DDoS metrics

Fine Tune Thresholds Before DDoS for Source IP
• View by: Client IP address
• List top TPS Avg IP’s
Go to Statistics ›› Analytics ›› HTTP ›› Transaction

Fine Tune Thresholds Before DDoS for Source IP
• By examining the client IP Address you can
conclude the averages of “normal” traffic you
expect to see from the top source IPs.
• Knowing “normal” averages can help defining
the TPS increase by ratio.
• The idea is that you can determine how much
traffic is allowed until assumed a ddos attack.

Fine Tune Thresholds Before DDoS for Geolocation
The same concept works for the
geolocation thresholds graph.
From the drilldown choose Countries on
AVR reports
“Which countries you expect to see traffic ?
Go to Security ›› Reporting ›› dos ››Application ›› transaction outcome

Fine Tune Thresholds Before DDoS for URL
The same idea applies to URL’s.
Sort graph by URL’s
“Which URL should have to highest RPS ?

Fine Tune Thresholds Before DDoS for URL

Fine Tune Thresholds Before DDoS for Site Wide
On the drilldown choose
Virtual Server
“This will help us understand
the over all traffic load that we
have when there is no DDoS
attack.

Fine Tune Thresholds Before DDoS for Site Wide
The overall traffic should be much higher than the other thresholds.
The values reflect the total amount of TPS that the virtual can handle.
Site wide = Virtual server

Fine Tune Thresholds During attack
Process:
1) Fine tune white list source – if needed
2) Identify sources that exceed thresholds (source IP’s, URL’s, Geo, SiteWide) by
looking at reporting.
3) Determine the attack type: from fixed/random source IP to fixed/ random URL.
Conclude which of the detection types you need (source IP only ? Source IP and URL
based only ? etc. )
4) Fine tune thresholds according to the exceeding sources (ratio / fixed)
5) Apply mitigation and decide what is working and what is not. Uncheck the
mitigations that are not effective
6) Go to step 1 and repeat

Fine Tune Thresholds During attack – Source IP
• Security ›› Reporting ›› dos
››Application ›› transaction
outcome
• On the drilldown choose
Client IP Address

Fine Tune Thresholds During attack – Geolocation
• Security ›› Reporting ›› dos ››
Application ›› transaction
outcome
Countries

Fine Tune Thresholds During attack – URL’s
outcome
• On the drilldown choose URL’s

Fine Tune Thresholds During attack - Site Wide
outcome
Virtual Servers

AVR reports and graphs
Mitigation type – can help
understand which of the
mitigation is effective and
when switching mitigation
occurred
Time line Attack start / end
Host IP
Number
of TPS
Attack ID : Clicking
will show graph
Security ›› Event Logs ›› DoS ›› Application Events

impact is the latency
on the backend for all
entities. The higher
the latency the higher
the impact is.
High, medium and low
impact allow to filter
high impact attacks
and deal with them
first
Security ›› Reporting ›› DoS ›› Application ›› Transaction Outcomes

Start and End points - red flags indicate the start of an attack and the green flags indicate the end of
an attack. Switching mitigation can occure several time over the DDoS attack.
Security ›› Reporting ›› DoS ›› Application ›› Transaction Outcomes

• Incomplete – Indicates traffic that was dropped by the server because the connection was
incomplete or the server did not respond.
• Blocked – Indicates traffic that was blocked as a result of the mitigation policy (any of the
prevention policies including bots blocking)
• Proactive Mitigation – Indicates the amount of time that the proactive bot defense mechanism
was severed
• CAPTCHA mitigation – Indicates the amount of time that the CAPTCHA challenge was severed
to offending sources
• CS integrity mitigation –Indicates the amount of time that the client-side integrity defense
challenge was severed to offending sources
• BIG IP Response – Indicates traffic that is a response to the client from the BIG-IP system.
• Cache by BIG IP – Indicates traffic that is served from cache configured (WA, RamCache)
• Whitelisted – Indicates traffic from IP Address that are in the whitelist of the DoS profile
• Pass through - Indicates traffic that is pass to the application trough ASM to the server

The AVR DoS graph now
shows the thresholds
that are set in the TPS
detection tab.
The Display Thresholds
check box will display
them or clear them from
the graph.

Before DDoS:
• Write the “normal” thresholds for the web site: (IP’s, Geolocation, URL’s, Site Wide)
• Set the ratio and the fix threshold for each of the above detection criteria (How much the web
site can take 2 times the traffic , 5 times etc…)
• Test the configuration and the prevention policy, then conclude which one is good for you
During DDoS:
• Identify the source IP’s, URL’s and entire site traffic increase and determine the attack type
• Set the fixed TPS number in each of the above criteria and apply mitigation
• Verify the results in the Transaction outcome graph
Fine Tune Thresholds– Summary

DDoS Bots - Detection & Mitigation

Layers of defense against Bots
Simple Bots
Impersonating Bots
Bots with cookies / JS capabilities
Bots acting as full browser
Gohogle
This bot section is mostly about
bots that DoS / DDoS. However,
Bots detection and prevention
can be used for various bot
problems the site is experiencing.

DDoS Bots
Servers Database
Google Web Bot
Unidentified
User
User
Users Or Bots
Web Site
Web Bot
Bots can be classify in many ways, mostly there are:
1. Simple bots
2. Impersonating Bots
3. Bots with cookies & JS capabilities
4. Bots acting as full browser

Enabling Bot signatures protection

Bots – Simple Bot
Server
ASM: yes, I have your signature. Sorry
mate you are blocked.
I’m a simple Bot
Simple bot can be any command line tool such as:
curl , wget , ab

Categorizing Bots
Bad Bots
Good Bots
Bad Bots aka Malicious
are well know command
line tools – we want
them out
Good Bots aka Benign are
well know search engine and
monitoring tools – we want
them in

Bot Signatures -
None
Report
Block
None
Report
Block
Each categories include:
• None – ignore
• Report – report only – used for monitoring
• Block – block

Excluding specific bot signatures from category settings
• Specific signature can be excluded from the
category setting
• Search the signature in Available signature list
and move it to the left pane.
• In this example ab tool will not be blocked
even if the category that includes it is in
blocking mode

First - White list good Bots
Web Server
Google
I’m a google Bot
ASM: let’s see if you really are. I’m doing
Reverse DNS lookup.
Yes, I see that, please continue.
DNS Server
Google
Thanks 
1
2
3
4

White list good Bots -
with their domain name
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
1. Request arrive with User
Agent : Googlebot/2.1
DNS Server
Web Server
2. ASM search the
google bot signature
3. The signature includes domain name.
ASM issue Reverse DNS query to verify
the origin of the request
4. Once approved ASM will allow the
google bot to access the web site

Bot Signature Repository
• Bot Signature repository for
the entire system is under
Options.
• Bot signatures repository is
update with the ASM
signature update
Security ›› Options ›› DoS Protection ›› Bot Signatures List

Bot Signature List: general signatures repository
Signatures can be sort by:
• Signature Category
• Signature Type:
Malicious / Benign
• User Define signatures
Yes / No
• Partition: signature can
be assign to a specific
partition
Clicking on any of the sorting
will change the order.

Sorting the Bot Signature Repository
Various filtering
Create new Bot Signature

Bot Signature Categories
Creating new category
for Malicious or Benign

Create a new bot signature: simple edit mode
Simple edit mode: inside
a user agent header or
in a URL.
Category
Domain name – execute
reverser DNS query to
verify origin. Add the
domain if the Bot has one
Bot Signature name
Create when done

Create a new bot signature - advanced edit mode
Signature syntax example:
headercontent: "sample_text"; useragentonly;
Advanced Edit Mode - rule granularity
For full details consult F5 document

• Signatures associated with a domain name are
validated with reverse DNS lookup.
• Blocking and reporting :
• Block flag - resets the connection and reports the
action as "bot signature block" with the bot
signature name.
• Report flag - Report bot name and categories
(AVR)
• Updating bot signature as part of the ASM
signature update
Bot signature facts

Bots – Impersonating Bot
Web Server
Gohogle
I’m a google Bot
ha ha ha
ASM: let’s see if you are. I’m doing Reverse
DNS lookup.
DNS Server
Gohogle
Bummer ASM: you are not google bot
Bye Bye -> block this creature !
ASM: Hey DNS, who’s this guy ?
DNS: no one important

Bots – Impersonating Bot
1 Request arrive with User
Agent : Googlebot/2.1
DNS Server
Web Server
2 ASM search the
google bot signature
3 The real google bot includes domain
name. ASM issue Reverse DNS query to
verify the origin of the request
4 If the source IP is not the expected one
according to the DNS query ASM will block
the impersonating bot

Bots with cookies & JS capability
Web ServersI’m a bot that can
understand JS and
support cookies
ASM: prove it, answer my challenges
No you are not, bye bye -> block this bot.
Bummer
Ha ?

Proactive Bot Defense
PBD is good for:
• Bots that can handle JS
• Bots that can handle JS and cookies
• Bots floods
• Under the radar bots
• Block any bot accessing the site (humans only web site)

Proactive Bot Defense and Bot Signature
Proactive Bot
Defense is now
integrated with
the bot
signatures.
When enabling
proactive bot
defense the
bot signature
feature will be
enabled as
well

• Send Client side challenges to ALL client and thus mitigate bots all the time
• Various challenges are sent and then validate by PBD – blocked or allowed
Proactive Bot Defense

PBD - Client side integrity defense - flow
First main page access
HTTP Request (no cookie)
Computational challenge
Solve challenge/
set cookie with time stamp
HTTP Request (cookie) Reconstruct request
Original HTTP Request
More object requests (cookie)
Validate cookie: format & time stamp
More object requests
More responses
More responsesDeliver page

• Always – sending CS all the time
• During attack – only if other component of the dos profile is in dos mode PBD
will send the CS challenge (acting as two layers of mitigation)
This allows second layer of protection (rate limit and PBD)
• Grace period - cookie expiration time 300 = 5min
• White list – exclude PBD on those IP’s
Proactive Bot Defense – configuration

Bots acting as full browser
Web Server
I’m a Bot that
simulate browser
ASM: ok, what are your capability ? If you will not
answer right you will have to answer a CAPTCHA
You are not human, byyyye -> block this unhuman !
DNS Server
Bummer
Capability ?
CAPTCHA ?

PBD – Additional bots identification with capabilities script
Bots: Bots acting as full browsers - Browser Simulation

How bots that simulate
browsers are evaluated ?
Block Suspicious Browsers – addition tests are done to understand if this is a bot or a browser. ASM
will evaluate the source and will give it a score:
if the score indicates that the source is a bot it will block it.
If the score indicate uncertainty and if CAPTCHA challenge is checked, then CAPTCHA will be presented
to the source. If answer it is a human if not, blocked.

Block Suspicious Browsers
• If Block Suspicious Browsers is unchecked  send CS Challenge
• If Block Suspicious Browsers is checked and CAPTCHA is checked  send Client
Capabilities challenge and give it a score: If score in doubt send a CAPTCHA for
human verification
• If Block Suspicious Browsers is checked but CAPTCHA Challenge is unchecked  do
not send CAPTCHA and only block if the score is more than a human

Client Capabilities -challenge script flow
First request GET /sell.php
GET /sell.php (no cookie)
Client Capabilities Challenge response
Return Client Capabilities
verification
Reconstruct request
HTTP Response (cookie)
HTTP Response
GET /img.png (cookie)
Blank page & Set cookie
Original HTTP Request + cookie
1. Authenticate and decrypted JS results,
2. Verify capabilities and set a score
3. Determine an action based on score
GET /img.png (cookie)
Validate cookie: format
& time stamp

DoS Bots Reporting

Bot signatures simulation Reporting ›› dos ›› Application ›› Transaction outcomes
Transaction outcomes
is very useful for
monitoring traffic and
indicates various
measurements

Bot signatures simulation
Analytics ›› HTTP ›› throughput ›› request throughput
AVR will provide details on DoS bot signatures (use drill downs )

• Simple bots can easily be detected and blocked
• White listing of bots = visibility to bot access and keep other bots out
• Impersonating bots can be monitored / blocked
• Bots that support JavaScript and cookies can now be noticed and be blocked
• Reporting on the visiting bots to your web site is available via AVR
• Custom bots signature is powerful tool to deal with bots
• Bots signature is updating via the ASM signatures update
Summary

Resources
Our documentation is free for all. Read and learn more:
BIG-IP Application Security Manager: Getting Started
BIG-IP Application Security Manager Operations Guide
BIG-IP Application Security Manager: Implementations
BIG-IP Application Security Manager: Custom Signature Reference
BIG-IP Analytics: Implementations

F5 ASM v12 DDoS best practices

F5 ASM v12 DDoS best practices

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to F5 ASM v12 DDoS best practices (20)

More from Lior Rotkovitch (12)

Recently uploaded (20)

F5 ASM v12 DDoS best practices