Identifying Web Servers: A First-look Into the Future of Web Server Fingerprinting

BlackHat Seattle, 2003 Identifying Web Servers: A First-look Into the Future of Web Server Fingerprinting

Introductions Jeremiah Grossman Founder and CEO, WhiteHat Security Bill Pennington Senior Engineer, WhiteHat Security

Web Server Fingerprinting Cross Site Tracing - XST Web Application Forensics Agenda

Determine the specific version of the target web server. Determine the configuration settings. Develop countermeasures to fingerprinting. Make patch delivery easier. Why Fingerprint?

Send the same HTTP request and get different responses Perform a single or standard set of HTTP request towards a web server. The varied differences in the responses will allow for accurate fingerprinting.

The Common Web Servers January 2003 Source: Netcraft

Quick Check IIS 4.0 - Public: OPTIONS, TRACE, GET, HEAD, POST, PUT, DELETE IIS 5.0 - Public: OPTIONS, TRACE, GET, HEAD, DELETE, PUT, POST, COPY, MOVE, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, SEARCH We can now differentiate between IIS 4.0 and IIS 5.0 and between Apache and IIS!

Quick Check Apache 2.0.x - Allow: GET, HEAD, POST, OPTIONS TRACE Apache 1.3.x - Allow: GET, HEAD, OPTIONS, TRACE We can now differentiate 1.3.x and 2.0.x because of the added POST OPTION.

Adequate Entropy The results from the sampling of HTTP output using only “OPTIONS *” provided enough data to start fingerprinting.

Server Responses Server Response Microsoft-IIS/4.0 Public: OPTIONS, TRACE, GET, HEAD, POST, PUT, DELETE Microsoft-IIS/5.0 Public: OPTIONS, TRACE, GET, HEAD, DELETE, PUT, POST, COPY, MOVE, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, SEARCH Allow: OPTIONS, TRACE, GET, HEAD, DELETE, PUT, POST, COPY, MOVE, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, SEARCH Apache/1.3.26 (Unix) Allow: GET, HEAD, OPTIONS,TRACE Apache/2.0.41-dev (Unix) Allow: GET,HEAD,POST,OPTIONS,TRACE Oracle9iAS/9.0.2 Oracle HTTP Server Oracle9iAS-Web-Cache/9.0.2.0.0 (N) Allow: GET, HEAD, OPTIONS, TRACE Netscape-Enterprise/3.6 SP2 Public: HEAD, GET, PUT, POST Netscape-Enterprise/4.0 Allow: HEAD, GET, PUT, POST Netscape-Enterprise/4.1 Allow: HEAD, GET, PUT, POST, DELETE, TRACE, OPTIONS, MOVE, INDEX, MKDIR, RMDIR Netscape-Enterprise/6.0 Allow: HEAD, GET, PUT, POST, DELETE, TRACE, OPTIONS, MOVE, INDEX, MKDIR, RMDIR

OPTIONS * Conclusions If the server allows and supports the “OPTIONS” HTTP Request Method, then with a reasonable level of certainty, we can conclude what the major version number is for a popular web server. The “Server” response header is no longer necessary to determine what a web server is running.

Other Request Methods Server Specific Methods Track - IIS only method Various HTTP response codes ///<dir> will return 400 status code on some Apache versions Various HTTP Status messages Alternating capitalization

Research is not complete! Larger pool of HTTP Requests More requests allow closer and more detailed accuracy of web server fingerprinting.

Fingerprinting Countermeasures Microsoft IIS URL Scan Secure IIS Server Mask Apache mod_rewrite httpd.conf changes source code modifications

Microsoft IIS URLScan Add the following line to your URLScan.ini file: [options] UseAllowVerbs=0 [Deny Verbs] OPTIONS Caution! Can cause some applications to break. (Frontpage,OWA)

Apache mod_rewrite Add the following to your httpd.conf RewriteEngine on RewriteCond %{REQUEST_METHOD} ^OPTIONS RewriteRule .* - [F]

Cross-Site Tracing A variation of cross-site scripting that increases the threat exposure. What can XST do that XSS cannot? Bypass HTTPOnly Restrictions Access to Basic Authentication Credentials Access to NTLM Credentials A web application is no longer required to cross-site script a user if the web server supports the TRACE request method. Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==

Exploit Requirements Cross Site Scripting A vulnerable web application A user that clicks on a link or views malicious content. Cross Site Tracing Web server that supports the trace request A place to host the XST code Cross domain bypass bug (if cross domain is required)

Steps of Cross-Site Scripting Attacker inserts code into a site or sends a malicsions HTML link to a user. User views the malicious content or clicks on the malicious link. Malicious code is executed with the hosting domain context, granting access to the cookie data., Cookie data is passed off domain to a third-party.

Steps of Cross-Site Tracing Attacker inserts code into a target site or hosts the code on a controlled web page. User views the web pages and malicious code executes within the browser. Code directs the browser to send a TRACE request to a target domain. Cookie, Basic Authentication, and NTLM credentials are sent back to the browser within the HTML Body. Authentication information is sent to a third-party. Server Attacker Victim Target Domain

XST Points to Remember This is a multi-platform multi-technology issue. Not restricted to ActiveX. Flash, Java, etc.

General Remedies Sufficiently patch all web browsers against known domain restriction bypass flaws. This is more important part of security policy now more than ever. Disable or disallow the TRACE Request method on production and development (unless needed) web servers. 3.Web server vendors should update their web server packages to disable TRACE by default. 4.Web server vendors should inform their users on how to disable or disallow TRACE on existing web servers. 5.ActiveX controls supporting arbitrary HTTP request should be marked unsafe for scripting by default. Other such technology vendors (Flash, Java, Shockwave, VBScript, etc.) should attempt to implement greater security mechanisms regarding disallowing unauthorized HTTP requests. Users have the ability to disable all active scripting and increase the safety of their credentials. However, this may negatively impact the functionality of many web sites.

Server Specific Server Specific (Resolutions should be confirmed by appropriate vendor) IIS - URL Scan Apache - Source Code Modification - Mod_Rewrite Module RewriteEngine on RewriteCond %{REQUEST_METHOD} ^TRACE RewriteRule .* - [F] (Thank you to Rain Forest Puppy) ** The Limit or LimitExcept directive in the httpd.conf file does not appear to be able to restrict TRACE. **

Microsoft IIS URLScan Add the following line to your URLScan.ini file: [options] UseAllowVerbs=0 [Deny Verbs] TRACE Caution! Can cause some applications to break. (Frontpage,OWA)

Apache mod_rewrite Add the following to your httpd.conf RewriteEngine on RewriteCond %{REQUEST_METHOD} ^TRACE RewriteRule .* - [F]

Forensics fo·ren·sics (f -r nsks, -z ks) n. (used with a sing. verb) 1. The art or study of formal debate; argumentation. 2. The use of science and technology to investigate and establish facts in criminal or civil courts of law.

Why? I could not find any good tools I only found one document and it was a marketing document. I was bored...

Typical Web Server Environment (Simplified) Web Server Application Server Database Server Internet Firewall

What are Web Attacks? SQL Injection Cross Site Scripting Parameter Tampering Directory Traversal Various Web Server Flaws Unicode Double Decode SSL Overflows

Avenues of Attack Port 80 - Clear text, easy to watch with an IDS system Port 443 - SSL encrypted can be watched with an IDS but advanced configuration is required, often not done

HTTP Request Get Request Easy everything is logged POST Request Only the Path is logged bummer...

Traditional Network IDS do not work Yes they will generally detect Nimda/CodeRed (Unicode/double decode) attacks. You could write rules to detect some basic attacks http://www.cgisecurity.com/web-attacks.rules It is almost impossible to detect certain attacks with a NIDS

Log Files IIS Log Files Stored in %winnt%/system32/logs/<servicename> Typically - C:/winnt/system32/logs/w3svc/*.log

IIS Log File Formats IIS log file format UserIP,UserName,Date,Time,Service,Computer Name,ServerIP,Time Taken,Bytes Sent,Bytes Received,Status Code, Windows Status,Request Type,Target,Parameters IIS can log to IIS,W3C Extended and NCSA common file format

File name determines type of log IIS format log files begin with in W3C extended log files begin with ex NCSA log files begin with nc

Apache Log File Locations ? %apache_home% /log/access.log

Apache Log File Formats By default Apache logs to NCSA common format or the combined log file format clientip,ident,username,date/time,request,status,bytes sent

Performance Tip Separate log files on a separate physical disk from content

Problems with web server logs POST data is rarely logged They are generally very large Contain lots of non-security related entries Many attacks can occur via POST request Some attacks can simply not be determined by log files

Log File Sizes www.whitehatsec.com From Jan 22nd - Feb 19th 466,829 lines eCom/Online Gaming Feb 1 - Feb 7 1,198,140 lines

What defines a bad request? 401 Response codes - Authentication required 500 Response codes - Server error, SQL injection 200 Response code - could be the worst of all, success

Weird Characters Some things should generally not be in a URL ‘ < > * .. etc...

Odd Request Methods 99% of applications use only GET and/or POST Why is someone HEADing me and should I let them?

Odd Request Methods Head Just returns server header no data. Used to probe for the existence of files Options Used to determine the capabilities of a web server and finger printing Trace Used for diagnostics. A possible attack vector XST Any WebDAV method (PropFind...) Used for managed web content. (Frontpage) and in some more robust web applications (OWA)

Introducing the HillBilly Not really an analysis tool, more of a data reduction tool Searches for odd URLs 500 errors Strange request methods

HillBilly Syntax ./hillbilly.pl -t <common,iis4,iis5> -l <logfile> -f <outputfile> -g (Look for odd GET request) -p (Look for 500 errors) -o (Look for odd request methods)

Odd URL search ./hillbilly.pl -t common -l access_log -g Regex = /[^A-Za-z0-9\.\/\?(%20)=_&-]/ Looks for request that contain characters other than these Will find Unicode,Double Decode, Cross Site Scripting, SQL Injection, Command Execution, Directory Traversal in a GET request.

Ecom log file reduction Log file reduced from 1,198,140 to 285,314

500 Errors Looks for any request method that generates a 500 error Large numbers of 500 errors from a single user over a short period can indicate a attack Check application server and SQL server logs Your time is synced right? ./hillbilly.pl -l access_log -t common -p 500 errors can indicate a SQL injection attack

Ecom odd request types Log file reduced from 1,198,140 to 0

Odd Request types ./hillbilly.pl -l access_log -t common -o Looks for any request type other than GET or POST Can point out probing request or finger printing attempts

Ecom odd request types Log file reduced from 1,198,140 to 2269

Prepare for the worst Configuring web server log files Know where they are! Additional utilities URLScan (IIS) mod_protect (Apache) Code Seeker (Cross platform)

Other logs SQL server logs Make sure they are on and at least logging errors Listen to your DBA whine about performance! Application Server Logs Make sure they are on Make sure you understand them

Time If you can’t sync it at least try to get it close You should really try to sync it, really

Using HillBilly as an IDS Danger this is untested!!! Danger this is probably insecure!!! CustomLog "|/usr/bin/hillbilly.pl -t common -l - -g >> /var/log/hillbilly.log" common Apache

Future Plans Recognize and automatically parse web server logs Use some sort of magic to profile the log file to look for truly deviant request Pretty output Add option to only look for successful request Write code others can read and use

Materials: www.whitehatsec.com

Identifying Web Servers: A First-look Into the Future of Web Server Fingerprinting

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Identifying Web Servers: A First-look Into the Future of Web Server Fingerprinting (20)

More from Jeremiah Grossman (20)

Recently uploaded (20)

Identifying Web Servers: A First-look Into the Future of Web Server Fingerprinting