SlideShare a Scribd company logo

Apache Wizardry - Ohio Linux 2011

Download as KEY, PDF
Rich Bowen, profile picture
Rich Bowen

This document provides an overview of Apache HTTP Server configuration recipes and new features. It discusses solutions for the CVE-2011-3192 vulnerability, using mod_substitute to rewrite content, setting up name-based virtual hosts with SSL and Server Name Indication (SNI), securing mod_dav deployments, and advantages of the new <If> directive in Apache 2.4 for conditional configuration blocks based on request properties. The document encourages users to implement web application firewall rules with mod_security and provides tips for troubleshooting slow PHP scripts.

Technology
1 of 198
Downloaded 29 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
Apache Wizardry Spells for conjuring your Apache HTTP Server / Rich Bowen - rbowen@apache.org
Shameless Plug Buy! Buy! Buy!
Old Hat ----> • 1.3 is end of life • 2.0 will be by the end of the year • 2.2 is now • 2.4 is tomorrow Photo CC by “Lost Albatross” - Flickr
URLs you need • http://slideshare.net/rbowen • http://wiki.apache.org/httpd • http://httpd.apache.org/docs/trunk/
Recipes Photo CC by Chemical Heritage Foundation - Flickr
CVE-2011-3192 (KillApache) • Exploits bug in handling of range requests • Range: requests a byte-range of a document • Range: 0-4,10-19,16-22,3-18 • Eg, single pages of a PDF document • Fixed in 2.2.20 and in trunk
What's the problem? • No limit to the number or size of ranges • Ranges can overlap • Apache builds the entire response in memory before sending • Very easy to construct a range set that consumes all available RAM
Solutions: Best (2.2) # Drop the Range header when more than 5 ranges. # CVE-2011-3192 SetEnvIf Range (,.*?){5,} bad-range=1 RequestHeader unset Range env=bad-range # optional logging. CustomLog logs/range-CVE-2011-3192.log common env=bad-range
Solution 2 # Reject request when more than 5 ranges in the # Range: header -- CVE-2011-3192 RewriteEngine on RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$) RewriteRule .* - [F] • Use this for 1.3 and 2.0, because they don't have the PCRE sauce in SetEnvIf • Better yet, don't use 1.3 and 2.0
Other solutions • Disallow Range requests entirely RequestHeader unset Range • Limit the size of the Range header LimitRequestFieldSize 200
Directory listings are boring
Say it with style IndexStyleSheet /styles/dir.css ... .odd { background-color: #eef; } .even { background-color: #fff; }
Caveat: Some features 2.4 only • In 2.2 and earlier, you can specify a style sheet, but no classes are added to the HTML • Useful, but not quite as useful
Now, with extra class <table id="indexlist"> <tr class="indexhead"> <th class="indexcolicon">... <th class="indexcolname">... <th class="indexcollastmod">... <th class="indexcolsize">... <th class="indexcoldesc">... <tr class="indexbreakrow">... <tr class="even"><td class="indexcolicon">...
An exercise for the reader: • Nifty mouse-over effects (JS in HeaderName file?) Photo CC by Ugglan - Flickr • AJAXy file interaction of some kind? • Photo gallery, entirely based on mod_autoindex and Javascript?
mod_substitite • New module in 2.2 • Rewrite content using regular expressions • Syntax is identical to sed
s/foo/bar/ • Simple example - switch fonts LoadModule substitute_module libexec/apache2/mod_substitute.so AddOutputFilterByType SUBSTITUTE text/html Substitute s/ariel/courier/i
Proxying • More useful example • Proxying to back-end server that returns fully-qualified URLs LoadModule substitute_module libexec/apache2/mod_substitute.so AddOutputFilterByType SUBSTITUTE text/html Substitute s/backend.local/www.example.com/ni
mod_security • Are you running mod_security?
mod_security • You should be
http firewall • The earlier you catch it ...
Holes Photo CC by Darwin Bell
SQL Injection # Prevent SQL injection attacks SecFilter "delete[[:space:]]+from" SecFilter "insert[[:space:]]+into" SecFilter "select.+from" 23
Make sure you ... # Inspect POST payloads SecFilterScanPOST On # Default action set SecFilterDefaultAction "deny,log,status:406" 24
While you’re at it # Inspect POST payloads SecFilterScanPOST On SecFilter “vidocin” 25
Acceptable arguments # Only for the FormMail script <Location /cgi-bin/FormMail> # Reject request where the value of parameter "recipient" # does not end with "@apache.org" SecFilterSelective ARG_recipient "![a-zA-Z0-9]+@apache.org$"> </Location> 26
Which PHP script is slagging my server? • Which process, and what is it doing? • Look at /server-status for a process list
Step 1: Look at top: • Run ‘top’ • Order by CPU usage • Pick off the httpd processes that are causing the problem and make note of their PIDs
Step 2: /server-status • Look at /server-status output • Look for the PIDs you noted • Move fast - the process may be gone already
/server-status
ExtendedStatus On • You’ll need “ExtendedStatus On” to get these details: <Location /server-status> SetHandler server-status </Location> ExtendedStatus On
SNI • General knowledge: SSL requires one certificate per IP address • That is, only one SSL site can be on each IP address • Limitation of SSL itself
SSL handshake
SSL handshake Certificate Hostname
Clearly this sucks
SNI
SNI • Server Name Indication • Passes server name in initial handshake • Simple solutions are always best
Caveats • You knew there would be a catch Mozilla Firefox 2.0 or later Opera 8.0 or later (the TLS 1.1 protocol must be enabled) Internet Explorer 7 (Vista or higher, not XP) or later Google Chrome (Vista or higher, not XP. OS X 10.5.7 or higher on Chrome 5.0.342.1 or newer) Safari Safari 3.2.1 and newer on Mac OS X 10.5.6 and Windows Vista or higher, not XP
Apache • 2.2.12 or later Listen 443 # Listen for virtual host requests on all IP addresses NameVirtualHost *:443 # Go ahead and accept connections for these vhosts # from non-SNI clients SSLStrictSNIVHostCheck off <VirtualHost *:443> DocumentRoot /www/example1 ServerName www.example.com </VirtualHost> <VirtualHost *:443> DocumentRoot /www/example2 ServerName www.example2.org </VirtualHost>
More Info http://wiki.apache.org/httpd/NameBasedSSLVHostsWithSNI
Secure mod_dav deployment • Security rule #1: Content is not writable • Corollary: Anyone telling you to ‘chmod 777’ is a monkey • Anyone telling you ‘chown apache something.php’ *might* be a monkey. Or they might be working around Apache’s annoying permissions model
However ... •Setting up WebDAV requires that content be writable ... •And owned by Apache •This is annoying
WebDAV <Directory /var/www/content> Dav On </Directory>
Why this is a problem • People write bad code • It’s easy to get PHP (or whatever else) to overwrite your content • Now your site is a radical terrorist site • This is a very unpleasant thing to wake up to on a Saturday morning
Secure DAV • It’s possible to set up Dav without having the files written by the Apache user • Sort of 45
Two Apache Processes Primary Secondary server, server, running as running as user www user dav File System, owned by dav 46
Two Apache Processes Read-only Running ordinary set of modules Running web apps, etc Primary server, running as user www File System, owned by dav 47
Two Apache Processes Read/Write Remove all extra modules SSL Secondary server, Authenticated running as user dav File System, owned by dav 48
Multi-server config. drwxr-xr-x 2 dav dav 68 Oct 3 12:41 /var/www 1 Listen *:80 User www Group www DocumentRoot /var/www/ Listen *:8080 2 DavLockDb /var/lock/dav User dav Group dav DocumentRoot /var/www/ 49
drwxr-xr-x 2 dav dav 68 Oct 3 12:41 /var/www 1 Listen *:80 Can write User www Group www DocumentRoot /var/www/ Listen *:8080 2 DavLockDb /var/lock/dav User dav Group dav DocumentRoot /var/www/ 50
drwxr-xr-x 2 dav dav 68 Oct 3 12:41 /var/www 1 Can’t Listen *:80 User www Group www DocumentRoot /var/www/ Listen *:8080 2 DavLockDb /var/lock/dav User dav Group dav DocumentRoot /var/www/ 51
<If> Picture by BrewBooks (Flickr) My favorite new feature in 2.4
<If> <If ‘$req{Host} = “www.example.com”’> RedirectMatch (.*) http://example.com/$1 </If>
<If> <If ‘$req{Host} = “www.example.com”’> RedirectMatch (.*) http://example.com/$1 </If> This was hard prior to 2.4, and probably required mod_rewrite, or a separate virtual host.
<If> <If ‘$req{Host} = “www.example.com”’> RedirectMatch (.*) http://example.com/$1 </If> $req $resp $env
Logging in 2.4 FYI: • 2.4 is currently in beta • Will release any day now • 2.0 will be declared "end of life" • 2.2 will be declared "maintenance only"
Per-module LogLevel • mod_proxy is very chatty • I really want to know what mod_substitute is doing • Each log line prefixed by the name of the module LogLevel warning substitute:debug
[Mon Aug 29 08:05:02.001881 2011] [core:warn] [pid 14974:tid 140735307352416] pid file /usr/local/apache2/logs/httpd.pid overwritten -- Unclean shutdown of previous Apache run? [Mon Aug 29 08:05:02.009024 2011] [mpm_event:notice] [pid 14974:tid 140735307352416] Apache/2.3.12-dev (Unix) PHP/5.3.6 DAV/2 configured -- resuming normal operations [Mon Aug 29 08:05:02.009411 2011] [core:notice] [pid 14974:tid 140735307352416] Command line: '/usr/local/apache2/bin/httpd' [Mon Aug 29 08:05:16.479215 2011] [negotiation:error] [pid 14977:tid 4322246656] (2)No such file or directory: [client ::1:54690] cannot access type map file: /Users/rbowen/devel/apache/httpd-trunk/docs/manual/ logging.html
LogLevel settings • emerg • alert • crit • error • warn • notice • info • debug
New ones:
LogLevels • Original LogLevel settings taken from syslog • New ones more simply named tracen • Noisier than debug • Not to be confused with TRACEEnable, which is something completely different
No more RewriteLog LogLevel alert rewrite:trace3 • Trace slows things down - use only for debugging • tail -f error_log | grep "rewrite:"
Configurable • Error log format is configurable, like the access log ErrorLogFormat "[%t] [%l] [pid %P] %F: %E: [client %a] %M"
[Sun Sep 04 18:30:47.846743 2011] [rewrite:trace1] [pid 15332:tid 4334051328] mod_rewrite.c(468): [client :: 1:61990] ::1 - - [localhost/sid#10080f190][rid#10286b2a0/ initial] pass through /favicon.ico [Sun Sep 04 18:30:47.850033 2011] [core:error] [pid 15332:tid 4334051328] [client ::1:61990] File does not exist: / usr/local/apache2/htdocs/favicon.ico
Whodunnit? (%L) • Correlate access log entries with error log entries • Which request caused this error? • Add %L to the LogFormat and to the ErrorLogFormat
FallbackResource # BEGIN WordPress   RewriteEngine On   RewriteRule ^index.php$ - [L]   RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f   RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d   RewriteRule . /index.php [L] # END WordPress
FallbackResource # BEGIN WordPress #  RewriteEngine On # RewriteRule ^index.php$ - [L] # RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f #  RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d #  RewriteRule . /index.php [L] # END WordPress FallbackResource /index.php # available in 2.2.8
Logging - Conditional Logging • Don’t log certain things • Per-directory logging 68
Conditional LogFormat • The LogFormat directive supports some conditionals in the variables • "%!200,304,302{Referer}i" logs Referer on all requests that do not return one of the three specified codes 69
Conditional LogFormat • "%400,501{User-agent}i" logs User-agent on 400 errors and 501 errors only • For other status codes, the literal string "-" will be logged 70
Conditional CustomLog • Stick Env=xyz on the end • or Env=!xyz • Yes, that’s =! not != 71
For example ... SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 72
For example ... SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 73
For example ... SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 74
So the images get logged here SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 75
And everything else goes here SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 76
Per directory SetEnvIf Request_URI ^/marketing mkt CustomLog marketing.log common env=mkt 77
PCRE Zero-width assertions • Match everything except one thing • While you can do this with RewriteRule, it would be nice if you could do it with other directives DirectoryMatch FilesMatch RedirectMatch
Negative Lookahead • PCRE provides a regex syntax to say “not preceded by” or “not followed by” • Negative lookbehind and negative lookahead, respectively
Andrei Rocks • While we’re on the topic: • Pretty much the best presentation on regular expressions anywhere: • http://www.slideshare.net/ andreizm/andreis-regex-clinic
Everything except ... • “I want to redirect everything except / images” • This is where you’d use a negative lookahead • Necessary because RedirectMatch doesn’t support negation RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
Everything except ... • Match anything ... • RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
Everything except ... • Match anything ... • That doesn’t start with images/ RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
Everything except ... This is called a “zero width” assertion, because it doesn’t fill $1, and doesn’t consume any characters in the match. RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
Apache Wizardry - Ohio Linux 2011
What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9
What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9 • Alas, not in .htaccess • Logs are always opened at startup
What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9 • Most entries between 1-4
What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9 • If there’s nothing in there, your rules are being ignored.
Logging - RewriteLog RewriteLog /var/log/rewrite.log RewriteLogLevel 9 90
Logging - RewriteLog RewriteLog /var/log/rewrite.log RewriteLogLevel 9 91
Logging - RewriteLog RewriteLog /var/log/rewrite.log RewriteLogLevel 9 92
93
Kind of intimidating, isn’t it? 94
Learn to ignore the irrelevant bits 95
Which bits are those? 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 96
Client address 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 97
Dunno what those are 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 98
Time 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 99
Unique id 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 100
Request ID 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 101
The useful bit 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 102
Wouldn’t it be nice if you could just see the useful part? 103
You could change the way mod_rewrite logs:
Piped logs • I actually use a piped lot handler to remove this superfluous stuff • Like so ... RewriteLog |/usr/local/bin/rewrite_log_pipe RewriteLogLevel 9 105
RewriteLog |/usr/local/bin/rewrite_log_pipe RewriteLogLevel 9 with ... #!/usr/bin/perl $|++; open (F, ">>/tmp/rewrite"); select F; while (<>) { s/^.*((d).*)/$1/; print; } 106
RewriteLog |/usr/local/bin/rewrite_log_pipe RewriteLogLevel 9 This bit says “instead of logging to a text file, invoke this script and send the log entries there.” 107
#!/usr/bin/perl $|++; open (F, ">>/tmp/rewrite"); select F; while (<>) { s/^.*((d).*)/$1/; print; } • Look for the (1) or (2) bit • drop everything before that 108
Results in: (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 109
Results in: (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' better? See? Isn’t that (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 110
Requested URI (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 111
Patterns applied (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 112
None of them matched (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 113
And now • We can actually make some sense of what’s happening • Less inscrutable noise • Yes, it means something, but not to normal people 114
Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • This was the result of RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 115
Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • It shows what the input variable looked like RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 116
Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • And what pattern was applied RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 117
Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • As well as what happened RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 118
Another example (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • Was a result of RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 119
Again ... (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • What was requested RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 120
And ... (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • What it was compared against RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 121
Matched? (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • If it matched, the next line will be the action log RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 122
The whole thing (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 123
The match: (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 124
Followed by (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 125
[R] (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 126
But it all runs together! • Look for: • (2) init rewrite engine with requested uri /atom/1 • ‘init rewrite engine’ shows where a new request started being rewritten 127
Apache Wizardry - Ohio Linux 2011
Load balancing
Apache Wizardry - Ohio Linux 2011
Fortunately ... Photo CC by Camo53 (flickr) • mod_proxy_balancer • Added in 2.1
Fairly simple to configure <Proxy balancer://mycluster> BalancerMember http://192.168.1.50:80 BalancerMember http://192.168.1.51:80 </Proxy> ProxyPass /test balancer://mycluster
Fairly simple to configure <Proxy balancer://mycluster> BalancerMember http://192.168.1.50:80 BalancerMember http://192.168.1.51:80 </Proxy> ProxyPass /test balancer://mycluster
Fairly simple to configure <Proxy balancer://mycluster> BalancerMember http://192.168.1.50:80 BalancerMember http://192.168.1.51:80 </Proxy> ProxyPass /test balancer://mycluster
Sticky Sessions Ensures that connections go to the same server they started with. ProxyPass / balancer://mycluster/ stickysession=PHPSESSIONID 135
Balancing measures ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 136
1.2.3.5 gets twice the traffic ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 137
Hot spare ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 138
bytraffic or byrequests ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 139
BalancerManager <Location /balancer-manager> SetHandler balancer-manager Order Deny,Allow Deny from all Allow from .example.com </Location> 140
141
Disable a particular host 142
Notes • The things on the other end don’t have to be Apache • This is a popular way to set up Ruby on Rails, with Mongrel 143
server-info <Location /server-info> SetHandler server-info </Location>
Apache Wizardry - Ohio Linux 2011
Apache Wizardry - Ohio Linux 2011
Demo • Insert /server-info demo here
Require • Order deny,allow • Order allow,deny • huh?
Require
Require
RequireAll <RequireAll> Require ip 10.1 Require Group marketing </RequireAll>
RequireAny <RequireAny> Require ip 10.1 Require Group marketing </RequireAny>
RequireNone <RequireNone> Require ip 10.1 Require Group marketing </RequireNone>
Combined <Directory /www/mydocs> <RequireAll> <RequireAny> Require user superadmin <RequireAll> Require group admins Require ldap-group cn=Administrators,o=Airius <RequireAny> Require group sales Require ldap-attribute dept="sales" </RequireAny> </RequireAll> </RequireAny> <RequireNone> Require group temps Require ldap-group cn=Temporary Employees,o=Airius </RequireNone> </RequireAll> </Directory>
Expressions • New expression parser in 2.4 • Available in most directives that do any kind of comparison or matching • Will be available more places as time goes on
Apache Wizardry - Ohio Linux 2011
Apache Wizardry - Ohio Linux 2011
Apache Wizardry - Ohio Linux 2011
Apache Wizardry - Ohio Linux 2011
Other ... • http://localhost/manual/expr.html
Bandwidth limiting Picture by Joachim S. Müller (Flickr) People always want their websites to run slower. Seems odd to me ...
In 2.4 ... • 2.4 adds two new modules for this purpose • mod_dialup • mod_ratelimit
mod_dialup Party like it’s 1999
v.92? Really? <Location /mysite> ModemStandard V.92 </Location> Also available: V.21 V.26bis V.32 V.92
mod_ratelimit <Location /downloads> SetHandler RATE_LIMIT SetEnv rate-limit 400 </Location> Speed is in kb/s
Prior to 2.4 A variety of other modules do bandwidth kind of things: mod_cband mod_bwshare mod_bw mod_evasive mod_limitipconn
Logging • mod_logio • mod_log_forensic
Obligatory ridiculous log photo Photo by Zevotron (Flickr)
Not enough • Your log files don’t tell you enough • Want more
Logging - mod_logio Complete INPUT and OUTPUT size 170
Logging - mod_logio Complete INPUT and OUTPUT size 171
Combined Log Format - Bytes transferred 172
Less than half the story • Only bytes transferred to the client • Doesn’t include headers • Doesn’t include data sent from the client to the server 173
mod_logio •Adds two additional variables •%I - Input bytes •%O - Output bytes •Includes headers, both directions 174
LogFormat 175
mod_log_forensic • Did it ever finish?
ForensicLog ForensicLog /var/log/httpd/forensic.log +TlvWvsCoyKYAAAPmBy8AAABG|GET /manual/images/feather.gif HTTP/1.1|Host:localhost| Connection:keep-alive|Referer:http%3a//localhost/manual/|User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.215 Safari/535.1| Accept:*/*|Accept-Encoding:gzip,deflate,sdch|Accept-Language:en-US,en;q=0.8,de;q=0.6|Accept- Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3 +TlvWvsCoyKYAAAPmBzAAAABB|GET /manual/style/css/manual.css HTTP/1.1|Host:localhost| Connection:keep-alive|Referer:http%3a//localhost/manual/|User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.215 Safari/535.1| Accept:text/css,*/*;q=0.1|Accept-Encoding:gzip,deflate,sdch|Accept-Language:en- US,en;q=0.8,de;q=0.6|Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3 +TlvWvsCoyKYAAAPmBzEAAABI|GET /manual/images/left.gif HTTP/1.1|Host:localhost| Connection:keep-alive|Referer:http%3a//localhost/manual/|User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.215 Safari/535.1| Accept:*/*|Accept-Encoding:gzip,deflate,sdch|Accept-Language:en-US,en;q=0.8,de;q=0.6|Accept- Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3 -TlvWvsCoyKYAAAPmBy8AAABG -TlvWvsCoyKYAAAPmBzAAAABB -TlvWvsCoyKYAAAPmBzEAAABI
ForensicLog ForensicLog /var/log/httpd/forensic.log • Post-process with the check- forensic script to tell you which URLs never exited
mod_whatkilledus • Third-party module • people.apache.org/~trawick/ Find it at http:// • Tells you what killed us
mod_whatkilledus [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus sig 11 crash [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus active connection: 127.0.0.1:43923->127.0.0.1:10101 (conn_rec 30683a38) [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus active request (request_rec 30689978): GET /silly/?fn=sigsegv HTTP/1.0|Connection:close [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus Active module: mod_silly2.c [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus end of report
mod_speling CheckSpelling On CheckCaseOnly On
Image Theft • <img src="http://your.site.com/ cool_picture.jpg">
Image Theft SetEnvIf Referer ".example.com/" local_referal # Allow browsers that do not send Referer info SetEnvIf Referer "^$" local_referal <Directory /web/images> Order Deny,Allow Deny from all Allow from env=local_referal </Directory>
Or ... RewriteEngine on RewriteCond %{HTTP_REFERER} !="" RewriteCond %{HTTP_REFERER} !example.com [NC] RewriteRule .(jpe?g|gif|png)$ - [F,NC]
Or ... RewriteEngine on RewriteCond %{HTTP_REFERER} !="" RewriteCond %{HTTP_REFERER} !example.com [NC] # depending upon in which context # you use the RewriteRule, # you might need a condition to # exclude the go_away.png to prevent # an internal redirect looping. We don't use a RegEx here: RewriteCond %{REQUEST_URI} !=/images/go_away.png RewriteRule .(jpe?g|gif|png)$ /images/go_away.png [NC,L]
Or ... RewriteEngine on RewriteCond %{HTTP_REFERER} !="" RewriteCond %{HTTP_REFERER} !example.com [NC] # depending upon in which context # you use the RewriteRule, # you might need a condition to # exclude the go_away.png to prevent # an internal redirect looping. We don't use a RegEx here: RewriteCond %{REQUEST_URI} !=/images/go_away.png RewriteRule .(jpe?g|gif|png)$ http://other.example.com/images/go_away.png [R,NC,L]
Or ... In 2.4: <Location /images> <If "$req{Referer} !~ /mysite.com/"> Require all denied </If> </Location>
Query Strings and Path Info
Query Strings monkeys.php?q=lemur&color=green&level=99.7&doc=info.txt • Query Strings are: • Ugly • Hard to type • Hard to remember • Potentially insecure
Query Strings monkeys.php?q=lemur&color=green&level=99.7&doc=info.txt • Query Strings are: • Ugly • Hard to type • Hard to remember • Potentially insecure
Path Info monkeys.php?q=lemur&color=green&level=99.7&doc=info.txt /monkeys/lemur/green/99.7/info
mod_rewrite monkeys.php?q=lemur&color=green&level=99.7&doc=info.txt /monkeys/lemur/green/99.7/info RewriteRule ^/monkeys/(.+)/(.+)/(.+)/(.+) /monkeys.php?q=$1&color=$2&level=$3&doc=$4.txt [PT,L,NE]
Engineering for failure • Why not do it right to begin with, and avoid the mess? • PHP makes this fairly easy • Use exactly the same technique in whatever your preferred language is
Step One: SetHandler /monkeys/lemur/green/99.7/info • We want monkeys to be a PHP script • We rename monkeys.php to monkeys, and then ... <Files monkeys> SetHandler application/x-httpd-php </Files> This goes in your server config, or in .htaccess
Step Two: explode() /monkeys/lemur/green/99.7/info • The rest of the solution is in your PHP: $args = explode( $_SERVER[‘PATH_INFO’] ); $type = $args[0]; $color = $args[1]; ...
While we’re at it • File extensions are *so* 1980s • All your files are php files, right? • Why give them a .php extension? RewriteCond %{REQUEST_URI} !. RewriteRule ^ - [H=application/x-httpd-php,PT]
... • If, by some miracle, we get this far ... • Questions?
Addresses you need • http://slideshare.net/rbowen • http://wiki.apache.org/httpd • http://httpd.apache.org/docs/trunk/ • rbowen@apache.org • http://omniti.com/is/hiring

More Related Content

KEY
Apache Cookbook - TekX Chicago 2010
Rich Bowen
 
PPTX
Real World Lessons on the Pain Points of Node.js Applications
Ben Hall
 
PPTX
WAF Bypass Techniques - Using HTTP Standard and Web Servers’ Behaviour
Soroush Dalili
 
ODP
Apache httpd 2.4: The Cloud Killer App
Jim Jagielski
 
PPTX
Real World Experience of Running Docker in Development and Production
Ben Hall
 
PPTX
Lessons from running potentially malicious code inside containers
Ben Hall
 
PDF
Varnish Configuration Step by Step
Kim Stefan Lindholm
 
PDF
Apache Traffic Server & Lua
Kit Chan
 
Apache Cookbook - TekX Chicago 2010
Rich Bowen
 
Real World Lessons on the Pain Points of Node.js Applications
Ben Hall
 
WAF Bypass Techniques - Using HTTP Standard and Web Servers’ Behaviour
Soroush Dalili
 
Apache httpd 2.4: The Cloud Killer App
Jim Jagielski
 
Real World Experience of Running Docker in Development and Production
Ben Hall
 
Lessons from running potentially malicious code inside containers
Ben Hall
 
Varnish Configuration Step by Step
Kim Stefan Lindholm
 
Apache Traffic Server & Lua
Kit Chan
 

What's hot (20)

PPTX
Installation Openstack Swift
ymtech
 
PDF
FreeBSD: Dev to Prod
Sean Chittenden
 
PDF
Load Balancing MySQL with HAProxy - Slides
Severalnines
 
PDF
Australian OpenStack User Group August 2012: Chef for OpenStack
Matt Ray
 
PPT
Why Managed Service Providers Should Embrace Container Technology
Sagi Brody
 
ODP
HAProxy scale out using open source
Ingo Walz
 
PDF
Hadoop meet Rex(How to construct hadoop cluster with rex)
Jun Hong Kim
 
PPTX
Apache Performance Tuning: Scaling Up
Sander Temme
 
PDF
Building your own CDN using Amazon EC2
SergeyChernyshev
 
PDF
Acus08 Advanced Load Balancing Apache2.2
Jim Jagielski
 
PDF
How To Set Up SQL Load Balancing with HAProxy - Slides
Severalnines
 
PDF
Integrated Cache on Netscaler
Mark Hillick
 
PDF
10 Million hits a day with WordPress using a $15 VPS
Paolo Tonin
 
PPTX
Dockerizing WordPress
dotCloud
 
PPT
Python Deployment with Fabric
andymccurdy
 
PPTX
Usenix LISA 2012 - Choosing a Proxy
Leif Hedstrom
 
PDF
Windows Server 2016 Webinar
Men and Mice
 
ODP
MNPHP Scalable Architecture 101 - Feb 3 2011
Mike Willbanks
 
PPTX
Performance all teh things
Marcus Deglos
 
PDF
Failsafe Mechanism for Yahoo Homepage
Kit Chan
 
Installation Openstack Swift
ymtech
 
FreeBSD: Dev to Prod
Sean Chittenden
 
Load Balancing MySQL with HAProxy - Slides
Severalnines
 
Australian OpenStack User Group August 2012: Chef for OpenStack
Matt Ray
 
Why Managed Service Providers Should Embrace Container Technology
Sagi Brody
 
HAProxy scale out using open source
Ingo Walz
 
Hadoop meet Rex(How to construct hadoop cluster with rex)
Jun Hong Kim
 
Apache Performance Tuning: Scaling Up
Sander Temme
 
Building your own CDN using Amazon EC2
SergeyChernyshev
 
Acus08 Advanced Load Balancing Apache2.2
Jim Jagielski
 
How To Set Up SQL Load Balancing with HAProxy - Slides
Severalnines
 
Integrated Cache on Netscaler
Mark Hillick
 
10 Million hits a day with WordPress using a $15 VPS
Paolo Tonin
 
Dockerizing WordPress
dotCloud
 
Python Deployment with Fabric
andymccurdy
 
Usenix LISA 2012 - Choosing a Proxy
Leif Hedstrom
 
Windows Server 2016 Webinar
Men and Mice
 
MNPHP Scalable Architecture 101 - Feb 3 2011
Mike Willbanks
 
Performance all teh things
Marcus Deglos
 
Failsafe Mechanism for Yahoo Homepage
Kit Chan
 
Ad

Similar to Apache Wizardry - Ohio Linux 2011 (20)

PDF
[Devconf.cz][2017] Understanding OpenShift Security Context Constraints
Alessandro Arrichiello
 
PPT
Apache web server installation/configuration, Virtual Hosting
webhostingguy
 
PDF
PowerPoint Presentation
webhostingguy
 
PDF
Vorontsov, golovko ssrf attacks and sockets. smorgasbord of vulnerabilities
DefconRussia
 
PPTX
Dockerizing WordPress
Docker, Inc.
 
PPTX
Dockerizing the Hard Services: Neutron and Nova
clayton_oneill
 
PDF
Road to Opscon (Pisa '15) - DevOoops
Gianluca Varisco
 
PDF
Apache2 BootCamp : Getting Started With Apache
Wildan Maulana
 
PPTX
WordPress Development Environments
Ohad Raz
 
PDF
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
Codemotion
 
PDF
Deploying to Ubuntu on Linode
WO Community
 
PPTX
Big Data in Container; Hadoop Spark in Docker and Mesos
Heiko Loewe
 
PDF
Top ten-list
Brian DeShong
 
PPTX
cache concepts and varnish-cache
Marc Cortinas Val
 
PDF
Build an autoversioning filesystem with Apache2
Javier Arturo Rodríguez
 
PPTX
Drupal, varnish, esi - Toulouse November 2
Marcus Deglos
 
KEY
Site Performance - From Pinto to Ferrari
Joseph Scott
 
PPTX
Nginx A High Performance Load Balancer, Web Server & Reverse Proxy
Amit Aggarwal
 
PDF
Postgres the hardway
Dave Pitts
 
PDF
WordPress At Scale. WordCamp Dhaka 2019
Anam Ahmed
 
[Devconf.cz][2017] Understanding OpenShift Security Context Constraints
Alessandro Arrichiello
 
Apache web server installation/configuration, Virtual Hosting
webhostingguy
 
PowerPoint Presentation
webhostingguy
 
Vorontsov, golovko ssrf attacks and sockets. smorgasbord of vulnerabilities
DefconRussia
 
Dockerizing WordPress
Docker, Inc.
 
Dockerizing the Hard Services: Neutron and Nova
clayton_oneill
 
Road to Opscon (Pisa '15) - DevOoops
Gianluca Varisco
 
Apache2 BootCamp : Getting Started With Apache
Wildan Maulana
 
WordPress Development Environments
Ohad Raz
 
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
Codemotion
 
Deploying to Ubuntu on Linode
WO Community
 
Big Data in Container; Hadoop Spark in Docker and Mesos
Heiko Loewe
 
Top ten-list
Brian DeShong
 
cache concepts and varnish-cache
Marc Cortinas Val
 
Build an autoversioning filesystem with Apache2
Javier Arturo Rodríguez
 
Drupal, varnish, esi - Toulouse November 2
Marcus Deglos
 
Site Performance - From Pinto to Ferrari
Joseph Scott
 
Nginx A High Performance Load Balancer, Web Server & Reverse Proxy
Amit Aggarwal
 
Postgres the hardway
Dave Pitts
 
WordPress At Scale. WordCamp Dhaka 2019
Anam Ahmed
 
Ad

More from Rich Bowen (7)

ODP
The apacheway
Rich Bowen
 
KEY
Why your employees should contribute to Open Source
Rich Bowen
 
KEY
URL Mapping, with and without mod_rewrite
Rich Bowen
 
KEY
Don't be a jerk
Rich Bowen
 
KEY
Write A Better FM - Ohio Linux 2011
Rich Bowen
 
KEY
mod_rewrite bootcamp, Ohio LInux 2011
Rich Bowen
 
KEY
Write a better FM
Rich Bowen
 
The apacheway
Rich Bowen
 
Why your employees should contribute to Open Source
Rich Bowen
 
URL Mapping, with and without mod_rewrite
Rich Bowen
 
Don't be a jerk
Rich Bowen
 
Write A Better FM - Ohio Linux 2011
Rich Bowen
 
mod_rewrite bootcamp, Ohio LInux 2011
Rich Bowen
 
Write a better FM
Rich Bowen
 

Recently uploaded (20)

PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
PDF
Modernizing your data center with Dell and AMD
Principled Technologies
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
AVTRON Technologies LLC
 
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
PDF
Advanced IT Governance
University of Hertfordshire
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
Teaching material agriculture food technology
LiaRayya
 
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
Modernizing your data center with Dell and AMD
Principled Technologies
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
AVTRON Technologies LLC
 
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
Advanced IT Governance
University of Hertfordshire
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 

Apache Wizardry - Ohio Linux 2011

  • 1. Apache Wizardry Spells for conjuring your Apache HTTP Server / Rich Bowen - rbowen@apache.org
  • 2. Shameless Plug Buy! Buy! Buy!
  • 3. Old Hat ----> • 1.3 is end of life • 2.0 will be by the end of the year • 2.2 is now • 2.4 is tomorrow Photo CC by “Lost Albatross” - Flickr
  • 4. URLs you need • http://slideshare.net/rbowen • http://wiki.apache.org/httpd • http://httpd.apache.org/docs/trunk/
  • 5. Recipes Photo CC by Chemical Heritage Foundation - Flickr
  • 6. CVE-2011-3192 (KillApache) • Exploits bug in handling of range requests • Range: requests a byte-range of a document • Range: 0-4,10-19,16-22,3-18 • Eg, single pages of a PDF document • Fixed in 2.2.20 and in trunk
  • 7. What's the problem? • No limit to the number or size of ranges • Ranges can overlap • Apache builds the entire response in memory before sending • Very easy to construct a range set that consumes all available RAM
  • 8. Solutions: Best (2.2) # Drop the Range header when more than 5 ranges. # CVE-2011-3192 SetEnvIf Range (,.*?){5,} bad-range=1 RequestHeader unset Range env=bad-range # optional logging. CustomLog logs/range-CVE-2011-3192.log common env=bad-range
  • 9. Solution 2 # Reject request when more than 5 ranges in the # Range: header -- CVE-2011-3192 RewriteEngine on RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$) RewriteRule .* - [F] • Use this for 1.3 and 2.0, because they don't have the PCRE sauce in SetEnvIf • Better yet, don't use 1.3 and 2.0
  • 10. Other solutions • Disallow Range requests entirely RequestHeader unset Range • Limit the size of the Range header LimitRequestFieldSize 200
  • 12. Say it with style IndexStyleSheet /styles/dir.css ... .odd { background-color: #eef; } .even { background-color: #fff; }
  • 13. Caveat: Some features 2.4 only • In 2.2 and earlier, you can specify a style sheet, but no classes are added to the HTML • Useful, but not quite as useful
  • 14. Now, with extra class <table id="indexlist"> <tr class="indexhead"> <th class="indexcolicon">... <th class="indexcolname">... <th class="indexcollastmod">... <th class="indexcolsize">... <th class="indexcoldesc">... <tr class="indexbreakrow">... <tr class="even"><td class="indexcolicon">...
  • 15. An exercise for the reader: • Nifty mouse-over effects (JS in HeaderName file?) Photo CC by Ugglan - Flickr • AJAXy file interaction of some kind? • Photo gallery, entirely based on mod_autoindex and Javascript?
  • 16. mod_substitite • New module in 2.2 • Rewrite content using regular expressions • Syntax is identical to sed
  • 17. s/foo/bar/ • Simple example - switch fonts LoadModule substitute_module libexec/apache2/mod_substitute.so AddOutputFilterByType SUBSTITUTE text/html Substitute s/ariel/courier/i
  • 18. Proxying • More useful example • Proxying to back-end server that returns fully-qualified URLs LoadModule substitute_module libexec/apache2/mod_substitute.so AddOutputFilterByType SUBSTITUTE text/html Substitute s/backend.local/www.example.com/ni
  • 19. mod_security • Are you running mod_security?
  • 20. mod_security • You should be
  • 21. http firewall • The earlier you catch it ...
  • 22. Holes Photo CC by Darwin Bell
  • 23. SQL Injection # Prevent SQL injection attacks SecFilter "delete[[:space:]]+from" SecFilter "insert[[:space:]]+into" SecFilter "select.+from" 23
  • 24. Make sure you ... # Inspect POST payloads SecFilterScanPOST On # Default action set SecFilterDefaultAction "deny,log,status:406" 24
  • 25. While you’re at it # Inspect POST payloads SecFilterScanPOST On SecFilter “vidocin” 25
  • 26. Acceptable arguments # Only for the FormMail script <Location /cgi-bin/FormMail> # Reject request where the value of parameter "recipient" # does not end with "@apache.org" SecFilterSelective ARG_recipient "![a-zA-Z0-9]+@apache.org$"> </Location> 26
  • 27. Which PHP script is slagging my server? • Which process, and what is it doing? • Look at /server-status for a process list
  • 28. Step 1: Look at top: • Run ‘top’ • Order by CPU usage • Pick off the httpd processes that are causing the problem and make note of their PIDs
  • 29. Step 2: /server-status • Look at /server-status output • Look for the PIDs you noted • Move fast - the process may be gone already
  • 31. ExtendedStatus On • You’ll need “ExtendedStatus On” to get these details: <Location /server-status> SetHandler server-status </Location> ExtendedStatus On
  • 32. SNI • General knowledge: SSL requires one certificate per IP address • That is, only one SSL site can be on each IP address • Limitation of SSL itself
  • 34. SSL handshake Certificate Hostname
  • 36. SNI
  • 37. SNI • Server Name Indication • Passes server name in initial handshake • Simple solutions are always best
  • 38. Caveats • You knew there would be a catch Mozilla Firefox 2.0 or later Opera 8.0 or later (the TLS 1.1 protocol must be enabled) Internet Explorer 7 (Vista or higher, not XP) or later Google Chrome (Vista or higher, not XP. OS X 10.5.7 or higher on Chrome 5.0.342.1 or newer) Safari Safari 3.2.1 and newer on Mac OS X 10.5.6 and Windows Vista or higher, not XP
  • 39. Apache • 2.2.12 or later Listen 443 # Listen for virtual host requests on all IP addresses NameVirtualHost *:443 # Go ahead and accept connections for these vhosts # from non-SNI clients SSLStrictSNIVHostCheck off <VirtualHost *:443> DocumentRoot /www/example1 ServerName www.example.com </VirtualHost> <VirtualHost *:443> DocumentRoot /www/example2 ServerName www.example2.org </VirtualHost>
  • 41. Secure mod_dav deployment • Security rule #1: Content is not writable • Corollary: Anyone telling you to ‘chmod 777’ is a monkey • Anyone telling you ‘chown apache something.php’ *might* be a monkey. Or they might be working around Apache’s annoying permissions model
  • 42. However ... •Setting up WebDAV requires that content be writable ... •And owned by Apache •This is annoying
  • 43. WebDAV <Directory /var/www/content> Dav On </Directory>
  • 44. Why this is a problem • People write bad code • It’s easy to get PHP (or whatever else) to overwrite your content • Now your site is a radical terrorist site • This is a very unpleasant thing to wake up to on a Saturday morning
  • 45. Secure DAV • It’s possible to set up Dav without having the files written by the Apache user • Sort of 45
  • 46. Two Apache Processes Primary Secondary server, server, running as running as user www user dav File System, owned by dav 46
  • 47. Two Apache Processes Read-only Running ordinary set of modules Running web apps, etc Primary server, running as user www File System, owned by dav 47
  • 48. Two Apache Processes Read/Write Remove all extra modules SSL Secondary server, Authenticated running as user dav File System, owned by dav 48
  • 49. Multi-server config. drwxr-xr-x 2 dav dav 68 Oct 3 12:41 /var/www 1 Listen *:80 User www Group www DocumentRoot /var/www/ Listen *:8080 2 DavLockDb /var/lock/dav User dav Group dav DocumentRoot /var/www/ 49
  • 50. drwxr-xr-x 2 dav dav 68 Oct 3 12:41 /var/www 1 Listen *:80 Can write User www Group www DocumentRoot /var/www/ Listen *:8080 2 DavLockDb /var/lock/dav User dav Group dav DocumentRoot /var/www/ 50
  • 51. drwxr-xr-x 2 dav dav 68 Oct 3 12:41 /var/www 1 Can’t Listen *:80 User www Group www DocumentRoot /var/www/ Listen *:8080 2 DavLockDb /var/lock/dav User dav Group dav DocumentRoot /var/www/ 51
  • 52. <If> Picture by BrewBooks (Flickr) My favorite new feature in 2.4
  • 53. <If> <If ‘$req{Host} = “www.example.com”’> RedirectMatch (.*) http://example.com/$1 </If>
  • 54. <If> <If ‘$req{Host} = “www.example.com”’> RedirectMatch (.*) http://example.com/$1 </If> This was hard prior to 2.4, and probably required mod_rewrite, or a separate virtual host.
  • 55. <If> <If ‘$req{Host} = “www.example.com”’> RedirectMatch (.*) http://example.com/$1 </If> $req $resp $env
  • 56. Logging in 2.4 FYI: • 2.4 is currently in beta • Will release any day now • 2.0 will be declared "end of life" • 2.2 will be declared "maintenance only"
  • 57. Per-module LogLevel • mod_proxy is very chatty • I really want to know what mod_substitute is doing • Each log line prefixed by the name of the module LogLevel warning substitute:debug
  • 58. [Mon Aug 29 08:05:02.001881 2011] [core:warn] [pid 14974:tid 140735307352416] pid file /usr/local/apache2/logs/httpd.pid overwritten -- Unclean shutdown of previous Apache run? [Mon Aug 29 08:05:02.009024 2011] [mpm_event:notice] [pid 14974:tid 140735307352416] Apache/2.3.12-dev (Unix) PHP/5.3.6 DAV/2 configured -- resuming normal operations [Mon Aug 29 08:05:02.009411 2011] [core:notice] [pid 14974:tid 140735307352416] Command line: '/usr/local/apache2/bin/httpd' [Mon Aug 29 08:05:16.479215 2011] [negotiation:error] [pid 14977:tid 4322246656] (2)No such file or directory: [client ::1:54690] cannot access type map file: /Users/rbowen/devel/apache/httpd-trunk/docs/manual/ logging.html
  • 59. LogLevel settings • emerg • alert • crit • error • warn • notice • info • debug
  • 61. LogLevels • Original LogLevel settings taken from syslog • New ones more simply named tracen • Noisier than debug • Not to be confused with TRACEEnable, which is something completely different
  • 62. No more RewriteLog LogLevel alert rewrite:trace3 • Trace slows things down - use only for debugging • tail -f error_log | grep "rewrite:"
  • 63. Configurable • Error log format is configurable, like the access log ErrorLogFormat "[%t] [%l] [pid %P] %F: %E: [client %a] %M"
  • 64. [Sun Sep 04 18:30:47.846743 2011] [rewrite:trace1] [pid 15332:tid 4334051328] mod_rewrite.c(468): [client :: 1:61990] ::1 - - [localhost/sid#10080f190][rid#10286b2a0/ initial] pass through /favicon.ico [Sun Sep 04 18:30:47.850033 2011] [core:error] [pid 15332:tid 4334051328] [client ::1:61990] File does not exist: / usr/local/apache2/htdocs/favicon.ico
  • 65. Whodunnit? (%L) • Correlate access log entries with error log entries • Which request caused this error? • Add %L to the LogFormat and to the ErrorLogFormat
  • 66. FallbackResource # BEGIN WordPress   RewriteEngine On   RewriteRule ^index.php$ - [L]   RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f   RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d   RewriteRule . /index.php [L] # END WordPress
  • 67. FallbackResource # BEGIN WordPress #  RewriteEngine On # RewriteRule ^index.php$ - [L] # RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f #  RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d #  RewriteRule . /index.php [L] # END WordPress FallbackResource /index.php # available in 2.2.8
  • 68. Logging - Conditional Logging • Don’t log certain things • Per-directory logging 68
  • 69. Conditional LogFormat • The LogFormat directive supports some conditionals in the variables • "%!200,304,302{Referer}i" logs Referer on all requests that do not return one of the three specified codes 69
  • 70. Conditional LogFormat • "%400,501{User-agent}i" logs User-agent on 400 errors and 501 errors only • For other status codes, the literal string "-" will be logged 70
  • 71. Conditional CustomLog • Stick Env=xyz on the end • or Env=!xyz • Yes, that’s =! not != 71
  • 72. For example ... SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 72
  • 73. For example ... SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 73
  • 74. For example ... SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 74
  • 75. So the images get logged here SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 75
  • 76. And everything else goes here SetEnvIf Request_URI .gif$ gif-image CustomLog gif-requests.log common env=gif-image CustomLog nongif-requests.log common env=!gif-image 76
  • 77. Per directory SetEnvIf Request_URI ^/marketing mkt CustomLog marketing.log common env=mkt 77
  • 78. PCRE Zero-width assertions • Match everything except one thing • While you can do this with RewriteRule, it would be nice if you could do it with other directives DirectoryMatch FilesMatch RedirectMatch
  • 79. Negative Lookahead • PCRE provides a regex syntax to say “not preceded by” or “not followed by” • Negative lookbehind and negative lookahead, respectively
  • 80. Andrei Rocks • While we’re on the topic: • Pretty much the best presentation on regular expressions anywhere: • http://www.slideshare.net/ andreizm/andreis-regex-clinic
  • 81. Everything except ... • “I want to redirect everything except / images” • This is where you’d use a negative lookahead • Necessary because RedirectMatch doesn’t support negation RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
  • 82. Everything except ... • Match anything ... • RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
  • 83. Everything except ... • Match anything ... • That doesn’t start with images/ RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
  • 84. Everything except ... This is called a “zero width” assertion, because it doesn’t fill $1, and doesn’t consume any characters in the match. RedirectMatch ^/(?!images/)(.*) http://other.myhost.com/$1
  • 86. What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9
  • 87. What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9 • Alas, not in .htaccess • Logs are always opened at startup
  • 88. What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9 • Most entries between 1-4
  • 89. What the heck is it doing? RewriteLog /var/log/rewrite.log RewriteLogLevel 9 • If there’s nothing in there, your rules are being ignored.
  • 90. Logging - RewriteLog RewriteLog /var/log/rewrite.log RewriteLogLevel 9 90
  • 91. Logging - RewriteLog RewriteLog /var/log/rewrite.log RewriteLogLevel 9 91
  • 92. Logging - RewriteLog RewriteLog /var/log/rewrite.log RewriteLogLevel 9 92
  • 93. 93
  • 94. Kind of intimidating, isn’t it? 94
  • 95. Learn to ignore the irrelevant bits 95
  • 96. Which bits are those? 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 96
  • 97. Client address 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 97
  • 98. Dunno what those are 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 98
  • 99. Time 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 99
  • 100. Unique id 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 100
  • 101. Request ID 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 101
  • 102. The useful bit 121.14.76.185 - - [24/Sep/2008:21:35:51 --0400] [wooga.drbacchus.com/sid#b83444a8][rid#b85b8d00/ initial] (4) [perdir /var/www/vhosts/drbacchus/] RewriteCond: input='/var/www/vhosts/drbacchus/ podcasts/poetry/maninmoon.mp3' pattern='!-f' => not- matched 102
  • 103. Wouldn’t it be nice if you could just see the useful part? 103
  • 104. You could change the way mod_rewrite logs:
  • 105. Piped logs • I actually use a piped lot handler to remove this superfluous stuff • Like so ... RewriteLog |/usr/local/bin/rewrite_log_pipe RewriteLogLevel 9 105
  • 106. RewriteLog |/usr/local/bin/rewrite_log_pipe RewriteLogLevel 9 with ... #!/usr/bin/perl $|++; open (F, ">>/tmp/rewrite"); select F; while (<>) { s/^.*((d).*)/$1/; print; } 106
  • 107. RewriteLog |/usr/local/bin/rewrite_log_pipe RewriteLogLevel 9 This bit says “instead of logging to a text file, invoke this script and send the log entries there.” 107
  • 108. #!/usr/bin/perl $|++; open (F, ">>/tmp/rewrite"); select F; while (<>) { s/^.*((d).*)/$1/; print; } • Look for the (1) or (2) bit • drop everything before that 108
  • 109. Results in: (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 109
  • 110. Results in: (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' better? See? Isn’t that (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 110
  • 111. Requested URI (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 111
  • 112. Patterns applied (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 112
  • 113. None of them matched (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched (3) applying pattern 'wp-rss2.php' to uri '/index.php' (3) applying pattern '(journal/)?index.rdf' to uri '/index.php' (3) applying pattern '^/wordpress/wp-comments' to uri '/index.php' (3) applying pattern '^/perm/(.*)' to uri '/index.php' (3) applying pattern '^/articles?/(.*)' to uri '/index.php' (3) applying pattern '^/blog/(.*)' to uri '/index.php' (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/index.php' (3) applying pattern '^/book/cookbook' to uri '/index.php' (3) applying pattern '^/book/2.2' to uri '/index.php' (3) applying pattern '^/booklink/(.*)' to uri '/index.php' (3) applying pattern '^/books?/(.+)' to uri '/index.php' (1) pass through /index.php 113
  • 114. And now • We can actually make some sense of what’s happening • Less inscrutable noise • Yes, it means something, but not to normal people 114
  • 115. Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • This was the result of RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 115
  • 116. Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • It shows what the input variable looked like RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 116
  • 117. Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • And what pattern was applied RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 117
  • 118. Examples (4) RewriteCond: input='wooga.drbacchus.com' pattern='!^wooga.drbacchus .com' [NC] => not-matched • As well as what happened RewriteCond %{HTTP_HOST} !^wooga.drbacchus.com [NC] 118
  • 119. Another example (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • Was a result of RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 119
  • 120. Again ... (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • What was requested RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 120
  • 121. And ... (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • What it was compared against RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 121
  • 122. Matched? (3) applying pattern '^/book/(mod)?_?rewrite' to uri '/ index.php' • If it matched, the next line will be the action log RewriteRule ^/book/(mod)?_?rewrite http://www.amazon.com/exec/obidos/asin/ 1590595610/drbacchus/ [R,L] 122
  • 123. The whole thing (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 123
  • 124. The match: (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 124
  • 125. Followed by (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 125
  • 126. [R] (3) applying pattern '^/books?/(mod)?_?rewrite' to uri '/books/rewrite' (2) rewrite '/books/rewrite' -> 'http://www.amazon.com/exec/obidos/ asin/1590595610/drbacchus/' (2) explicitly forcing redirect with http://www.amazon.com/exec/ obidos/asin/1590595610/drbacchus/ (1) escaping http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ for redirect (1) redirect to http://www.amazon.com/exec/obidos/asin/1590595610/ drbacchus/ [REDIRECT/302] 126
  • 127. But it all runs together! • Look for: • (2) init rewrite engine with requested uri /atom/1 • ‘init rewrite engine’ shows where a new request started being rewritten 127
  • 131. Fortunately ... Photo CC by Camo53 (flickr) • mod_proxy_balancer • Added in 2.1
  • 132. Fairly simple to configure <Proxy balancer://mycluster> BalancerMember http://192.168.1.50:80 BalancerMember http://192.168.1.51:80 </Proxy> ProxyPass /test balancer://mycluster
  • 133. Fairly simple to configure <Proxy balancer://mycluster> BalancerMember http://192.168.1.50:80 BalancerMember http://192.168.1.51:80 </Proxy> ProxyPass /test balancer://mycluster
  • 134. Fairly simple to configure <Proxy balancer://mycluster> BalancerMember http://192.168.1.50:80 BalancerMember http://192.168.1.51:80 </Proxy> ProxyPass /test balancer://mycluster
  • 135. Sticky Sessions Ensures that connections go to the same server they started with. ProxyPass / balancer://mycluster/ stickysession=PHPSESSIONID 135
  • 136. Balancing measures ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 136
  • 137. 1.2.3.5 gets twice the traffic ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 137
  • 138. Hot spare ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 138
  • 139. bytraffic or byrequests ProxyPass / balancer://hotcluster/ <Proxy balancer://hotcluster> BalancerMember http://1.2.3.4:8009 loadfactor=1 BalancerMember http://1.2.3.5:8009 loadfactor=2 # The below is the hot standby BalancerMember http://1.2.3.6:8009 status=+H ProxySet lbmethod=bytraffic </Proxy> 139
  • 140. BalancerManager <Location /balancer-manager> SetHandler balancer-manager Order Deny,Allow Deny from all Allow from .example.com </Location> 140
  • 141. 141
  • 142. Disable a particular host 142
  • 143. Notes • The things on the other end don’t have to be Apache • This is a popular way to set up Ruby on Rails, with Mongrel 143
  • 144. server-info <Location /server-info> SetHandler server-info </Location>
  • 147. Demo • Insert /server-info demo here
  • 148. Require • Order deny,allow • Order allow,deny • huh?
  • 151. RequireAll <RequireAll> Require ip 10.1 Require Group marketing </RequireAll>
  • 152. RequireAny <RequireAny> Require ip 10.1 Require Group marketing </RequireAny>
  • 153. RequireNone <RequireNone> Require ip 10.1 Require Group marketing </RequireNone>
  • 154. Combined <Directory /www/mydocs> <RequireAll> <RequireAny> Require user superadmin <RequireAll> Require group admins Require ldap-group cn=Administrators,o=Airius <RequireAny> Require group sales Require ldap-attribute dept="sales" </RequireAny> </RequireAll> </RequireAny> <RequireNone> Require group temps Require ldap-group cn=Temporary Employees,o=Airius </RequireNone> </RequireAll> </Directory>
  • 155. Expressions • New expression parser in 2.4 • Available in most directives that do any kind of comparison or matching • Will be available more places as time goes on
  • 160. Other ... • http://localhost/manual/expr.html
  • 161. Bandwidth limiting Picture by Joachim S. Müller (Flickr) People always want their websites to run slower. Seems odd to me ...
  • 162. In 2.4 ... • 2.4 adds two new modules for this purpose • mod_dialup • mod_ratelimit
  • 163. mod_dialup Party like it’s 1999
  • 164. v.92? Really? <Location /mysite> ModemStandard V.92 </Location> Also available: V.21 V.26bis V.32 V.92
  • 165. mod_ratelimit <Location /downloads> SetHandler RATE_LIMIT SetEnv rate-limit 400 </Location> Speed is in kb/s
  • 166. Prior to 2.4 A variety of other modules do bandwidth kind of things: mod_cband mod_bwshare mod_bw mod_evasive mod_limitipconn
  • 167. Logging • mod_logio • mod_log_forensic
  • 168. Obligatory ridiculous log photo Photo by Zevotron (Flickr)
  • 169. Not enough • Your log files don’t tell you enough • Want more
  • 170. Logging - mod_logio Complete INPUT and OUTPUT size 170
  • 171. Logging - mod_logio Complete INPUT and OUTPUT size 171
  • 172. Combined Log Format - Bytes transferred 172
  • 173. Less than half the story • Only bytes transferred to the client • Doesn’t include headers • Doesn’t include data sent from the client to the server 173
  • 174. mod_logio •Adds two additional variables •%I - Input bytes •%O - Output bytes •Includes headers, both directions 174
  • 176. mod_log_forensic • Did it ever finish?
  • 177. ForensicLog ForensicLog /var/log/httpd/forensic.log +TlvWvsCoyKYAAAPmBy8AAABG|GET /manual/images/feather.gif HTTP/1.1|Host:localhost| Connection:keep-alive|Referer:http%3a//localhost/manual/|User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.215 Safari/535.1| Accept:*/*|Accept-Encoding:gzip,deflate,sdch|Accept-Language:en-US,en;q=0.8,de;q=0.6|Accept- Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3 +TlvWvsCoyKYAAAPmBzAAAABB|GET /manual/style/css/manual.css HTTP/1.1|Host:localhost| Connection:keep-alive|Referer:http%3a//localhost/manual/|User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.215 Safari/535.1| Accept:text/css,*/*;q=0.1|Accept-Encoding:gzip,deflate,sdch|Accept-Language:en- US,en;q=0.8,de;q=0.6|Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3 +TlvWvsCoyKYAAAPmBzEAAABI|GET /manual/images/left.gif HTTP/1.1|Host:localhost| Connection:keep-alive|Referer:http%3a//localhost/manual/|User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.215 Safari/535.1| Accept:*/*|Accept-Encoding:gzip,deflate,sdch|Accept-Language:en-US,en;q=0.8,de;q=0.6|Accept- Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3 -TlvWvsCoyKYAAAPmBy8AAABG -TlvWvsCoyKYAAAPmBzAAAABB -TlvWvsCoyKYAAAPmBzEAAABI
  • 178. ForensicLog ForensicLog /var/log/httpd/forensic.log • Post-process with the check- forensic script to tell you which URLs never exited
  • 179. mod_whatkilledus • Third-party module • people.apache.org/~trawick/ Find it at http:// • Tells you what killed us
  • 180. mod_whatkilledus [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus sig 11 crash [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus active connection: 127.0.0.1:43923->127.0.0.1:10101 (conn_rec 30683a38) [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus active request (request_rec 30689978): GET /silly/?fn=sigsegv HTTP/1.0|Connection:close [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus Active module: mod_silly2.c [Fri Mar 11 14:25:18 2005] pid 16934 mod_whatkilledus end of report
  • 181. mod_speling CheckSpelling On CheckCaseOnly On
  • 182. Image Theft • <img src="http://your.site.com/ cool_picture.jpg">
  • 183. Image Theft SetEnvIf Referer ".example.com/" local_referal # Allow browsers that do not send Referer info SetEnvIf Referer "^$" local_referal <Directory /web/images> Order Deny,Allow Deny from all Allow from env=local_referal </Directory>
  • 184. Or ... RewriteEngine on RewriteCond %{HTTP_REFERER} !="" RewriteCond %{HTTP_REFERER} !example.com [NC] RewriteRule .(jpe?g|gif|png)$ - [F,NC]
  • 185. Or ... RewriteEngine on RewriteCond %{HTTP_REFERER} !="" RewriteCond %{HTTP_REFERER} !example.com [NC] # depending upon in which context # you use the RewriteRule, # you might need a condition to # exclude the go_away.png to prevent # an internal redirect looping. We don't use a RegEx here: RewriteCond %{REQUEST_URI} !=/images/go_away.png RewriteRule .(jpe?g|gif|png)$ /images/go_away.png [NC,L]
  • 186. Or ... RewriteEngine on RewriteCond %{HTTP_REFERER} !="" RewriteCond %{HTTP_REFERER} !example.com [NC] # depending upon in which context # you use the RewriteRule, # you might need a condition to # exclude the go_away.png to prevent # an internal redirect looping. We don't use a RegEx here: RewriteCond %{REQUEST_URI} !=/images/go_away.png RewriteRule .(jpe?g|gif|png)$ http://other.example.com/images/go_away.png [R,NC,L]
  • 187. Or ... In 2.4: <Location /images> <If "$req{Referer} !~ /mysite.com/"> Require all denied </If> </Location>
  • 188. Query Strings and Path Info
  • 189. Query Strings monkeys.php?q=lemur&color=green&level=99.7&doc=info.txt • Query Strings are: • Ugly • Hard to type • Hard to remember • Potentially insecure
  • 190. Query Strings monkeys.php?q=lemur&color=green&level=99.7&doc=info.txt • Query Strings are: • Ugly • Hard to type • Hard to remember • Potentially insecure
  • 192. mod_rewrite monkeys.php?q=lemur&color=green&level=99.7&doc=info.txt /monkeys/lemur/green/99.7/info RewriteRule ^/monkeys/(.+)/(.+)/(.+)/(.+) /monkeys.php?q=$1&color=$2&level=$3&doc=$4.txt [PT,L,NE]
  • 193. Engineering for failure • Why not do it right to begin with, and avoid the mess? • PHP makes this fairly easy • Use exactly the same technique in whatever your preferred language is
  • 194. Step One: SetHandler /monkeys/lemur/green/99.7/info • We want monkeys to be a PHP script • We rename monkeys.php to monkeys, and then ... <Files monkeys> SetHandler application/x-httpd-php </Files> This goes in your server config, or in .htaccess
  • 195. Step Two: explode() /monkeys/lemur/green/99.7/info • The rest of the solution is in your PHP: $args = explode( $_SERVER[‘PATH_INFO’] ); $type = $args[0]; $color = $args[1]; ...
  • 196. While we’re at it • File extensions are *so* 1980s • All your files are php files, right? • Why give them a .php extension? RewriteCond %{REQUEST_URI} !. RewriteRule ^ - [H=application/x-httpd-php,PT]
  • 197. ... • If, by some miracle, we get this far ... • Questions?
  • 198. Addresses you need • http://slideshare.net/rbowen • http://wiki.apache.org/httpd • http://httpd.apache.org/docs/trunk/ • rbowen@apache.org • http://omniti.com/is/hiring

Editor's Notes