Stupid iptables tricks

Stupid iptables TricksJim MacLeod@shewfigBsides Las Vegas 7/28/2010

Who I amNetwork hacker, not OS hackerFavorite language is bashIf a tool exists to do what I want, why re-invent the wheel?Sorry, Frank^2

What I get paid to doRetrofit security onto existing productsTry to keep 80% of you from breaking into my product in less than 1 weekTry to make my job go away by convincing my team to write secure code

OverviewDefining the box:Refresher on what iptables isMeasuring the box:How I learned what’s in this presentationOutside the box:Automating & abstracting policy creationCreating static tables with dynamic elementsPushing full packets to userspace

IptablesUmbrella termLinux kernel-level stateful packet filterPart of Netfilter – “iptables” is user-facing appIntegrates with PBR & other fun toysLinux characteristicsRich set of optionsMoon-man languageUser-friendly, but picky whom it considers a friend

Why we ignore iptables“It’s the Linux firewall, right? We already have a border firewall.”“It’s old technology” – L4, not L7[Insert Linux vs. Proprietaryargument]“Linux is a server”

Why we shouldn’t ignore iptablesStateful packet filter – most common firewall (!)Dynamic (non-destructive) policy updatesRich configuration optionsMultiple complex inter-connected policiesMakes it easy(er) to optimizeNested policiesShorter logic path in common caseVery complex policies for special casesExtensible state tablesExternal hooksPass the entire packet off to a userspace app

LimitationsRules exclusive to L2/L3/L4Iptables / ArptablesVulnerable to IP spoofing“Everything interesting” is at L7Only L7 capability is pattern matchAnyone remember 1st gen IDS?Subject to false positives if not pre-qualified

Overcoming limitationsThe “Wrong Way”TOFU auth (Trust On First Use)Not “meaty” enoughIP spoofing & session stealing can fry TOFUOptional cryptoE.g. SSL cleartext injection / renegotiationDo it all in L7What could possibly go wrong?“Business Logic” != networking

Overcoming limitationsThe “Cheap Way”State machine: is the connection in a state which implies that it has been authenticated?“Stateful” packet filter – ESTABLISHED state(TCP session established)How is this different than TOFU?If you don’t have a GOOD answer, it’s not good enough

Overcoming limitationsThe “Right Way”Strong authNon-persistent: require for every new connectionMandatory cryptoCan’t hijack / inject on properly implemented cryptoCrypto is an implicit auth of the established sessionHowever: crypto does not imply initial auth – anonymous crypto is possibleIPtables can’t do this for you 

Relevance?What does this have to do with iptables?Reminder that no Linux app or service exists in isolationUseful as 1 of multiple layersIPtables can be coupled tightly with additional security tools & methods

Advantages of using IPtablesAdd a bump in the wire before ANY app.If a problem can be “solved” in iptables, the app will inherit the solutionIPtables on-box has insight into the appProcess owner, SElinux context, etc.App can have insight & control of IPtables

Disadvantages of Solving Problems in IPtablesThere is more than one way to do itMany ways are “wrong”Hopefully this talk will show some “right” ways

Origin of this talk: my jobClustered, appliance-based web appProblem: how to add a new node?Fake new node would be “bad”™Pwn node before added -> pwn cluster

Problem 1: Stealth new nodeMust allow “real” inbound connectionsShould not reveal open ports to just anyoneNot enough time to implement auth

Solution 1: Port knockingCreate policy to allow only inbound traffic which behaves like a real clusterSpecific order:ICMP pingTCP echoBack-end portDatabase portProfit!If not in order, later ports won’t open

Implementation 1 (Step 1)iptables -A INPUT -p icmp --icmp-type 8 -m recent --name seenping --set-j ACCEPT

Implemenation 1 (Step 2)iptables -N ADD_ECHOiptables -A INPUT -p tcp --dport echo-m recent --name seenping --rcheck-j ADD_ECHOiptables -A ADD_ECHO-m recent --name seenecho --set-j ACCEPT(implicit return to INPUT)Can also explicitly specify with -j RETURN

Implementation 1 (Step 3)iptables -N ADD_BACKENDiptables -A INPUT -p tcp --dport 8000-m recent --name seenecho --rcheck--seconds 10-j ADD_BACKENDiptables -A ADD_BACKEND-m recent --name seenbackend --set-j ACCEPT

Implementation 1 (Step 4)iptables -A INPUT -p tcp --dport 3306-m recent --name seenbackend --rcheck-j ACCEPT

Problem 2: Database syncDatabase sync uses 2 opposite (A->B, B->A) TCP connectionsConnection from old to new: see Problem1Empty database, no interesting dataConnection from new to old: Family jewels exposedMust not open port to just anyone(yes, we’re using auth…)

Solution 2: Quid Pro QuoAka “Reverse Port Knocking”Old node connects to database outboundIPtables tracks outbound connectionExplicitly allows inbound connection

Implementation 2On existing node:iptables -A OUTPUT -p tcp --dport 3306-m recent --name dbout --rdest-j ACCEPTiptables -A INPUT -p tcp --dport 3306-m recent --name dbout --rcheck-j ACCEPT

Problem 3: Log ClogFirewall exceptions “should” be loggedSome protocols are really chattyWindows broadcastsLots of background noiseIPtables doesn’t have rich logging solutionSyslog as kernel messageHard to filter kernel facility“Previous message repeated 5 million times”

Solution 3: Seen-It SuppressionAdd source address to specific recency tableLog first time, ignore for next ‘n’ minutesSuppress the log clogDoesn’t log # of ignored packetsSorry, Marcus

Implementation 3iptables -A INPUT -p udp --dport 138-m recent --name udp-138 --rcheck --seconds 300-j DROPiptables -A INPUT -p udp --dport 138-m recent --name udp-138 --setiptables -A INPUT -j LOG --log-prefix “INPUT-drop”iptables -A INPUT -j DROP

Recap (So far)Recency tables for:Port knockingReverse port knockingLog suppressionAll withinIPtables

Version 1.3: Data-driven FirewallPlug-In architectureEach app might open different ports in/outEndpoints might be dynamicE.g. defined by user in in GUI

Version 1.3 SolutionXML firewall policy filesDefine inbound/outbound per-plug-inDefinition can also be database entryWatchdog to scan for changesChanges in filesPoll database for changesVersioning of policiesPolicy updates can be dynamic: insert/remove policies

Version 1.3 Implementation[1 KLOC of perl] - watchdog[1 KLOC of bash] - policy-checker[1 KLOC of perl] - XML-policy-parser[1 KLOC of bash] - iptables policy generator

Revisiting Problem 1: Add AuthPort knocking is an “open secret”Watch the wire & learn the comboCreate auth “shim” layerConnect to 1 port for authService adds permission for client to connect to “real” port use “-m recent” in iptables policyecho IP > /proc/net/xt_recent/[name]Note: must run as root 

Persistent PoliciesConfession: my data-driven firewall doesn’t actually intelligently modify the policyFlush tables, bulk pushFlushes ESTABLISHED and RELATED tables tooRules will re-add existing connectionsonly if packets come from client to serverWhat about packets from server to client?“Destination” and “Source” are a matter of perspective and initial state

Implementing Persistent PoliciesPolicy:iptables -p tcp --dport 8000 -j ACCEPTPersistent reverse policy:iptables -s [local IP] -p tcp --sport 8000--tcp-flags SYN NONE -j ACCEPTAllow packet in the “wrong” direction if it’s not the start of a TCP sessionDangerous territory: can reduce stealth“nmap -sA” will waltz right through if ‘-s’ omitted

Policy VersioningQuick hack: use commentsiptables -A PUSHDATE -m comment--comment `date ‘+%s’`Embed the date the policy is pushed as a commentChain PUSHDATE (0 references)num target prot opt source destination 1 all -- 0.0.0.0/0 0.0.0.0/0 /* 1280187751 */

Expanding the boxaka “what I haven’t actually done yet”Policy logicPattern matching less blindlyQ&D blocking IDSURL filtering per user

Policy LogicEach chain can call other chainse.g. -j ADD_PINGChains can return to their callere.g. -j RETURNDoesn’t this sound like function calls?recency tables =~ variablesSo is FWMARK

Canonical Firewall PolicyFirewall access rulesadmins can ssh inFirewall stealth ruleNo one else can access the firewallSeries of access/stealth rulesServers, subnets, “3-Tier”, etc.Outbound accessLog/Drop all

Optimized Firewall PolicyRule of thumb: more policies = more latencyAll packets to firewall go to separate chain-j FIREWALLImplement access / stealth / log / drop / smiteReduce the # of policies for each packet: reduce the overall latency

More “Policy Programming”FWMARKAdd “mark” to packet in kernellocal scope: only in the box, not on the wireiptables -j MARK --set-mark [unsigned int]iptables -m mark --mark [unsigned int]FWMARK is also recognized by:ip rule / policy-based routingipvs (L3 load balancer)

FWMARK exampleBind multiple protocols / ports together in a load balancer policyTypical web site: front page is http:80login is https:443content is http:80 againspawn whiteboard http:9012All connections from same client will go to same back end server

FWMARK vs. recencyRecency tables store IP & timestampsuseful for applying rules to an address that has been seen before in a different connectionno port numbers or protocolsFWMARK is per-packetno concept of connectionPotentially interesting in combinationmark all packets from a recent IPApply PBR

Pattern MatchingArbitrary pattern matches are too coarsePattern with offset:Helps if pattern GUARANTEED to be at that locationTCP options, encapsulation, etc. can move offsetUse chains to pre-qualify dataPort 80? That’s the HTTP_PATTERNS chain…DISCLAIMER:I have no actual knowledge of pattern matchingin iptables

Q&D blocking IDShttp://www.snowman.net/projects/ipt_recent/Example 1: block annoyance for 60 secondsiptables -A FORWARD -m recent --name noplacelike--rcheck--seconds 60 -j DROPiptables -A FORWARD -i eth0 -d 127.0.0.0/8-m recent --name noplacelike --set -j DROPExample 2: rolling blockiptables -A FORWARD -m recent --name noplacelike--update --seconds 60 -j DROPiptables -A FORWARD -i eth0 -d 127.0.0.0/8-m recent --name noplacelike --set -j DROP

URL FilteringTransparent Proxy is your friendhttp://www.ex-parrot.com/pete/upside-down-ternet.htmlTransparent Proxy uses iptablesiptables -t nat -A PREROUTING -s192.168.0.0/255.255.255.0-p tcp --dport 80 -j REDIRECT --to-port 8080Userecency tables to redirect some IPsiptables -t nat -A PREROUTING-m recent --name kittenwar-p tcp --dport 80 -j REDIRECT --to-port 8081Use with user auth, “guest” access, or annoyance triggers

Captive Portal“Yes I won’t hack you” page on WiFiiptables -m recent --name allowed -p tcp --dport 80 -j ACCEPTiptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000Captive Portal site will add IP to “allowed” list

Stealth ServicesUse iptables to snag traffic before it hits the real serviceiptables -t nat -A PREROUTING -p tcp --dport 80 -m recent --name secretweb -j REDIRECT --to-ports 84Simple way to run multiple VirtualHosts with 1 IP and a border firewall allowing 1 inbound portInteresting application of pattern match: HTTP Host header

Anti-Spam BlacklistingPre-populaterecency table with RBNiptables -p tcp --dport 25 -m recent --name spammer --rcheck -j REJECT

Anti-Spam Q&DGraylistingBlock initially, allow connect after 2 minutesiptables -p tcp --dport 25 -m recent --name smtpdelay --rcheck --seconds 120 -j DROPiptables -p tcp --dport 25 -m recent --name smtpdelay --rcheck ‘!’ --seconds 120 -j ALLOWiptables -p tcp --dport 25 -m recent --name smtpdelay --set -j DROP

Box?What box?Expandingiptables with -j QUEUEPass entire packet to userspace appWhole new world to extend iptablesCan only be one app running for this!!!If multiple “tricks” in place, app must use input validation to determine what is appropriate for which packet.

QUEUE tutorialhttp://michael.toren.net/slides/ipqueue/slide001.htmlOld (2003) but excellent!

Application: Packet Rewriting(Stolen blatantly from http://michael.toren.net/slides/ipqueue/slide025.html )my ($dns, $err) = Net::DNS::Packet->new(\$udp->{data}); ... for my $section qw(answer authority additional) { my @rr; while (my $i = $dns->pop($section)) { if ($i->class eq "IN" && $i->type eq "A" && $i->name eq "slashdot.org") { $i->{address} = "127.0.0.1"; } push @rr, $i; } while (my $i = pop @rr) { $dns->push($section, $i); } }

QUEUE example: countertraceAdd “imaginary” hopsOptionally add latencyLow-priority users get slower net accessQuick & dirty honeynet / tarpit

Reject with TTL ExpiredThe “ultimate” unreachable responseAlternatives:TCP RST: host exists, port not listening to youICMP port / host / net unreachableBlack-hole routeThrow away packetTTL expired implies you can’t get there from hereSo stop trying

Application AwarenessWindows Firewall has exceptions per-app“Outlook” can use internet, “Notepad” can’tIptables is port-focused onlyNo app awarenessCreate queue handler to look up app for the socket, whitelist apps.

Application Awareness example(Stolen blatantly from http://michael.toren.net/slides/ipqueue/slide017.html ) my %approved = qw( 561c1c9071e8c5723c641273e725c1e3 /usr/bin/telnet 9dc35c04c16d3f2ce2a8537961980913 /usr/bin/nc ); ... my ($user, $pid) = getuserfromtcp $ip, $tcp; open EXE, "/proc/$pid/exe"; my $md5 = Digest::MD5->new->addfile(*EXE)->hexdigest; close EXE; if ($pid > 0 && ! $approved{$md5}) { $ipq->set_verdict($msg->packet_id, NF_DROP);syslog "warning", "Blocking outbound connection attempt by unauthorized program"; } else { $ipq->set_verdict($msg->packet_id, NF_ACCEPT); }

Local DarknetMultipoint VPV to avoid eavesdroppingMulticast goes everywhere locally, plus potentially through routersDetect outbound multicast & splitIptables TEE targetCreate duplicate of packet & send someplace elseMay require patching iptables, maybe kernel

NAT’d host IDFingerprint incoming packets to differentiate a single host behind a shared NAT addressBase on super-session state: already seen connections?TOFU again

“Signed” TCPTCP includes a checksum – why not a MD?Add transparent proxy at each endUse REDIRECT or just QUEUESlightly more intrusive with netcat / ssh / etc.“Signature” is H(checksum + SEQ + PW)Easy to generate, easy to verify (re-generate)Password can be negotiated on first connectHash mismatch -> discard packetNo session hijack possible?Easier than IPSec, and “might” survive NAT.

Closing the Boxiptables is almost a programming languageIf iptables isn’t enough of a programming language, write a program to config itIf it can be done to a packet, it can likely be done in iptablesIf it can’t be done in iptables, it can be done externally

Stupid iptables tricks

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Stupid iptables tricks (20)

Recently uploaded (20)

Stupid iptables tricks