SlideShare a Scribd company logo
SQL-RISC
        New Directions in SQLi Prevention




Nick Galbreath                      RSA 2013-02-27
SQL-RISC:
    NEW DIRECTIONS IN SQLI
    DETECTION AND
    PREVENTION


    Nick Galbreath
    IPONWEB




►    Session ID:   ASEC-­‐W23

►    Session Classification:    Advanced
Latest version of these slides:
http://client9.com/20130227/

                   Tweet tweet: @ngalbreath


                    Originally presented at:
                    RSA Conference USA
                    Moscone Center
                    San Francisco, California
                    February 27, 2013
                    10:40 AM, Room 132
What if we could substantially
reduce the SQLi attack surface
    of a web application?




            @ngalbreath
without new hardware or
firewalls?




           @ngalbreath
without application changes?
           @ngalbreath
and comes with
free SQLi attack
monitoring?




            @ngalbreath
Wild Speculation?




    @ngalbreath
A Brief History of

SQL
1970s: SQL Invented
► Hey that's 40 years ago.
► "Structured Query Language"
► Why is it still around and so popular?
► Exercise for the reader: Pick any query you like, and
  write the equivalent in your favorite programming
  language.
► SQL is scriptable data structures.




                    @ngalbreath
1972: Oracle Releases the First
Commercial Database




   Also, Coppola releases The Godfather

               @ngalbreath
Remember the 80s?

                  ► Networking sucked!
                  ► TCP/IP not widespread
                  ► Computers are fragile,
                    expensive and slow
                  ► Shoulder-pads




                   so what do you do?


          @ngalbreath
Centralize
► Move computation as close as possible to the data
► Move to super-servers
► Have cheap/dumb clients




                     @ngalbreath
1988: Oracle V6
►   The Database is now a full programming environment
►   Unicode/UTF8 not standard
►   Complexity explosion
►   ... but most clients are private


                Also in 1988:
                Crack Cocaine 'invented'




                      @ngalbreath
And then the 1990s
  ►   TCP/IP
  ►   Cheap CPUs
  ►   Web Browsers
  ►   Attaching databases
      to public networks

This is why most of us
are here today.




                         @ngalbreath
2000+ Web Scale
► Discovery that data problems are a lot more painful
  than CPU problems.
► Turns out disk drives are mechanical
► If your database maxes out, you have big problems...
  so move everything out of the database.
► Complete reversal in strategy




                    @ngalbreath
SQL isn't going anywhere
► For front-ends, general trend is federating data across
  cheap machines, using stripped down SQL
► SQL-like languages used by Amazon, Google and
  others.
► Still great for analytics and reporting on the back end,
  and generic data storage.
► But we stuck with legacy of the past.




                     @ngalbreath
SQL'S DARK
                  CORNERS




► bit.ly/UkQd7Z
SQL is Huge
►   40+ years of built up crud
►   1992 spec is 625 pages of plain text
►   2003 spec is 128 pages of BNF
►   No one is completely compliant
►   Every one has special extensions


     SQL is more complicated
     than you think....


                        @ngalbreath
SQL Integer Forms

• [0-9]+
• 0x[0-9a-fA-F]+ 0xDEADbeef
   MySQL, MSSQL
   0x is case sensitive
• 0x MSSQL only
• x'DEADbeef' PgSQL
• b'10101010' MySQL, PgSQL
• 0b010101 MySQL




                 @ngalbreath
SQL Floating Point Forms
►   digits
►   digits[.]
►   digits[.]digits
►   digits[eE]digits
►   digits[eE][+-]digits
►   digits[.][eE]digits
►   digits[.]digits[eE][+-]digits
                                          ►   http://bit.ly/Qp6KTu
►   digits[.]digits[eE]digits
►   [.]digits
►   [.]digits[eE]digits
►   [.]digits[eE][+-]digits
                                    Optional starts with [+-]
►   "binary_float_infinity" (O)       Optional ending with [dDfF]
                                    (Oracle)


                             @ngalbreath
SQL Money Literals
►   MSSQL has a money type.
►   -$45.12
►   $123.0
►   +$1,000,000.00 Commas ignored
►   Many symbols are accepted for currency type




                       @ngalbreath
SQL Ridiculous Operators
▶ != not equals, standard
▶ <=> mysql
▶ <> mssql
▶ ^= oracle
▶ !>, !< not less than mssql
▶ / oracle
▶ !! factorial (pgsql)
▶ |/ square root (pgsql)
▶ ||/ cube root (pgsql)
▶ ** exponential (oracle)


                   @ngalbreath
SQL Strings, Charset & Comments

                        Such a tangled mess,
                        I defer to my DEFCON
                        20 talk:
                        http://
                        client9.com/
                        20120727/




          @ngalbreath
SQLi Detection
Keyword Detection

    s/UNION.ALL/i
► The dumbest possible regexp.
► I've used this regexp as a goof for a while,
  but oddly works well in detecting
  SQLi scans.
                                                       dow
► Almost zero false positives
                                               ore sha
                                            f



                      @ngalbreath
By Using Regular Expressions
Trying to catch more SQLi attacks leads to the question of
is user input SQLi or not? Using regular expressions you
end up with something like this:
  (?:)s*whens*d+s*then)|(?:"s*(?:#|--|{))|(?:/*!s?d+)|(?:ch(?:a)?rs*(s*d)|(?:(?:(n?and|x?or|not)s+||||&&)s*w+
  ()(?:[s()]cases*()|(?:)s*likes*()|(?:havings*[^s]+s*[^ws])|(?:ifs?([dw]s*[=<>~])(?:"s*ors*"?d)|(?:x(?:23|
  27|3d))|(?:^.?"$)|(?:(?:^["]*(?:[d"]+|[^"]+"))+s*(?:n?and|x?or|not||||&&)s*[w"[+&!@(),.-])|(?:[^ws]w+s*[|-]s*"s*
  w)|(?:@w+s+(and|or)s*["d]+)|(?:@[w-]+s(and|or)s*[^ws])|(?:[^ws:]s*dW+[^ws]s*".)|(?:Winformation_schema|
  table_nameW)(?:"s**.+(?:or|id)W*"d)|(?:^")|(?:^[ws"-]+(?<=ands)(?<=ors)(?<=xors)(?<=nands)(?<=nots)(?<=||)(?<=&
  &)w+()|(?:"[sd]*[^ws]+W*dW*.*["d])|(?:"s*[^ws?]+s*[^ws]+s*")|(?:"s*[^ws]+s*[Wd].*(?:#|--))|(?:".**s*
  d)|(?:"s*ors[^d]+[w-]+.*d)|(?:[()*<>%+-][w-]+[^ws]+"[^,])(?:d"s+"s+d)|(?:^admins*"|(/*)+"+s?(?:--|#|/*|{)?)|
  (?:"s*or[ws-]+s*[+<>=(),-]s*[d"])|(?:"s*[^ws]?=s*")|(?:"W*[+=]+W*")|(?:"s*[!=|][ds!=+-]+.*["(].*$)|(?:"s*[!=|][d
  s!=]+.*d+$)|(?:"s*likeW+[w"(])|(?:siss*0W)|(?:wheres[sw.,-]+s=)|(?:"[<>~]+")(?:unions*(?:all|distinct|[(!@]*)?
  s*[([]*s*select)|(?:w+s+likes+")|(?:likes*"%)|(?:"s*likeW*["d])|(?:"s*(?:n?and|x?or|not ||||&&)s+[sw]+=s*w+
  s*having)|(?:"s**s*w+W+")|(?:"s*[^?ws=.,;)(]+s*[(@"]*s*w+W+w)|(?:selects*[[]()sw.,"-]+from)|(?:find_in_sets*
  ()(?:ins*(+s*select)|(?:(?:n?and|x?or|not ||||&&)s+[sw+]+(?:regexps*(|soundss+likes*"|[=d]+x))|("s*ds*(?:--|
  #))|(?:"[%&<>^=]+ds*(=|or))|(?:"W+[w+-]+s*=s*dW+")|(?:"s*iss*d.+"?w)|(?:"|?[w-]{3,}[^ws.,]+")|(?:"s*iss*[d.]+
  s*W.*")(?:[dW]s+ass*["w]+s*from)|(?:^[Wd]+s*(?:union|select|create|rename|truncate|load|alter|delete|update|insert|
  desc))|(?:(?:select|create|rename|truncate|load|alter|delete|update|insert|desc)s+(?:(?:group_)concat|char|load_file)s?(?)|
  (?:ends*);)|("s+regexpW)|(?:[s(]load_files*()(?:@.+=s*(s*select)|(?:d+s*ors*d+s*[-+])|(?:/w+;?s+(?:having|and|
  or|select)W)|(?:ds+groups+by.+()|(?:(?:;|#|--)s*(?:drop|alter))|(?:(?:;|#|--)s*(?:update|insert)s*w{2,})|(?:[^w]SETs*@
  w+)|(?:(?:n?and|x?or|not ||||&&)[s(]+w+[s)]*[!=+]+[sd]*["=()])(?:"s+ands*=W)|(?:(s*selects*w+s*()|(?:*/
  from)|(?:+s*d+s*+s*@)|(?:w"s*(?:[-+=|@]+s*)+[d(])|(?:coalesces*(|@@w+s*[^ws])|(?:W!+"w)|(?:";s*(?:if|while|
  begin))|(?:"[sd]+=s*d)|(?:orders+bys+ifw*s*()|(?:[s(]+cased*W.+[tw]hen[s(])(?:(select|;)s+(?:benchmark|if|sleep)
  s*?(s*(?s*w+)(?:creates+functions+w+s+returns)|(?:;s*(?:select|create|rename|truncate|load|alter|delete|update|insert|
  desc)s*[[(]?w{2,})(?:alters*w+.*characters+sets+w+)|(";s*waitfors+times+")|(?:";.*:s*goto)(?:procedures+analyses*
  ()|(?:;s*(declare|open)s+[w-]+)|(?:creates+(procedure|function)s*w+s*(s*)s*-)|(?:declare[^w]+[@#]s*w+)|(execs*
  (s*@)(?:selects*pg_sleep)|(?:waitfors*delays?"+s?d)|(?:;s*shutdowns*(?:;|--|#|/*|{))(?:sexecs+xp_cmdshell)|(?:"s*!
  s*["w])|(?:fromW+information_schemaW)|(?:(?:(?:current_)?user|database|schema|connection_id)s*([^)]*)|(?:";?s*(?:select|
  union|having)s*[^s])|(?:wiifs*()|(?:execs+master.)|(?:union select @)|(?:union[w(s]*select)|(?:select.*w?user()|
  (?:into[s+]+(?:dump|out)files*")(?:merge.*usings*()|(executes*immediates*")|(?:W+d*s*havings*[^s-])|(?:matchs*[w(),
  +-]+s*againsts*()(?:,.*[)da-f"]"(?:".*"|Z|[^"]+))|(?:Wselect.+W*from)|((?:select|create|rename|truncate|load|alter|delete|
  update|insert|desc)s*(s*spaces*()(?:[$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|or)])(?:(sleep((s*)(d*)
  (s*))|benchmark((.*),(.*))))(?:(union(.*)select(.*)from))(?:^(-0000023456|4294967295|4294967296|2147483648|2147483647|
  0000012345|-2147483648|-2147483649|0000023456|2.2250738585072007e-308|1e309)$)




                                                     @ngalbreath
libinjection




First presented at
Black Hat USA 2012
http://client9.com/20120725   iSEC Partners party at Bellagio
libinjection
▶ Takes input and create tokens as if it SQL
▶ Compares first 5 tokens to
   "things that match SQLi"
▶ 50k+ SQLi samples, some from wild, some hand
  made, some from scans
▶ C, 100k checks per second
▶ Open Source, BSD License
▶ http://client9.com/libinjection




                  @ngalbreath
Why do we have UNION at all?




        ▶   http://www.deepthoughtsbyjackhandey.com



and what else do I, the developer,
 never use, but is commonly used
        by SQLi attackers?
               @ngalbreath
Let's use libinjection to find
   features of SQL used in
   SQLi!
► That's what I wrote in the abstract.
► Turns out to be not necessary
► Used the 50,000+ SQLi samples library from
  libinjection and ...
► ... the Awesome Power of Grep
  (well ... python regexp actually)



                     @ngalbreath
A Highly Unscientific collection of
    50,000+ SQLi attacks collected
    from actual attacks, scanners,
    how to guides, etc.
    But doesn't take into account:
►   Frequency
►   Severity
►   Uniqueness
                                                     hat!
                                              So	
  W
►   Actual successful attacks versus probes
►   Doesn't look at exfiltration techniques



                          @ngalbreath
SQL used in SQLi
       union                    75%

 comments (any type)            70%

     concat, etc                23%

 hex number literals            22%

     subselects                 22%

   chr(),char()                 6%

aes, hex, compress              4%

    SQL variables               2%

                  @ngalbreath
95% reduction in SQLi

By eliminating the following in SQL:

     ➡unions
     ➡comments
     ➡subselects

More than 95% of all SQLi samples were
prevented.
                 @ngalbreath
98+% reduction in SQLi

► By eliminating the remaining 'unusual' SQL, more than
  98% of SQLi samples were prevented or rejected.
► Remaining 2% of SQLi attacks are all equivalent of

      1 OR 1=1

► Those remaining SQLi probes are mostly annoying, and
  mostly harmless.



                      @ngalbreath
And of the course the obvious

Not represented by frequency, but all database extensions
that allow shell commands, networking calls, etc.
    •Oracle: dbms_pipe functions (networking, shell)
    •Oracle: ctxsys functions (admin info)
    •MSSQL: xp_cmdshell (shell access)
    •MSSQL: opendatasource (networking)
    •Oracle: utl_smtp (sending email)
    •You probably know of others.
Most vendors provide ways to disable these commands, or
off by default. Will focus on pure SQL.

                      @ngalbreath
Introducing SQL-RISC
    I call this simplified SQL –
    SQL that limits SQLi damage --
    SQL-RISC in honor of RISC computing.
http://en.wikipedia.org/wiki/Reduced_instruction_set_computing




                        @ngalbreath
Feasibility:
Can public applications
run using SQL-RISC?
Can we replace unions?
► Strongly suspect most? many? websites do not use
  unions.
► All unions can be rewritten into two queries.
  Minor overhead cost.
► Applications can create 'views' if absolutely required.




                     @ngalbreath
Can we remove SQL
comments?
► Uhh, I have a hard time getting developers to write any
  comments, let alone comments in SQL.
► Or just allow /* */ only at start of query
  (some ORMs generate a comment on where the query
  is coming from)




                    @ngalbreath
Can we replace SQL subselects?


► All subselects can be re-written as joins , with almost
  no application level code changes needed.
► SQLi that uses subselects can not be rewritten as joins
  (except in rare cases)




                    @ngalbreath
Can we replace SQL string
 functions?

► Including: substring, concatenation, encoding,
  encrypting, char functions, replacements
► Easy to move into application logic
► Suspect many web apps don't use these.




                    @ngalbreath
Other SQL Functions
sleep/waitfor/shutdown   3.2%    Having a hard time seeing any
in boolean mode          1.7%    need for them in public
drop|create|replace      1.6%
rand|random
                                 applications.
                         1.4%
dbms_                    1.0%
convert                  0.9%    Again, I was going to analyze
updatexml|xmltype        0.9%
generate_series          0.9%
                                 "real world" benign SQL, but
randomblob               0.8%    I've never used any of these
waitfor                  0.5%    functions, ever, in any web
extractvalue             0.5%    application.
begin                    0.5%
load_file                 0.3%
ascii                    0.2%
nvarchar                 0.1%
as binary                0.1%
into outfile              0.01%

                           @ngalbreath
Enterprise Apps
► If SQL-RISC were implemented as a separate client,
  then
  ► public apps could use SQL-RISC
  ► internal apps could use regular SQL,
    and use all functions if required.
► This would make adoption much easier.




                       @ngalbreath
SQL-RISC
Benefits
Fixing SQLi the Old Way

► Ensuring that every user input is
  properly validated is intractable
  (true, some frameworks help, but ....
     only if you are using them)
► Parameterized queries helps but some common SQL
  expressions cannot be expressed with
  parameterization (e.g. IN lists)
► Auditing is very slow
► Every code change may introduce new problem.


                  @ngalbreath
Fixing SQLi using SQL-RISC
► Using SQL-RISC may some require some application
  changes, however, this is
  a greppable finite problem.
  find . -name '*.php' | 
        xargs grep -i union
► Feasibility, conversion & testing can be done before
  deployment of SQL-RISC.
► Once complete, the entire application, current and
  future is protected.




                    @ngalbreath
Auto-Detecting Attacks

By using SQL-RISC, critical SQLi features are de-activated.

           SQLi Attacks
             turn into
         SQL Syntax Errors


                      @ngalbreath
SQL Syntax Errors are
   Easy to Monitor
    ► SQL Syntax errors are annoying but harmless, and are put into
      logs (database and/or application)
    ► Trivial to monitor using existing tools
       (e.g. grep -i 'syntax error' *.log )
    ► Now you know where input validation isn't being done


▶ You should be monitoring SQL
  syntax errors anyways! Major alert
  if you are being scanner!

                         @ngalbreath
But won't attackers just use
 another part of SQL?
Of course they will!
This is not a 100% solution but provides a good start on
security by default.
Possible pathways is not unlimited. Maybe I missed
something, but then it's likely to be some 'exotic' part of
SQL
The more exotic the SQL, the easier it is to detect.




                        @ngalbreath
Next Steps




             食品サンプル
Proof of Concept Patch
► Making a source code patch to deactivating functions
  and features should be a relatively simple task.
► May be possible to produce a binary patch:
  perl -p -i -e mysqld
       's/UNION/blah/g'




                   @ngalbreath
Access Control
► However, best done via access control.
► Different clients could enable or disable SQL functions
  and features depending on need.
► This is a more complicated patch!




                    @ngalbreath
Closed-Source

► I challenge you!
► Provide controls for 'advanced' SQL features or provide
  a simplified parser option.




                      @ngalbreath
Let's Eat!
 ► Help wanted!
 ► contact me nickg@client9.com
 ► libinjection home page:
   http://client9.com/libinjection


    http://
    client9.com
    /20130227/
 @ngalbreath

More Related Content

KEY
libinjection: new technique in detecting SQLi attacks, iSEC Partners Open Forum
KEY
libinjection and sqli obfuscation, presented at OWASP NYC
PDF
libinjection: from SQLi to XSS  by Nick Galbreath
KEY
New techniques in sql obfuscation, from DEFCON 20
KEY
libinjection: a C library for SQLi detection, from Black Hat USA 2012
PDF
Finding Needles in Haystacks
PDF
Ruxmon feb 2013 what happened to rails
PDF
Ruxmon cve 2012-2661
libinjection: new technique in detecting SQLi attacks, iSEC Partners Open Forum
libinjection and sqli obfuscation, presented at OWASP NYC
libinjection: from SQLi to XSS  by Nick Galbreath
New techniques in sql obfuscation, from DEFCON 20
libinjection: a C library for SQLi detection, from Black Hat USA 2012
Finding Needles in Haystacks
Ruxmon feb 2013 what happened to rails
Ruxmon cve 2012-2661

What's hot (20)

PDF
Ln monitoring repositories
PDF
Owasp tds
PDF
Defcon CTF quals
PDF
Harder Faster Stronger
PDF
Entomology 101
PDF
Carlo Sciolla - Above and beyond type systems with clojure.spec - Codemotion ...
PDF
Php Debugging from the Trenches
PDF
PHP7 - The New Engine for old good train
PPTX
Old code doesn't stink
PPTX
Vulnerabilities in data processing levels
PDF
Design patterns
PDF
Paris Web - Javascript as a programming language
PDF
The secret of PHP7's Performance
PPT
Building a Testable Data Access Layer
PDF
Survive JavaScript - Strategies and Tricks
PDF
Exception Handling: Designing Robust Software in Ruby
PDF
How to analyze your codebase with Exakat using Docker - Longhorn PHP
PDF
Php7 傳說中的第七隻大象
PDF
Zen of Akka
PDF
[HES2013] Virtually secure, analysis to remote root 0day on an industry leadi...
Ln monitoring repositories
Owasp tds
Defcon CTF quals
Harder Faster Stronger
Entomology 101
Carlo Sciolla - Above and beyond type systems with clojure.spec - Codemotion ...
Php Debugging from the Trenches
PHP7 - The New Engine for old good train
Old code doesn't stink
Vulnerabilities in data processing levels
Design patterns
Paris Web - Javascript as a programming language
The secret of PHP7's Performance
Building a Testable Data Access Layer
Survive JavaScript - Strategies and Tricks
Exception Handling: Designing Robust Software in Ruby
How to analyze your codebase with Exakat using Docker - Longhorn PHP
Php7 傳說中的第七隻大象
Zen of Akka
[HES2013] Virtually secure, analysis to remote root 0day on an industry leadi...
Ad

Similar to SQL-RISC: New Directions in SQLi Prevention - RSA USA 2013 (20)

ODP
Vote NO for MySQL
PDF
Rafael Bagmanov «Scala in a wild enterprise»
PPTX
MySQL Without the SQL -- Oh My! Longhorn PHP Conference
PPTX
Empowering the AWS DynamoDB™ application developer with Alternator
PDF
Catalyst - refactor large apps with it and have fun!
ODP
MySQL Without the MySQL -- Oh My!
PDF
Scala in a wild enterprise
PDF
Ohio Linux Fest -- MySQL's NoSQL
PPSX
Vote Early, Vote Often: From Napkin to Canvassing Application in a Single Wee...
PDF
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015
PDF
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
PDF
Oracle's Take On NoSQL
PDF
MySQL up and running 30 minutes.pdf
PDF
Buildingsocialanalyticstoolwithmongodb
PDF
PostgreSQL - масштабирование в моде, Valentine Gogichashvili (Zalando SE)
PPTX
Power of linked list
PDF
The Future of Fast Databases: Lessons from a Decade of QuestDB
PDF
Making It To Veteren Cassandra Status
PPTX
ETL with SPARK - First Spark London meetup
PDF
Avoiding GraphQL insecurities with OWASP SKF - OWASP HU meetup
Vote NO for MySQL
Rafael Bagmanov «Scala in a wild enterprise»
MySQL Without the SQL -- Oh My! Longhorn PHP Conference
Empowering the AWS DynamoDB™ application developer with Alternator
Catalyst - refactor large apps with it and have fun!
MySQL Without the MySQL -- Oh My!
Scala in a wild enterprise
Ohio Linux Fest -- MySQL's NoSQL
Vote Early, Vote Often: From Napkin to Canvassing Application in a Single Wee...
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
Oracle's Take On NoSQL
MySQL up and running 30 minutes.pdf
Buildingsocialanalyticstoolwithmongodb
PostgreSQL - масштабирование в моде, Valentine Gogichashvili (Zalando SE)
Power of linked list
The Future of Fast Databases: Lessons from a Decade of QuestDB
Making It To Veteren Cassandra Status
ETL with SPARK - First Spark London meetup
Avoiding GraphQL insecurities with OWASP SKF - OWASP HU meetup
Ad

More from Nick Galbreath (12)

PDF
Making operations visible - devopsdays tokyo 2013
PDF
Faster Secure Software Development with Continuous Deployment - PH Days 2013
PDF
Fixing security by fixing software development
PDF
DevOpsDays Austin 2013 Reading List
PDF
Rebooting Software Development - OWASP AppSecUSA
KEY
Continuous Deployment - The New #1 Security Feature, from BSildesLA 2012
KEY
Time tested php with libtimemachine
PPT
Data Driven Security, from Gartner Security Summit 2012
KEY
Slide show font sampler, black on white
PPTX
Fraud Engineering, from Merchant Risk Council Annual Meeting 2012
KEY
Rate Limiting at Scale, from SANS AppSec Las Vegas 2012
KEY
DevOpsSec: Appling DevOps Principles to Security, DevOpsDays Austin 2012
Making operations visible - devopsdays tokyo 2013
Faster Secure Software Development with Continuous Deployment - PH Days 2013
Fixing security by fixing software development
DevOpsDays Austin 2013 Reading List
Rebooting Software Development - OWASP AppSecUSA
Continuous Deployment - The New #1 Security Feature, from BSildesLA 2012
Time tested php with libtimemachine
Data Driven Security, from Gartner Security Summit 2012
Slide show font sampler, black on white
Fraud Engineering, from Merchant Risk Council Annual Meeting 2012
Rate Limiting at Scale, from SANS AppSec Las Vegas 2012
DevOpsSec: Appling DevOps Principles to Security, DevOpsDays Austin 2012

Recently uploaded (20)

PDF
Approach and Philosophy of On baking technology
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Machine learning based COVID-19 study performance prediction
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
gpt5_lecture_notes_comprehensive_20250812015547.pdf
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PPTX
A Presentation on Artificial Intelligence
PDF
Assigned Numbers - 2025 - Bluetooth® Document
PDF
Empathic Computing: Creating Shared Understanding
PDF
Heart disease approach using modified random forest and particle swarm optimi...
PPTX
SOPHOS-XG Firewall Administrator PPT.pptx
PDF
A comparative analysis of optical character recognition models for extracting...
PDF
Getting Started with Data Integration: FME Form 101
PPTX
TLE Review Electricity (Electricity).pptx
Approach and Philosophy of On baking technology
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Programs and apps: productivity, graphics, security and other tools
Machine learning based COVID-19 study performance prediction
Diabetes mellitus diagnosis method based random forest with bat algorithm
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
gpt5_lecture_notes_comprehensive_20250812015547.pdf
MIND Revenue Release Quarter 2 2025 Press Release
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Building Integrated photovoltaic BIPV_UPV.pdf
A Presentation on Artificial Intelligence
Assigned Numbers - 2025 - Bluetooth® Document
Empathic Computing: Creating Shared Understanding
Heart disease approach using modified random forest and particle swarm optimi...
SOPHOS-XG Firewall Administrator PPT.pptx
A comparative analysis of optical character recognition models for extracting...
Getting Started with Data Integration: FME Form 101
TLE Review Electricity (Electricity).pptx

SQL-RISC: New Directions in SQLi Prevention - RSA USA 2013

  • 1. SQL-RISC New Directions in SQLi Prevention Nick Galbreath RSA 2013-02-27
  • 2. SQL-RISC: NEW DIRECTIONS IN SQLI DETECTION AND PREVENTION Nick Galbreath IPONWEB ► Session ID: ASEC-­‐W23 ► Session Classification: Advanced
  • 3. Latest version of these slides: http://client9.com/20130227/ Tweet tweet: @ngalbreath Originally presented at: RSA Conference USA Moscone Center San Francisco, California February 27, 2013 10:40 AM, Room 132
  • 4. What if we could substantially reduce the SQLi attack surface of a web application? @ngalbreath
  • 5. without new hardware or firewalls? @ngalbreath
  • 7. and comes with free SQLi attack monitoring? @ngalbreath
  • 8. Wild Speculation? @ngalbreath
  • 10. 1970s: SQL Invented ► Hey that's 40 years ago. ► "Structured Query Language" ► Why is it still around and so popular? ► Exercise for the reader: Pick any query you like, and write the equivalent in your favorite programming language. ► SQL is scriptable data structures. @ngalbreath
  • 11. 1972: Oracle Releases the First Commercial Database Also, Coppola releases The Godfather @ngalbreath
  • 12. Remember the 80s? ► Networking sucked! ► TCP/IP not widespread ► Computers are fragile, expensive and slow ► Shoulder-pads so what do you do? @ngalbreath
  • 13. Centralize ► Move computation as close as possible to the data ► Move to super-servers ► Have cheap/dumb clients @ngalbreath
  • 14. 1988: Oracle V6 ► The Database is now a full programming environment ► Unicode/UTF8 not standard ► Complexity explosion ► ... but most clients are private Also in 1988: Crack Cocaine 'invented' @ngalbreath
  • 15. And then the 1990s ► TCP/IP ► Cheap CPUs ► Web Browsers ► Attaching databases to public networks This is why most of us are here today. @ngalbreath
  • 16. 2000+ Web Scale ► Discovery that data problems are a lot more painful than CPU problems. ► Turns out disk drives are mechanical ► If your database maxes out, you have big problems... so move everything out of the database. ► Complete reversal in strategy @ngalbreath
  • 17. SQL isn't going anywhere ► For front-ends, general trend is federating data across cheap machines, using stripped down SQL ► SQL-like languages used by Amazon, Google and others. ► Still great for analytics and reporting on the back end, and generic data storage. ► But we stuck with legacy of the past. @ngalbreath
  • 18. SQL'S DARK CORNERS ► bit.ly/UkQd7Z
  • 19. SQL is Huge ► 40+ years of built up crud ► 1992 spec is 625 pages of plain text ► 2003 spec is 128 pages of BNF ► No one is completely compliant ► Every one has special extensions SQL is more complicated than you think.... @ngalbreath
  • 20. SQL Integer Forms • [0-9]+ • 0x[0-9a-fA-F]+ 0xDEADbeef MySQL, MSSQL 0x is case sensitive • 0x MSSQL only • x'DEADbeef' PgSQL • b'10101010' MySQL, PgSQL • 0b010101 MySQL @ngalbreath
  • 21. SQL Floating Point Forms ► digits ► digits[.] ► digits[.]digits ► digits[eE]digits ► digits[eE][+-]digits ► digits[.][eE]digits ► digits[.]digits[eE][+-]digits ► http://bit.ly/Qp6KTu ► digits[.]digits[eE]digits ► [.]digits ► [.]digits[eE]digits ► [.]digits[eE][+-]digits Optional starts with [+-] ► "binary_float_infinity" (O) Optional ending with [dDfF] (Oracle) @ngalbreath
  • 22. SQL Money Literals ► MSSQL has a money type. ► -$45.12 ► $123.0 ► +$1,000,000.00 Commas ignored ► Many symbols are accepted for currency type @ngalbreath
  • 23. SQL Ridiculous Operators ▶ != not equals, standard ▶ <=> mysql ▶ <> mssql ▶ ^= oracle ▶ !>, !< not less than mssql ▶ / oracle ▶ !! factorial (pgsql) ▶ |/ square root (pgsql) ▶ ||/ cube root (pgsql) ▶ ** exponential (oracle) @ngalbreath
  • 24. SQL Strings, Charset & Comments Such a tangled mess, I defer to my DEFCON 20 talk: http:// client9.com/ 20120727/ @ngalbreath
  • 26. Keyword Detection s/UNION.ALL/i ► The dumbest possible regexp. ► I've used this regexp as a goof for a while, but oddly works well in detecting SQLi scans. dow ► Almost zero false positives ore sha f @ngalbreath
  • 27. By Using Regular Expressions Trying to catch more SQLi attacks leads to the question of is user input SQLi or not? Using regular expressions you end up with something like this: (?:)s*whens*d+s*then)|(?:"s*(?:#|--|{))|(?:/*!s?d+)|(?:ch(?:a)?rs*(s*d)|(?:(?:(n?and|x?or|not)s+||||&&)s*w+ ()(?:[s()]cases*()|(?:)s*likes*()|(?:havings*[^s]+s*[^ws])|(?:ifs?([dw]s*[=<>~])(?:"s*ors*"?d)|(?:x(?:23| 27|3d))|(?:^.?"$)|(?:(?:^["]*(?:[d"]+|[^"]+"))+s*(?:n?and|x?or|not||||&&)s*[w"[+&!@(),.-])|(?:[^ws]w+s*[|-]s*"s* w)|(?:@w+s+(and|or)s*["d]+)|(?:@[w-]+s(and|or)s*[^ws])|(?:[^ws:]s*dW+[^ws]s*".)|(?:Winformation_schema| table_nameW)(?:"s**.+(?:or|id)W*"d)|(?:^")|(?:^[ws"-]+(?<=ands)(?<=ors)(?<=xors)(?<=nands)(?<=nots)(?<=||)(?<=& &)w+()|(?:"[sd]*[^ws]+W*dW*.*["d])|(?:"s*[^ws?]+s*[^ws]+s*")|(?:"s*[^ws]+s*[Wd].*(?:#|--))|(?:".**s* d)|(?:"s*ors[^d]+[w-]+.*d)|(?:[()*<>%+-][w-]+[^ws]+"[^,])(?:d"s+"s+d)|(?:^admins*"|(/*)+"+s?(?:--|#|/*|{)?)| (?:"s*or[ws-]+s*[+<>=(),-]s*[d"])|(?:"s*[^ws]?=s*")|(?:"W*[+=]+W*")|(?:"s*[!=|][ds!=+-]+.*["(].*$)|(?:"s*[!=|][d s!=]+.*d+$)|(?:"s*likeW+[w"(])|(?:siss*0W)|(?:wheres[sw.,-]+s=)|(?:"[<>~]+")(?:unions*(?:all|distinct|[(!@]*)? s*[([]*s*select)|(?:w+s+likes+")|(?:likes*"%)|(?:"s*likeW*["d])|(?:"s*(?:n?and|x?or|not ||||&&)s+[sw]+=s*w+ s*having)|(?:"s**s*w+W+")|(?:"s*[^?ws=.,;)(]+s*[(@"]*s*w+W+w)|(?:selects*[[]()sw.,"-]+from)|(?:find_in_sets* ()(?:ins*(+s*select)|(?:(?:n?and|x?or|not ||||&&)s+[sw+]+(?:regexps*(|soundss+likes*"|[=d]+x))|("s*ds*(?:--| #))|(?:"[%&<>^=]+ds*(=|or))|(?:"W+[w+-]+s*=s*dW+")|(?:"s*iss*d.+"?w)|(?:"|?[w-]{3,}[^ws.,]+")|(?:"s*iss*[d.]+ s*W.*")(?:[dW]s+ass*["w]+s*from)|(?:^[Wd]+s*(?:union|select|create|rename|truncate|load|alter|delete|update|insert| desc))|(?:(?:select|create|rename|truncate|load|alter|delete|update|insert|desc)s+(?:(?:group_)concat|char|load_file)s?(?)| (?:ends*);)|("s+regexpW)|(?:[s(]load_files*()(?:@.+=s*(s*select)|(?:d+s*ors*d+s*[-+])|(?:/w+;?s+(?:having|and| or|select)W)|(?:ds+groups+by.+()|(?:(?:;|#|--)s*(?:drop|alter))|(?:(?:;|#|--)s*(?:update|insert)s*w{2,})|(?:[^w]SETs*@ w+)|(?:(?:n?and|x?or|not ||||&&)[s(]+w+[s)]*[!=+]+[sd]*["=()])(?:"s+ands*=W)|(?:(s*selects*w+s*()|(?:*/ from)|(?:+s*d+s*+s*@)|(?:w"s*(?:[-+=|@]+s*)+[d(])|(?:coalesces*(|@@w+s*[^ws])|(?:W!+"w)|(?:";s*(?:if|while| begin))|(?:"[sd]+=s*d)|(?:orders+bys+ifw*s*()|(?:[s(]+cased*W.+[tw]hen[s(])(?:(select|;)s+(?:benchmark|if|sleep) s*?(s*(?s*w+)(?:creates+functions+w+s+returns)|(?:;s*(?:select|create|rename|truncate|load|alter|delete|update|insert| desc)s*[[(]?w{2,})(?:alters*w+.*characters+sets+w+)|(";s*waitfors+times+")|(?:";.*:s*goto)(?:procedures+analyses* ()|(?:;s*(declare|open)s+[w-]+)|(?:creates+(procedure|function)s*w+s*(s*)s*-)|(?:declare[^w]+[@#]s*w+)|(execs* (s*@)(?:selects*pg_sleep)|(?:waitfors*delays?"+s?d)|(?:;s*shutdowns*(?:;|--|#|/*|{))(?:sexecs+xp_cmdshell)|(?:"s*! s*["w])|(?:fromW+information_schemaW)|(?:(?:(?:current_)?user|database|schema|connection_id)s*([^)]*)|(?:";?s*(?:select| union|having)s*[^s])|(?:wiifs*()|(?:execs+master.)|(?:union select @)|(?:union[w(s]*select)|(?:select.*w?user()| (?:into[s+]+(?:dump|out)files*")(?:merge.*usings*()|(executes*immediates*")|(?:W+d*s*havings*[^s-])|(?:matchs*[w(), +-]+s*againsts*()(?:,.*[)da-f"]"(?:".*"|Z|[^"]+))|(?:Wselect.+W*from)|((?:select|create|rename|truncate|load|alter|delete| update|insert|desc)s*(s*spaces*()(?:[$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|or)])(?:(sleep((s*)(d*) (s*))|benchmark((.*),(.*))))(?:(union(.*)select(.*)from))(?:^(-0000023456|4294967295|4294967296|2147483648|2147483647| 0000012345|-2147483648|-2147483649|0000023456|2.2250738585072007e-308|1e309)$) @ngalbreath
  • 28. libinjection First presented at Black Hat USA 2012 http://client9.com/20120725 iSEC Partners party at Bellagio
  • 29. libinjection ▶ Takes input and create tokens as if it SQL ▶ Compares first 5 tokens to "things that match SQLi" ▶ 50k+ SQLi samples, some from wild, some hand made, some from scans ▶ C, 100k checks per second ▶ Open Source, BSD License ▶ http://client9.com/libinjection @ngalbreath
  • 30. Why do we have UNION at all? ▶ http://www.deepthoughtsbyjackhandey.com and what else do I, the developer, never use, but is commonly used by SQLi attackers? @ngalbreath
  • 31. Let's use libinjection to find features of SQL used in SQLi! ► That's what I wrote in the abstract. ► Turns out to be not necessary ► Used the 50,000+ SQLi samples library from libinjection and ... ► ... the Awesome Power of Grep (well ... python regexp actually) @ngalbreath
  • 32. A Highly Unscientific collection of 50,000+ SQLi attacks collected from actual attacks, scanners, how to guides, etc. But doesn't take into account: ► Frequency ► Severity ► Uniqueness hat! So  W ► Actual successful attacks versus probes ► Doesn't look at exfiltration techniques @ngalbreath
  • 33. SQL used in SQLi union 75% comments (any type) 70% concat, etc 23% hex number literals 22% subselects 22% chr(),char() 6% aes, hex, compress 4% SQL variables 2% @ngalbreath
  • 34. 95% reduction in SQLi By eliminating the following in SQL: ➡unions ➡comments ➡subselects More than 95% of all SQLi samples were prevented. @ngalbreath
  • 35. 98+% reduction in SQLi ► By eliminating the remaining 'unusual' SQL, more than 98% of SQLi samples were prevented or rejected. ► Remaining 2% of SQLi attacks are all equivalent of 1 OR 1=1 ► Those remaining SQLi probes are mostly annoying, and mostly harmless. @ngalbreath
  • 36. And of the course the obvious Not represented by frequency, but all database extensions that allow shell commands, networking calls, etc. •Oracle: dbms_pipe functions (networking, shell) •Oracle: ctxsys functions (admin info) •MSSQL: xp_cmdshell (shell access) •MSSQL: opendatasource (networking) •Oracle: utl_smtp (sending email) •You probably know of others. Most vendors provide ways to disable these commands, or off by default. Will focus on pure SQL. @ngalbreath
  • 37. Introducing SQL-RISC I call this simplified SQL – SQL that limits SQLi damage -- SQL-RISC in honor of RISC computing. http://en.wikipedia.org/wiki/Reduced_instruction_set_computing @ngalbreath
  • 39. Can we replace unions? ► Strongly suspect most? many? websites do not use unions. ► All unions can be rewritten into two queries. Minor overhead cost. ► Applications can create 'views' if absolutely required. @ngalbreath
  • 40. Can we remove SQL comments? ► Uhh, I have a hard time getting developers to write any comments, let alone comments in SQL. ► Or just allow /* */ only at start of query (some ORMs generate a comment on where the query is coming from) @ngalbreath
  • 41. Can we replace SQL subselects? ► All subselects can be re-written as joins , with almost no application level code changes needed. ► SQLi that uses subselects can not be rewritten as joins (except in rare cases) @ngalbreath
  • 42. Can we replace SQL string functions? ► Including: substring, concatenation, encoding, encrypting, char functions, replacements ► Easy to move into application logic ► Suspect many web apps don't use these. @ngalbreath
  • 43. Other SQL Functions sleep/waitfor/shutdown 3.2% Having a hard time seeing any in boolean mode 1.7% need for them in public drop|create|replace 1.6% rand|random applications. 1.4% dbms_ 1.0% convert 0.9% Again, I was going to analyze updatexml|xmltype 0.9% generate_series 0.9% "real world" benign SQL, but randomblob 0.8% I've never used any of these waitfor 0.5% functions, ever, in any web extractvalue 0.5% application. begin 0.5% load_file 0.3% ascii 0.2% nvarchar 0.1% as binary 0.1% into outfile 0.01% @ngalbreath
  • 44. Enterprise Apps ► If SQL-RISC were implemented as a separate client, then ► public apps could use SQL-RISC ► internal apps could use regular SQL, and use all functions if required. ► This would make adoption much easier. @ngalbreath
  • 46. Fixing SQLi the Old Way ► Ensuring that every user input is properly validated is intractable (true, some frameworks help, but .... only if you are using them) ► Parameterized queries helps but some common SQL expressions cannot be expressed with parameterization (e.g. IN lists) ► Auditing is very slow ► Every code change may introduce new problem. @ngalbreath
  • 47. Fixing SQLi using SQL-RISC ► Using SQL-RISC may some require some application changes, however, this is a greppable finite problem. find . -name '*.php' | xargs grep -i union ► Feasibility, conversion & testing can be done before deployment of SQL-RISC. ► Once complete, the entire application, current and future is protected. @ngalbreath
  • 48. Auto-Detecting Attacks By using SQL-RISC, critical SQLi features are de-activated. SQLi Attacks turn into SQL Syntax Errors @ngalbreath
  • 49. SQL Syntax Errors are Easy to Monitor ► SQL Syntax errors are annoying but harmless, and are put into logs (database and/or application) ► Trivial to monitor using existing tools (e.g. grep -i 'syntax error' *.log ) ► Now you know where input validation isn't being done ▶ You should be monitoring SQL syntax errors anyways! Major alert if you are being scanner! @ngalbreath
  • 50. But won't attackers just use another part of SQL? Of course they will! This is not a 100% solution but provides a good start on security by default. Possible pathways is not unlimited. Maybe I missed something, but then it's likely to be some 'exotic' part of SQL The more exotic the SQL, the easier it is to detect. @ngalbreath
  • 51. Next Steps 食品サンプル
  • 52. Proof of Concept Patch ► Making a source code patch to deactivating functions and features should be a relatively simple task. ► May be possible to produce a binary patch: perl -p -i -e mysqld 's/UNION/blah/g' @ngalbreath
  • 53. Access Control ► However, best done via access control. ► Different clients could enable or disable SQL functions and features depending on need. ► This is a more complicated patch! @ngalbreath
  • 54. Closed-Source ► I challenge you! ► Provide controls for 'advanced' SQL features or provide a simplified parser option. @ngalbreath
  • 55. Let's Eat! ► Help wanted! ► contact me nickg@client9.com ► libinjection home page: http://client9.com/libinjection http:// client9.com /20130227/ @ngalbreath