Java security in the real world (Ryan Sciampacone)

Ryan Sciampacone – IBM Java Runtime Lead
3rd October 2012

Security in the Real World

© 2012 IBM Corporation

Important Disclaimers

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR
INFORMATIONAL PURPOSES ONLY.
WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF
THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”,
WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN
A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON
HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.
ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A
GUIDE.
IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON
IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE
BY IBM, WITHOUT NOTICE.
IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES
ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR
ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE
EFFECT OF:
- CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED
COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS
2 © 2012 IBM Corporation

Introduction to the speaker

■ Ryan Sciampacone
■ 15 years experience developing and deploying Java SDKs
■ Recent work focus:
■ Managed Runtime Architecture
■ Java Virtual Machine improvements
■ Multi-tenancy technology
■ Native data access and heap density
■ Footprint and performance
■ Garbage Collection
■ Scalability and pause time reduction
■ Advanced GC technology
■ My contact information:
– Ryan_Sciampacone@ca.ibm.com


What should you get from this talk?

■ You should have a clearer picture of some of the attack vectors that have been
used recently in Java and what steps were taken to address them. You should
also understand the current state of security in Java, and how both Java class
libraries and the JVM work hard to keep developer lives simple.


The problem with keeping anything secure

■ "The only secure computer is one that's unplugged, locked in a safe,
and buried 20 feet under the ground in a secret location... and I'm
not even too sure about that one"
-- (attributed) Dennis Huges, FBI.

■ Security isn’t just a padlock on a door
■ A complex system will have many attack vectors
■ Key is keeping the system
– Stable
– Performant
– Secure
■ All while keeping easy development accessible to developers


Security that doesn’t interfere with the system

■ Java and the Java Virtual Machine provide defense in depth
– Class loaders
– Verification
– Access Controller / Security Manager
– Java Cryptography Extensions (JCE)
– Java Secure Sockets Extension (JSSE)
– Java Authentication and Authorization Service (JAAS)

Reference: http://en.wikipedia.org/wiki/Riot_control

■ Available implicit or explicitly during development / deployment
■ Security is expected to be a trusted resource
– It just works
– It has been verified (thoroughly) by vendors
■ Be aware of what isn’t secured!
■ Key: Avoiding having to build (and verify!) your own security layers


Security Layers in Java

■ Some things you get “for free”
■ Others you use when you need them
Diagram Reference: Java Security, Scott Oaks, O’Reilly Media, May 24, 2001, Second Edition, ISBN-10: 0596001576, ISBN-13: 978-0596001575


Hashing Denial-of-Service Attack (CVE-2011-4858)


Hashing Denial-of-Service Attack

■ String hash codes and hashing structures have been around “for ever”
■ A combination of
– Performance short comings
– Documented / predictable behavior
■ Can be used to exploit vulnerabilities in existing software
■ Algorithmic Complexity Attack


Hashing Denial-of-Service Attack – How String Hashing Works

■ String hashing algorithm is well known and reversible

http://docs.oracle.com/javase/7/docs/api/java/lang/String.html

■ Easy to construct strings that have identical hash codes

== 2112

== 2031744

http://stackoverflow.com/questions/8669946/application-vulnerability-due-to-non-random-hash-functions


Hashing Denial-of-Service Attack – How Hashing Structures Work

HashMap



HashMap

Array to hold the various
<key,value> pairs



HashMap
Use the hash code for
“QuantityAa”
to find a location in Array to hold the various
the array <key,value> pairs



HashMap

Find the appropriate
“bucket” and add the < “QuantityAa”, “1234” >
entry



HashMap

< “QuantityAa”, “1234” >



HashMap


< “QuantityBB”, “987” >



HashMap



Warning: Lookup / Insertion
requires a string comparison!!!



HashMap



Warning: Lookup requires a string
comparison!!!

■ Keys with identical hashes will always fall into the same bucket


Hashing Denial-of-Service Attack – Strings as Keys in Hashing Structures

■ Deep buckets with malicious keys can cause serious performance issues

HashMap

< “AaAaAaAaAa … AaAaAa”, “1234” >

< “AaAaAaAaAa … AaAaBB”, “987” >

Near duplicate string with difference at the end


Hashing Denial-of-Service Attack – Result

■ Websites make use of parameters as part of client / server communication

■ Server is responsible for managing the parameters for the servlet
– Hash structures typical way of managing these <key,value> pairs

■ Issue: Long insert / lookup times for parameters that have high hash collision rate

Reference: http://www.nruns.com/_downloads/advisory28122011.pdf

■ Result: Web servers could be effectively “disabled” with simple requests


Hashing Denial-of-Service Attack – Current Solution

■ Hashing structures now use an alternate hash code for Strings
– Use alternate only at a certain capacity
– Algorithm where the hash code cannot be calculated externally

■ Why not modify String.hashCode()?
– It’s spec!
– Reliance in existing software

■ NOTE: With alternate hash, iteration order is now changed!
– Spec’d as “unspecified”
– Doesn’t matter – code relies on this anyways
– Solution can cause existing working software to fail!


Hashing Denial-of-Service Attack – Current Solution

■ The JVM now supports a system property to enable at thresholds
-Djdk.map.althashing.threshold=<threshold>
■ Apache Tomcat property maxParameterCount to limit number of parameters


Gondvv Vulnerability (CVE-2012-4681)


Java Security Manager Bypass (Gondvv) Vulnerability

■ Imagine visiting a website and your calculator application pops up

■ How did that happen?

■ Arbitrary code has been run on your machine – how compromised are you?


Java Security Manager Bypass (Gondvv) Vulnerability – Key Change

■ A simple access modifier change (within a larger change) exposed a vulnerability




Set the security permissions to that
of the current code (privileged) in
place of the callers security permissions




Use reflection to acquire a Field object
on the given class




Set the reflect object Field usage to
ignore access checks. Privileged action
permitted through doPrivileged()


Java Security Manager Bypass (Gondvv) Vulnerability – In Action

com.sun.beans.finder
ClassFinder



ClassFinder

findClass()

sun.awt
SunToolkit



ClassFinder

java.beans
findClass() Statement

sun.awt “setSecurityManager()”
SunToolkit
AccessControlContext



ClassFinder

java.beans

SunToolkit
getField()



ClassFinder

java.beans

SunToolkit
getField()

java.lang.reflect
Field



ClassFinder

java.beans

SunToolkit
getField()
AccessControlContext Elevated permissions
for statement

set()
java.lang.reflect
Field



ClassFinder

java.beans

SunToolkit
getField()
for statement

set()
java.lang.reflect
Field



ClassFinder

java.beans
execute() Elevated permissions
for sandbox
SunToolkit
getField()
for statement

set()
java.lang.reflect
Field



ClassFinder

java.beans
for sandbox
SunToolkit
getField()
for statement

set()
java.lang.reflect
Field

java.lang
Runtime



ClassFinder

java.beans
for sandbox
SunToolkit
getField()
for statement

set()
java.lang.reflect
Field

java.lang
Runtime
exec(“…”)



ClassFinder

java.beans
for sandbox
SunToolkit
getField()
for statement

set()
java.lang.reflect
Field

java.lang
Runtime
exec(“…”)

Ref: http://en.wikipedia.org/wiki/Castle
Ref: http://en.wikipedia.org/wiki/Key_(lock)


Java Security Manager Bypass (Gondvv) Vulnerability – Epilogue

■ Need to be running untrusted code
■ Java7 VM required
– Thankfully, most systems were still running Java6
■ Simple change to an access modifier

■ NOTE: A fix was turned around in very short order


Invokespecial Security Fix (CVE-2012-1725)


Bytecode Verifier

■ Java Language Security
– Rule enforcement at compile, verification (typically load), runtime
■ You want to trust that the code you are running (foreign or otherwise) is still valid
– Not accessing things that it shouldn’t
– Not bypassing rules that might violate the guarantee of the integrity of the JVM (or
platform) as a whole
■ Levels of integrity checking on a class file
– Valid bytecode sequences
– Arguments, local variables and intermediate values are correctly typed
– Enforces access protection rules (public, private, protected, package protected)
■ Big one: Accessing arbitrary memory
– Casting an int to an Object and then dereferencing
■ Of course, native code can’t be helped here!
■ The verifier function and security has changed (and improved) over Java releases


Bytecode Verifier – Invokespecial security fix

■ Verifier now enforces that either another constructor on the same class
this(…)
or a direct superclass is called
super(…)
■ May affect dynamic proxies that have previously “cheated”
– Skipped creating intermediate classes

static class Child extends Throwable {
public Child() {
0: aload_0
1: invokespecial #8 // calls method java/lang/Object."<init>":()V
// should be method java/lang/Throwable."<init>":()V
4: return
}
}



■ Example: Bypassing important setup and security measures




Per instance ID for work and privilege purposes




Control point – privileged action if you are “blessed”




Bypassing the init!




Bypassing the init!

Can’t express this in Java syntax – must be generated


Method Handles


Method Handles

■ JSR 292: Supporting Dynamically Typed Languages on the JavaTM Platform
– A new bytecode for custom dynamic linkage (invokedynamic)
– MethodHandle (and support classes) as a “function pointer” interface for linkage

■ Fast invocation of bound methods
– Method handle invocation speed can be far superior to reflect methods

■ A MethodHandle resembles java.lang.reflect.Method
– Access checking is performed at lookup, not at every call
– Conversion available from reflection side to MethodHandle types


Method Handles – Access and Security Checks

Reflection MethodHandles
SecurityManager checks at lookup Yes Yes
Access checks at lookup No Yes
Access checks at invocation Yes No
Checks at setAccessible(true) Yes N/A
Anyone can invoke? No: by default Yes – by default
Yes: setAccessible(true)


Method Handles – Security Where It Matters


Method Handles – A Word of Caution

■ The lookup mechanism has interesting privilege characteristics
– Be careful about what code has access to it


Bytecode Verification and Speed


Bytecode Verifier – StackMapTable Attribute

■ Verify that jump targets agree on stack contents




Complex / Interesting Points!




Frame 1

Frame 2

Frame 3

Frame 4




Frame 1
StackMapTable
Frame 2 Frame 1
Frame 2
Frame 3
Frame 3 Frame 4

Frame 4

■ Typically a (slow) flow based walk
■ V.50+ (Java6+) Class Files now contain basic block maps for a (fast) linear walk



■ May seem trivial but harder problems exist
■ Exceptions!

■ And almost everything generates it’s own bytecodes now…



■ StackMapTable attribute speeds up verification
– Provides a “proof” that the typechecking verifier checks if the stack matches.
■ Requirements
– Mandatory for class file v.51+ (Java7 compiled)
– Optional for v.50 with fallback to old type inference verifier
■ Possible speed improvement on class loading? (startup times)


Class Loaders and Spoofing


Class Loaders

■ Part of the Java “Sandbox”
– Offers isolation between groups of classes at level of choosing
■ Programmatic way of specifying where your classes come from
■ Name space
■ Opportunities for data de-duplication (Shared Classes)
■ Useful as part of a module system


Class Loaders – Class Spoofing

■ Duplicate named classes are completely valid within a JVM
– Visibility creates a namespace
– Each is in fact a distinct type

ClassLoader A

Class
Class
A
Parent ClassLoader A
Class

Class Class
Class
A Class
A
A
Class A
Class

ClassLoader B

Class
Class
A
A
Class



■ Prevent situations where duplicate named classes circumvent protection

Boot ClassLoader ClassLoader A

Class Class Class
Class
A Class Class
A
A
A
Class A
Class A
Class





Class Class Class
Class
A Class Class
A
A
A
Class A
Class A
Class





Class Class Class
Class
A Class Class
A
A
A
Class A
Class A
Class

Different protection levels!





Class Class Class
Class
A Class Class
A
A
A
Class A
Class A
Class





Class Class Class
Class
A Class Class
A
A
A
Class A
Class A
Class

Bad News



Here’s some rules to remember why this isn’t allowed…




To violate a constraint, the following 4 conditions must be met:
■ There exists a loader L such that L has been recorded by the JVM as an initiating
loader of a class C named N.
■ There exists a loader L’ such that L’ has been recorded by the JVM as an
initiating loader of a class C ’ named N.
■ The equivalence relation defined by the (transitive closure of the) set of imposed
constraints implies N L = N L’.
■ C≠C’




To violate a constraint, the following 4 conditions must be met:
■ There exists a loader L such that L has been recorded by the JVM as an initiating
loader of a class C named N.
■ There exists a loader L’ such that L’ has been recorded by the JVM as an
initiating loader of a class C ’ named N.
■ The equivalence relation defined by the (transitive closure of the) set of imposed
constraints implies N L = N L’.
■ C≠C’

Bottom line: You are protected from this


And after all that…


So what’s being done about security?

■ IBM and Oracle are working to ensure Java is (and remains) secure!
■ Reporting Issues:

http://www-03.ibm.com/security/secure-engineering/report.html
http://www.oracle.com/us/support/assurance/reporting/index.html


Conclusion

■ Java Security is defense in depth
■ Trust, but Verify
■ Java and JVM designed to provide security at a low cost to developers
■ Many moving parts in security – Things can go wrong, but quick to resolve
– Security is Hard – Rolling your own is even worse


Questions?


References

■ Get Products and Technologies:
– IBM Java Runtimes and SDKs:
• https://www.ibm.com/developerworks/java/jdk/
– IBM Monitoring and Diagnostic Tools for Java:
• https://www.ibm.com/developerworks/java/jdk/tools/

■ Learn:
– IBM Java InfoCenter:
• http://publib.boulder.ibm.com/infocenter/java7sdk/v7r0/index.jsp

■ Discuss:
– IBM Java Runtimes and SDKs Forum:
• http://www.ibm.com/developerworks/forums/forum.jspa?forumID=367&start=0


Copyright and Trademarks

© IBM Corporation 2012. All Rights Reserved.

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of
International Business Machines Corp., and registered in many jurisdictions
worldwide.

Other product and service names might be trademarks of IBM or other companies.

A current list of IBM trademarks is available on the Web – see the IBM “Copyright
and trademark information” page at URL: www.ibm.com/legal/copytrade.shtml


Java security in the real world (Ryan Sciampacone)

More Related Content

Viewers also liked (7)

Similar to Java security in the real world (Ryan Sciampacone) (13)

More from Chris Bailey (20)

Java security in the real world (Ryan Sciampacone)