javalightspeed-jakartatech-2023.pdf
- 1. Jakarta EE applications at light speed Rich Hagarty IBM Developer Advocate Jakarta Tech Talks October 12, 2023
- 2. Agenda 01 02 03 04 05 06 07 08 4 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine (JVM) JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 3. 5 Application Styles and the Java Runtimes that support them
- 4. There is a Spectrum of Architecture Styles Microservices Monolith “Macroservices” Virtual Machines Containers 7
- 5. Architectural style Functions Monolith Microservice Scale Smaller Larger Macroservice Lightweight Runtimes and Frameworks Full Function Runtimes Function as a Service 8 Different Java runtimes often have different sweet spots
- 6. 10 Runtime Characteristic Required to Span the Spectrum Modular/Flexible runtime – Full API set, but only what each different workload needs – Small memory footprint Cloud optimized – Consistent availability on all Cloud platforms – Full-stack Java optimization in containers – High performance and low cost during both steady state and scale-out Simple Delivery and Developer experience – Integration with all popular IDEs and build tools – Vendor neutral APIs – Jakarta EE and MicroProfile – Best-practice cloud deployment configuration by default – Continuous Integration without Fear
- 7. Agenda 01 02 03 04 05 06 07 08 11 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine (JVM) JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 8. 12 Open Liberty Lightweight, open framework built for cloud-native apps
- 10. 6 reasons why Liberty Just enough runtime Low operating cost Continuous delivery Zero migration Kubernetes optimized 80% disk and 56% memory saving 4x increased density over Tomcat & Spring Boot Zero-effort security fixing & zero technical debt 100% v2v & fixpack migration saving Self-tuned optimal perf, production-ready, kube-native Developer experience Container & kube-native experience, rapid inner loop Lightweight, highly- efficient runtime CI/CD optimized operational experience Simple true-to- production developer experience
- 11. <feature>cdi-4.0</feature> <feature>restfulWS-3.1</feature> Just enough runtime for right-size deployments Kernel cdi-4.0 appmgr restfulWS-3.1 Java EE With a Traditional App Server, the Full API stack as well as administration and operations features are loaded in each server instance With Liberty, you control which features are loaded into each server instance jndi-1.0 jsonp-2.1 15
- 12. CI/CD Friction in staying current leads to increased technical debt and security risk Liberty Zero Migration makes staying current easy No configuration behavior changes No runtime feature behavior changes No removals Liberty Continuous Delivery gives fast and reliable access to the latest features and fixes Full release every 4 weeks Includes fixes since previous release Effortlessly eliminate technical debt and stay secure Skipping a release does not introduce additional migration work Stay current with a rebuild (no app or config changes necessary) Simple seamless currency to eliminate technical debt 17
- 13. Developer experience Repositories Build IDEs Dev Mode APIs Testing
- 14. 21 Dev Mode No rebuild No redeploy No install No restart Just code!
- 15. Plan Code Build Test Release Deploy Run Monitor Cloud-native Liberty Testcontainers Zero Migration Continuous Delivery Quality Security An end-to-end cloud-native DevOps delivery experience with Liberty 29
- 16. Agenda 01 02 03 04 05 06 07 08 30 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine (JVM) JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 17. 31 Java on Cloud Always a good fit?
- 18. Time Responses per second Startup Steady state Time Memory Resident Set Time How much of application has runtime seen Varies with load Varies with load Varies with load Traditional focus of Java runtime technologies Rampup CPU consumption Traditional focus on steady state performance
- 19. Architecture Styles vs Typical Java workloads Architecture Characteristics Compatibility Monoliths Long-lived, efficient Macroservices Long-lived Microservices Fast start-up/ramp-up for scaling up and down (to zero) to meet demand. Small footprint. Serverless/Functions Fast start-up (in ms), short-lived
- 20. Agenda 01 02 03 04 05 06 07 08 34 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 21. 35 Java Virtual Machine (JVM) Some background
- 22. 36 What is the JVM? The Java Virtual Machine (JVM) is the runtime environment to execute Java code. The JVM is responsible for taking your application bytecode and converting it to a machines language for execution. Most other languages compile code for a specific system but in Java, the compiler converts the code for a Java Virtual Machine that can be used on any system hence the phrase “Write once, run anywhere”.
- 23. How the JVM works Class Loader Bytecode Verifier Interpreter JIT Java Bytecode JVM Java Code Java Compiler javac Run Time Compile Time OS Hardware
- 24. JVM Interpreter Class Loader Bytecode Verifier Interpreter JIT Java Bytecode JVM • Machine independent bytecodes are interpreted by the JVM at runtime • This ensures portability of Java programs across different architectures • But it affects performance because interpretation is relatively slow. OS Hardware
- 25. Just-in-Time Compiler Class Loader Bytecode Verifier Interpreter JIT Java Bytecode JVM • The answer to the performance issue is the JIT compiler, which transforms sequences of bytecodes into machine code that is cached by the JVM • Unit of compilation is a method. To save overhead, only “hot” methods are compiled • Compiled native machine code executes 10x faster than a bytecode-by-bytecode interpreter • Generated code is saved in a "code cache" for future use for lifetime of JVM OS Hardware
- 26. Agenda 01 02 03 04 05 06 07 08 40 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine (JVM) JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 27. 41 The JVM and JIT The good and the bad
- 28. JVM – The Good 42 • Device independent – write once, run anywhere • More than 25 years of improvements • JIT produces optimized machine code through use of Profilers • Efficient garbage collection • Longer it runs, the better it runs (JVM collects more profile data, JIT compiles more methods)
- 29. • Initial execution run is “interpreted”, which is relatively slow • “Hot” methods compiled by JIT can create CPU and memory spikes • CPU spikes cause lower QoS • Memory spikes cause OOM issues, including crashes • Slow start-up time • Slow ramp-up time JVM – The Bad (in containers)
- 30. JVM Spikes 0 50 100 150 200 250 300 350 400 0 30 60 90 CPU utilization (%) Time (sec) Daytrader7 CPU consumption CPU spikes caused by JIT compilation 0 100000 200000 300000 400000 500000 600000 0 30 60 90 Resident set size (KB) Time (sec) Daytrader7 memory footprint Footprint spikes caused by JIT compilation
- 31. Problem #1: Container Memory Footprint Main issues: • Need to over-provision to avoid OOM • Very hard to do – JVMs have a non-deterministic behavior 0 100000 200000 300000 400000 500000 600000 0 30 60 90 Resident set size (KB) Time (sec) Daytrader7 memory footprint Footprint spikes caused by JIT compilation
- 32. #2: Container Instances (Horizontal Auto-Scaling) Main issues: • Slow start-up and ramp-up times • CPU spikes can cause auto-scaler to INCORRECTLY launch additional instances (false positives)
- 33. Solution Requirements • Minimize/eliminate CPU and memory spikes • Improve start-up and ramp-up times
- 34. Agenda 01 02 03 04 05 06 07 08 48 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine (JVM) JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 35. 49 Eclipse OpenJ9 JVM and IBM Semeru Runtimes The foundation for Open Liberty
- 36. Overview of Eclipse OpenJ9 JVM Designed from the start to span all the operating systems needed by IBM products This JVM can go from small to large Can handle constrained environments or memory rich ones Renowned for its small footprint, fast start-up and ramp-up time Common heritage with IBM J9 JVM in IBM SDK for Java; is the JVM in IBM Semeru Runtimes https://github.com/eclipse-openj9/openj9
- 37. •IBM-built OpenJDK runtimes powered by the Eclipse OpenJ9™ JVM (the foundation of IBM’s Java-based-software for 20+ years). •The default Java for Liberty in all environments. •Highly optimized across all IBM platforms based on 20+ years experience. •New Semeru and Liberty innovations together make Java part of the Cloud. Mount Semeru - the tallest mountain on the island of Java in Indonesia OpenJ9 is optimized for Java in the cloud Liberty container App instance Semeru JVM IBM Semeru Runtimes Optimized Java for the Cloud 51
- 38. IBM Semeru Runtimes “The part of Java that’s really in the clouds” Other OpenJDK Distributions: • OpenJDK builds by Oracle • Adoptium Eclipse Temurin • Azul Zulu • BellSoft Liberica • Amazon Corretto • Alibaba Dragonwell • Microsoft Build of OpenJDK • Red Hat Build of OpenJDK • Oracle GraalVM
- 39. Agenda 01 02 03 04 05 06 07 08 53 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine (JVM) JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 40. 54 Semeru Cloud Compiler For Microservices
- 41. 55 Semeru “Cloud Compiler” makes Java part of the Cloud Offloads memory and CPU intensive JIT compilation to separate server Achieves higher density of application instances for the same throughput à lower cost In-process JIT Using Semeru Cloud Compiler Semeru Cloud Compiler (JIT as a Service Cloud Compiler increases app density à fewer nodes à less cost Cloud Compiler config Default config Liberty container App instance Semeru JVM JIT x Liberty container App instance Semeru JVM JIT x Liberty container App instance Semeru JVM JIT x semeruCloudCompiler: enable: true Simple to deploy with Liberty operator
- 42. SCC improves container density and cost Default config AM 500 B 550 C 550 F 450 P 450 P 450 B 550 F 450 AM 500 A 350 AM 500 M 200 Q 350 P 450 Q 350 D 600 D 1000 F 450 B 550 Q 350 AM 500 AM 500 AM 500 B 550 B 550 A 350 C 550 F 450 M 200 P 450 P 450 P 450 Q 350 Q 350 D 1000 AM 500 B 550 P 450 AM 500 B 550 B 550 C 550 C 550 F 450 F 450 P 450 Q 350 Q 350 D 1000 D 1000 Q 350 AM 250 AM 250 P 250 P 250 F 250 F 250 B 400 C 350 Q 150 Q 150 M 150 AM 250 AM 250 P 250 P 250 F 250 B 400 Q 150 Q 150 J 1200 A 250 B 400 B 400 C 350 D 1000 D 1000 D 600 AM 250 AM 250 P 250 P 250 F 250 F 250 B 400 C 350 Q 150 Q 150 M 150 AM 250 AM 250 P 250 P 250 F 250 B 400 Q 150 Q 150 J 1200 A 250 B 400 B 400 C 350 D 1000 D 1000 SCC config Legend: AM: AcmeAir monolithic A: Auth service B: Booking service C: Customer service D: Database (mongo/postgres) F: Flight service J: JITServer M: Main service P: Petclinic Q: Quarkus Total=8250 MB Total=8550 MB Total=8600 MB Total=9250 MB Total=9850 MB 6.3 GB less
- 43. Horizontal Pod Autoscaling in Kubernetes • Better autoscaling behavior with SCC due to faster ramp-up • Less risk to trick the HPA due to transient JIT compilation overhead Setup: Single node Microk8s cluster (16 vCPUs, 16 GB RAM) JVMs limited to 1 CPU, 500MB JITServer limited to 8 CPUs and has AOT cache enabled Load applied with JMeter, 100 threads, 10 ms think-time, 60s ramp-up time Autoscaler: scales up when average CPU utilization exceeds 0.5P. Up to 15 AcmeAir instances 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 0 60 120 180 240 300 360 420 480 Throughput (pages/sec) Time (sec) AcmeAir throughput when using Kubernetes autoscaling Baseline JITServer+AOTcache
- 44. 58 Demo
- 45. Semeru Cloud Compiler - Summary • JIT provides advantage, but compilation adds overhead • Disaggregate JIT from JVM è JIT compilation as a service • Semeru Cloud Compiler o Available now on Linux for Java 8, Java 11 and Java 17 (IBM Semeru Runtimes) o Especially good for constrained environments (micro-containers) o Kubernetes ready (Liberty Operator, Prometheus integration) o Can improve ramp-up, autoscaling and performance of short lived applications o Can reduce peak memory footprint, increase app density and reduce costs o Competing solutions are proprietary (not open-source) and require licensing
- 46. Agenda 01 02 03 04 05 06 07 08 60 Application Styles Open Liberty Java on cloud – a good fit? The Java Virtual Machine (JVM) JVM & JIT – good and bad OpenJ9 JVM and Semeru Runtimes Semeru Cloud Compiler Liberty InstantOn
- 48. Is Native Image the answer? Static AOT compile to native image (GraalVM Native Image) PROS CONS • Small footprint • Fast start-up • Great for short-lived apps • Limited JVM (static vs dynamic compiler optimization, limited GC) • Limited Java features (dynamic, APIs) • Moving target (frameworks, ecosystem, standards) • Dev/Prod parity (devs use compiler)
- 49. 63 Built-in “InstantOn” Java without compromise Characteristics Semeru InstantOn Semeru JVM Graal Native Full Java support Yes Yes No ‘Instant on’ Yes No Yes High throughput Yes Yes No Low memory (under load) Yes Yes No Dev-prod parity Yes Yes No Dev Build Prod Prod Prod checkpoint restore https://openliberty.io/blog/2023/06/29/rapid-startup-instanton.html Liberty and Semeru InstantOn innovation for near- native startup time for Java application containers. Up to 15x faster startup With all the benefits of the JVM and none of the compromises of Native Image. Leverages Linux CRIU to perform checkpoint / restore Ideal for cloud function and scale-to-zero workloads. restore restore
- 50. Checkpoints can be created at different points: - “beforeAppStart” – after app metadata is processed - “afterAppStart” – after app is started (fastest) Container starts up Container reaches snapshot point Liberty runs pre snapshot hooks Liberty initiates snapshot Build time Container exits after snapshot Restore Container from snapshot Liberty runs post restore hooks Run time Restore Container from snapshot Liberty runs post restore hooks Run time Application continues running Application continues running 64 Liberty InstantOn RUN checkpoint.sh applications afterAppStart
- 51. InstantOn: Where to checkpoint? 65 Liberty InstantOn leverages Semeru to provide a seamless checkpoint/restore solution for developers. With checkpoint restore, there is a tradeoff between startup time and the complexity of the restore. Checkpoint phases • beforeAppStart • afterAppStart Choice of where to checkpoint depends on what application start code does, e.g., any of the below would make afterAppStart unsuitable • Accessing a remote resource, such as a database • Reading configuration that is expected to change when the application is deployed • Starting a transaction Kernel start and feature runtime processing Process application Start application Accept requests afterAppStart beforeAppStart Later checkpoint == Faster restore time Later checkpoint == More complexity 300 – 2200ms 100 – 3000ms 0 – ???ms
- 52. 66 Demo
- 53. Adapting and thriving in the cloud requires evolving your development, deployment and runtime for a cloud- native world Liberty is a versatile runtime for these purposes, providing full-stack Java optimization with Semeru Runtimes InstantOn is a key capability that makes Liberty a great runtime for serverless use cases – https://openliberty.io – https://github.com/eclipse-openj9/openj9 – http://ibm.biz/InstantOn_HowToBlog – https://openliberty.io/blog/2023/06/29/rapid- startup-instanton.html – https://openliberty.io/docs/latest/instanton.html – https://blog.openj9.org/2022/09/26/getting-started- with-openj9-criu-support – https://blog.openj9.org/2022/10/14/openj9-criu- support-a-look-under-the-hood 67 Liberty InstantOn - Summary
- 54. 68 Wrap-up
- 55. Architecture Styles sweet spot for Open Liberty Architecture Solution Compatibility Monoliths Liberty with Semeru Runtimes Macroservices Liberty with Semeru Runtimes Microservices Liberty with Semeru Runtimes + Semeru Cloud Compiler Serverless/Functions Liberty with Semeru Runtimes + InstantOn
- 56. 70
- 57. Open Liberty IBM TechXchange / © 2023 IBM Corporation 71 Useful Links Why choose Liberty for Microservices https://ibm.biz/6ReasonsW hyLiberty Choosing the right Java runtime https://ibm.biz/ChooseJava Runtime How to approach application modernization https://ibm.biz/ModernizeJa vaApps Open Liberty Site https://www.openliberty.io Open Liberty Guides https://www.openliberty.io/ guides https://openliberty.io
- 58. Rapid Java startup without compromise: A deep dive October 31 10:00AM EST 72
- 59. Thank You Rich Hagarty IBM Developer Advocate rich.hagarty@ibm.com https://www.linkedin.com/in/rhagarty/ @rhagarty8 https://www.slideshare.net/RichHagarty/javalightspeed-jakartatech-2023pdf 73