Adapting Java for the Serverless World at JUG Barcelona
- 1. Adopting Java for the Serverless world from the perspective of the AWS developer By Vadym Kazulkin, ip.labs GmbH
- 2. Contact Vadym Kazulkin, ip.labs GmbH v.kazulkin@gmail.com https://www.linkedin.com/in/vadymkazulkin/ @VKazulkin Co-Organizer of Java User Group Bonn & Serverless Bonn Meetup https://www.iplabs.de/
- 5. https://www.cleveroad.com/blog/programming-languages-ranking Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 7. 2020 Magic Quadrant for Cloud Infrastructure & Platform Services https://pages.awscloud.com/GLOBAL-multi-DL-gartner-mq-cips-2020-learn.html?pg=LWIAWS Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 8. Vadym Kazulkin @VKazulkin , ip.labs GmbH“State of Serverless 2020 report” https://codingsans.com/blog/serverless-trends
- 9. 2020 AWS Lambda Benchmark Report for Developers, DevOps, and Decision Makers https://newrelic.com/resources/ebooks/serverless-benchmark-report-aws-lambda-2020 Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 10. Life of the Java Serverless developer on AWS
- 11. Java Versions Support • Java 8 • with long-term support • Java 11 (since 2019) Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://aws.amazon.com/de/corretto/
- 12. Java ist very fast and mature programming language… Image: burst.shopify.com/photos/a-look-across-the-landscape-with-view-of-the-sea Christian Bannes and Vadym Kazulkin @VKazulkin , ip.labs GmbH … but Serverless adoption of Java looks like this
- 13. Vadym Kazulkin @VKazulkin , ip.labs GmbH“State of Serverless 2020 report” https://codingsans.com/blog/serverless-trends
- 14. 2020 AWS Lambda Benchmark Report for Developers, DevOps, and Decision Makers https://newrelic.com/resources/ebooks/serverless-benchmark-report-aws-lambda-2020 Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 15. Developers love Java and will be happy to use it for Serverless Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 16. Creating AWS Lambda with Java 1/2 : Source https://blog.runscope.com/posts/how-to-write-your-first-aws-lambda-function
- 17. Creating AWS Lambda with Java 2/2 : Source https://blog.runscope.com/posts/how-to-write-your-first-aws-lambda-function
- 18. Challenge Number 1 with Java is a big cold-start Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://www.serverless.com/blog/keep-your-lambdas-warm
- 19. Cold Start : Source: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
- 20. Bootstrap the Java Runtime Phase • AWS Lambda starts the JVM • Java runtime loads and initializes handler class • Static initializer block of the handler class is executed • Boosted host full CPU access up to 10 seconds • Lambda calls the handler method • Full CPU access only approx. from 1.8 GB “assigned” memory to the function Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
- 21. AWS Lambda cold start duration per programming language Source: Mikhail Shilkov: „AWS Lambda: Cold Start Duration per Language. 2020 edition” https://mikhail.io/serverless/coldstarts/aws/languages/
- 22. Cold start duration with Java • Below 1 second is best-case cold start duration for very simple Lambda like HelloWorld • It goes up significantly with more complex scenarios • Dependencies to multiple OS projects • Clients instantiation to communicate with other (AWS) services (e.g. DynamoDB, SNS, SQS, 3rd party) • To achieve the minimal cold start duration apply all best practices from these talk • Worst-case cold starts can be higher than 10 and even 20 seconds Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
- 23. 2020 AWS Lambda Benchmark Report for Developers, DevOps, and Decision Makers https://newrelic.com/resources/ebooks/serverless-benchmark-report-aws-lambda-2020 Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 24. Provisioned Concurrency for Lambda Functions Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Yan Cui: https://lumigo.io/blog/provisioned-concurrency-the-end-of-cold-starts/
- 25. Jeremy Daly: “Mixing VPC and Non-VPC Lambda Functions for Higher Performing Microservices” https://www.jeremydaly.com/mixing-vpc-and-non-vpc-lambda-functions-for-higher-performing-microservices/ Vadym Kazulkin @VKazulkin , ip.labs GmbH Lambda behind the Virtual Private Cloud (VPC)
- 26. Lambda in VPC As function’s execution environment scales • More network interfaces are created and attached to the Lambda infrastructure • The exact number of network interfaces created and attached is a factor of your function configuration and concurrency • Caused additional the cold start up to approx. 10 seconds Chris Munns: "Announcing improved VPC networking for AWS Lambda functions” https://aws.amazon.com/de/blogs/compute/announcing-improved-vpc-networking-for-aws-lambda-functions/
- 27. Lambda in VPC Improvements: • The network interface creation happens when Lambda function is created or its VPC settings are updated. • Because the network interfaces are shared across execution environments, only a handful of network interfaces are required per function • Reduced additional cold start from approx. 10 seconds to below 1 second Chris Munns: "Announcing improved VPC networking for AWS Lambda functions” https://aws.amazon.com/de/blogs/compute/announcing-improved-vpc-networking-for-aws-lambda-functions/
- 28. Improvements 1/4 • Switch to the AWS SDK 2.0 for Java • Lower footprint and more modular • Allows to configure HTTP Client of our choice (e.g. Java own Basic HTTP Client) • Initialize and prime dependencies during initialization phase • Use static initialization in the handler class • Provide all known values (for building clients e.g. DynamoDBClient) to avoid auto-discovery • credential provider, region, endpoint • Less (dependencies, classes) is more Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
- 29. Improvements 2/4 Avoid Reflection Or use DI Frameworks like Dagger which aren‘t reflection-based Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
- 30. Improvements 3/4 Strive for cost optimization Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 31. AWS Lambda pricing model Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 32. Cost for Lambda REQUEST DURATION Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 33. Request Tier $ 0.20 Per 1 Mio Requests Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 34. Duration Tier $ 0.00001667 Per GB-Second Christian Bannes and Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 35. GB-Second ONE SECOND ONE GB Christian Bannes and Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 36. Example • 1 Mio requests • Lambda with 512MiB • Each lambda takes 200ms 0.5 GiB * 0.2 sec * 1 Mio = 100 000 GB-Seconds Requests: $0.20 GB-Seconds: $1.67 Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 37. Cost scales linearly with memory Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 38. Improvements 4/4 More memory = more expensive? Christian Bannes and Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 39. Lambda Power Tuning • Executes different settings in parallel • Outputs the optimal setting Image: https://github.com/alexcasalboni/aws-lambda-power-tuning Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 40. Monitor the Java Virtual Machine Garbage Collection on AWS Lambda Source: Steffen Grunwald „Monitoring the Java Virtual Machine Garbage Collection on AWS Lambda” https://aws.amazon.com/de/blogs/architecture/field-notes-monitoring-the-java-virtual-machine-garbage-collection-on-aws-lambda/
- 41. Cost optimization • Java is well optimized for long running server applications • High startup times • High memory utilization Vadym Kazulkin @VKazulkin , ip.labs GmbH And both memory and execution time are cost dimensions, when using Serverless in the cloud
- 42. GraalVM enters the scene Source: https://www.graalvm.org/
- 43. Project Metropolis Goals: Low footprint ahead-of-time mode for JVM-based languages High performance for all languages Convenient language interoperability and polyglot tooling Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
- 45. GraalVM Architecture Sources: Practical Partial Evaluation for High-Performance Dynamic Language Runtimes http://chrisseaton.com/rubytruffle/pldi17-truffle/pldi17-truffle.pdf „The LLVM Compiler Infrastructure“ https://llvm.org/
- 46. GraalVM Architecture Sources: Practical Partial Evaluation for High-Performance Dynamic Language Runtimes http://chrisseaton.com/rubytruffle/pldi17-truffle/pldi17-truffle.pdf „The LLVM Compiler Infrastructure“ https://llvm.org/
- 47. SubstrateVM Source: Oleg Šelajev, Thomas Wuerthinger, Oracle: “Deep dive into using GraalVM for Java and JavaScript” https://www.youtube.com/watch?v=a-XEZobXspo
- 48. GraalVM and SubstrateVM Source: Oleg Selajev, Oracle : “Run Code in Any Language Anywhere with GraalVM” https://www.youtube.com/watch?v=JoDOo4FyYMU
- 49. GraalVM on SubstrateVM A game changer for Java & Serverless? Java Function compiled into a native executable using GraalVM on SubstrateVM reduces • “cold start” times • memory footprint by order of magnitude compared to running on JVM. And both memory and execution time are cost dimensions, when using Serverless in the cloud
- 50. GraalVM on SubstrateVM A game changer for Java & Serverless? Current challenges with native executable using GraalVM : • Most Cloud Providers (AWS) doesn’t provide GraalVM as Java Runtime out of the box, only Open JDK (e.g. AWS provides Corretto) • Some Cloud Providers (e.g. AWS) provide Custom Runtime Option
- 51. Lambda Layers & Lambda Runtime API Christian Bannes and Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 53. GraalVM Complitation Modes Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
- 54. AOT vs JIT Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
- 55. GraalVM Profile-Guided Optimizations Source: „https://www.graalvm.org/reference-manual/native-image/PGO”
- 56. Support of GraalVM native images in Frameworks Spring Framework: working toward GraalVM native image support without requiring additional configuration or workaround is one of the themes of upcoming Spring Framework 5.3 Spring Boot: Ongoing work on experimental Spring Graal Native project. Probably ready for the 2.4 release Quarkus: a Kubernetes Native Java framework developed by Red Hat tailored for GraalVM and HotSpot, crafted from best-of-breed Java libraries and standards. Micronaut: a modern, JVM-based, full-stack framework for building modular, easily testable microservice and serverless applications. Source: „GraalVM native image support“ https://github.com/spring-projects/spring-framework/wiki/GraalVM-native-image-support
- 57. Steps to deploy to AWS • Installation prerequisites • Framework of your choice (Micronaut, Quarkus, Spring) • Java 8 or 11 • Apache Maven or Gradle • AWS CLI and AWS SAM CLI (for local testing) • Build Linux executable of your application with GraalVM native-image • Deploy Linux executable as AWS Lambda Custom Runtime • Function.zip with bootstrap Linux executable Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://github.com/awslabs/aws-serverless-java-container/tree/master/samples/micronaut/pet-store
- 58. AWS Lambda Deployment of Custom Runtime with SAM Source: https://github.com/awslabs/aws-serverless-java-container/tree/master/samples/micronaut/pet-store
- 60. AWS Lambda with Micronaut Framework
- 61. Testing AWS Lambda with Micronaut Framework
- 62. Micronaut Additional Features • Custom Validators • AWS API Gateway integration • Spring annotation processor available Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
- 63. Build GraalVM Native Image with Micronaut Framework
- 65. AWS Lambda with Quarkus Framework
- 66. Testing AWS Lambda with Quarkus Framework
- 67. Build GraalVM Native Image with Quarkus Framework mvn –Pnative package
- 68. Quarkus Additional Features • Website for creating the App • AWS API Gateway integration • Funqy Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
- 70. Quarkus-Fanqy AWS Serverless Support • AWS Lambda • AWS API Gateway Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 71. Spring (Boot) Framework Source: https://spring.io/
- 72. Spring GraalVM Native Project Vadym Kazulkin @VKazulkin , ip.labs GmbH
- 73. AWS Lambda with Spring Framework using Spring Graal Native and Spring Cloud Functions
- 74. Bean Registration with Spring Framework using Spring Graal Native and Spring Cloud Functions
- 75. Build GraalVM Native Image with Spring Framework mvn –Pnative package
- 76. Framework Comparison • Project Initializer • Programming Model • Database Support • Test Support • Native Image • Native image size • Startup time • Heap size Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: „Battle Of The Microservice Frameworks: Micronaut versus Quarkus edition! by Michel Schudel“ https://www.youtube.com/watch?v=hnEXOqcNXPs „Micronaut 2.0 vs Quarkus 1.3.1 vs Spring Boot 2.3 Performance on JDK 14“ https://www.youtube.com/watch?v=rJFgdFIs_k8
- 77. Conclusion • GraalVM and Frameworks are really powerful with a lot of potential • But in combination with Native Image currently not without challenges • AWS Lambda Custom runtime requires Linux executable only • Windows and Mac developers may only build Linux executable via Docker • plenty of 'No instances of … are allowed in the image heap’ and other errors when building a native image • Lots of experimentation with additional build arguments like “initialize-at-runtime” or “delay-class-initialization-to-runtime“ required • Once again: Less (dependencies, classes) is more • AWS Lambda function should be small and shouldn’t have many dependencies Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: „https://stackoverflow.com/questions/63328298/how-do-you-debug-a-no-instances-of-are-allowed-in-the-image-heap-when-buil
- 78. Try it yourselves • Micronaut • https://github.com/micronaut-guides/micronaut-function-aws-lambda • Quarkus • https://github.com/JosemyDuarte/quarkus-terraform-lambda-demo/tree/dynamo-terraform • Spring Boot • https://github.com/spring-projects-experimental/spring-graalvm-native/tree/master/spring- graalvm-native-samples/function-aws • Misc • https://github.com/awslabs/aws-serverless-java-container/tree/master/samples Vadym Kazulkin @VKazulkin , ip.labs GmbH