Java and AI with LangChain4j: Jakarta EE and SmallRye LLM
0 likes79 views
Generative AI burst on to the public scene in November 2022, over ten years after Andrew Ng’s Machine Learning premiered on Coursera. Java developers have long felt like second class citizens, compared to the cool Python kids. LangChain4J changes the game. Java is cool with AI now! This lecture from 30-year industry veteran Ed Burns explores why it took so long for Java developers to have access to easy-to-use AI libraries, compared to Python developers. LangChain4J is the most popular of the Java libraries, but others exist. After a brief look at the landscape, we’ll take a deeper look at LangChain4J and how you use it to perform the most popular AI usage pattern: Retrieval Augmented Generation (RAG). RAG is basically a way to bring your own domain specific data to an existing AI model, and benefit from its power.
Java and AI with LangChain4j: Jakarta EE and SmallRye LLM
- 1. Java and AI with LangChain4j Code the future with AI Ed Burns @edburns Principal Architect PENDING download link
- 2. Ed Burns professional biography Client NCSA Mosaic (1994) SGI Cosmo Web Authoring Sun Netscape 6 OJI Server J2EE JSF (2002) Oracle Java EE Servlet, JSF, Bean Validation, etc. Cloud Java Champion Microsoft Azure Cloud (2019) Jakarta EE 11 committer and release coordinator Books
- 3. Change in content from posted agenda Still 100% Java and AI with LangChain4j. Removed content: After a brief look at the landscape. How you use it to perform the most popular AI usage pattern: Retrieval Augmented Generation (RAG). Added content: langchain4j-microprofile-jakarta. If you want the RAG content, I recommend Easy RAG with LangChain4J and Docker by Julien Dubois.
- 4. JUG Tour Around JavaLand 2025
- 5. Breaking Jakarta EE news Jakarta EE 11 Web Profile started release review ballot Monday! https://github.com/jakartaee/specifications/pull/802 https://deploy-preview-802--jakartaee- specifications.netlify.app/specifications/webprofile/11/ Many EE 12 component specifications in Plan Review stage. MicroProfile is discussing whether and how to host MicroProfile in Jakarta EE. The need for a unified Config solution for EE 12. Coincidentally timed soon after Evolving our middleware strategy blog post from Red Hat.
- 6. AI related career advice
- 7. Everyone I know admits: AI is moving faster than anything that has come before https://aka.ms/DeveloperCareerMasterplan/slides https://aka.ms/RockStarProgrammerSecrets
- 8. Scott Hanselman: Let’s all just chill From a trust perspective Very important to keep the “intelligence” in AI in perspective. LLMs cannot answer simple questions such as “Which episode of Gilligan’s Island was about mind reading?” The question that no LLM can answer and why it is important https://www.mindprison.cc/p/the-question-that-no-llm-can-answer
- 9. Java meets AI Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 10. Jakarta EE meets AI smallrye-llm langchain4j- microprofile -jakarta Microprofile config Microprofile Fault Tolerance Microprofile OpenTelemetry CDI Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 11. Components of LangChain4J Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 12. Quick look at LangChain4j examples https://github.com/langchain4j/langchain4j-examples
- 13. Components of SmallRye LLM/LangChain4J Jakarta 1. AI Service Interface Processing: o Detects interfaces annotated with @RegisterAIService o Generates proxy implementations using LangChain4J o Makes them available as CDI beans 2. Plugin Bean Creation: o Reads configuration from properties with smallrye.llm.plugin.* prefix o Creates beans for LangChain4J components (models, embedding stores) o Makes these components available for injection 3. MicroProfile Config Integration: o Translates properties to LangChain4J configuration o Handles environment variable substitution (e.g., ${azure.openai.api.key}) 4. Fault Tolerance: o Provides integration with MicroProfile Fault Tolerance o Adds resilience to AI service calls 5. Telemetry: o Provides observability for AI service calls o Integrates with OpenTelemetry
- 14. Some Code
- 15. CDI Injection Injection.java @Inject @Named(value = "ollama") ChatLanguageModel chatModel; @Inject @Named(value = "embedding-store-retriever") ContentRetriever retriever; Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 16. AI Service SimpleAIService.java @RegisterAIService(chatLanguageModelName = "ollama", tools = {org.wildfly.ai.websocket.Weather.class,org.wildfly.ai.websocket.Cal culator.class}, scope = SessionScoped.class) public interface SimpleAIService { @SystemMessage(""" You are an AI named Bob answering general question. Your response must be polite, use the same language as the question, and be relevant to the question.""") String chat(@UserMessage String question); } Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 17. Another approach, with MCP
- 18. But what is MCP?
- 19. But what is MCP? https://modelcontextprotocol.io/
- 20. But what is MCP? https://github.com/modelcontextprotocol/servers
- 21. Demo Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 22. MCP Server MCP tools workflow MCP Server Web Application LLM MCP Server 1 2 3 4 MCP Server 5 6 Content from Emmanuel Hugonnet http://www.ehsavoie.com/ MCP Server Database Filesystem Message queue JavaFX
- 23. AI Service with MCP Tools SimpleAIService.java @RegisterAIService(streamingChatLanguageModelName= "streaming- ollama", toolProviderName = "mcp") public interface ChatAiService { @SystemMessage(""" You are an AI named Bob answering general question. Your response must be polite, use the same language as the question, and be relevant to the question.""") TokenStream streamingChat(@UserMessage String question); } Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 24. Demo Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 25. Conclusion Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 26. What we saw External configuration instead of code CDI Injection AI Service support Cross-vendors: Micronaute, OpenLiberty, Quarkus, WildFly Enterprise ready : OpenTelemetry / Fault Tolerance / MCP Makes LangChain4J even easier to use ! Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 27. Resources • LangChain4J: o https://docs.langchain4j.dev/ • Model Context Protocol: o https://spec.modelcontextprotocol.io/ • smallrye-llm: a cross vendor LangChain4J -CDI integration library o https://github.com/smallrye/smallrye-llm • WildFly AI Feature Pack: o https://github.com/wildfly-extras/wildfly-ai-feature-pack • WildFly MCP: to interact with WildFly servers through MCP o https://github.com/wildfly-extras/wildfly-mcp • Admin your WildFly with LLM and MCP o https://www.youtube.com/watch?v=wg1hAdOoe2w • Making WildFly Glow with Intelligence: o https://www.wildfly.org/news/2025/02/10/Glowing-with-AI/ • Playing with Generative AI with WildFly: o https://www.wildfly.org/news/2024/11/04/WildFly-playing-with-generative-ai/
- 28. Thank you Ed Burns @edburns on most of the things • Please fill out this survey • 5 winners will be chosen to win a $100 digital gift card • https://aka.ms/edburns/survey2025
- 29. 1.What is your key decision making process when it comes to app modernization? 2.What’s your key challenge when modernizing your Java workload? 3.Are you a Java only shop or multiple language shop? 4.What is the tech stack/framework for your Java apps? 5.When migrating, what is your primary destination on cloud? What platform are you choosing? VM? PaaS? Containers? Why? 6.How did you or your team make the above decisions?