The Trinity in GenAI - Spring AI, LangChain4J and OpenAI

The Trinity in GenAI - Spring AI,
LangChain4J and OpenAI
I Love Tech ROMANIA 2024
Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist
ORACLE
Copyright © 2024, Oracle and/or its affiliates

Juarez Barbosa Junior
Senior Principal Java Developer Evangelist
• Coming from Dublin, Ireland
• 20 years of experience in SW Engineering & DevRel
• Oracle, Microsoft, IBM, Nokia, Unisys, Accenture, startups
• Microsoft Azure Developer Relations Lead
• IBM Watson Tech Evangelist & Cloud Rockstar
• IBM Mobile Tech Evangelist & Global Thought Leader
• Nokia Developers Global Champion
• Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI,
Blockchain, Rust
• Speaker at conferences
• Oracle CloudWorld, Oracle Code, Microsoft Ignite, Microsoft
TechX, jPrime, JCON, DevConf.cz, GeeCon, DevOpsDays,
DeveloperWeek, DevOps Institute, CloudLand, DWX, The
Developer’s Conference (TDC), Sec4Dev, JSNation, NodeConf,
Conf42, Shift Conf, Global Azure, Open-Source Lisbon,
CodeFrenzy, Mêlée Numérique, React Summit, Test.js Summit,
Porto TechHub Conf, Pyjamas, MiTechCon, JUGs, meetups, etc.
@juarezjunior
@juarezjunior

Agenda
• AI and GenAI – Synthetic Content
• Artificial Intelligence and GenAI
• OpenAI, Cohere
• Spring AI
• LangChain4J
• Enabling GenAI with AI Vector Search in
Oracle Database
• Java App Dev with the Oracle Database
• Technical References
• Audience: Business and Technical
audiences (not Machine Learning
Engineers)

AI and GenAI –
Synthetic Content
Bucharest the Capital of Romania
pictured in a futuristic scenario in one
image

Artificial Intelligence and GenAI
• Core concepts in a nutshell
• GenAI - Artificial intelligence algorithms and transformer models with the
capability of generating content.
• Content - Text, images, videos, code, and others.
• Generation – a result of an input prompt (prompt engineering).
• Training data – dataset
• Apps – Applied currently in the fields of content development, entertainment,
drug discovery, software development, healthcare, material generation
sciences, and others.
• Ethical issues – misuse, deep fakes, bias and discrimination, automation and
job replacements, cybersecurity and cybercriminals.
• AI is now a commodity!

OpenAI
• Generative AI with OpenAI
• Natural Language Processing (NLP)
• Powerful, pre-trained and customizable Machine Learning Models
• OpenAI API (REST interface)
• Scalable infrastructure (powered by Microsoft Azure)
• Artificial General Intelligence (AGI) Research – ultimate goal
• https://openai.com/sora

Spring AI
• Spring AI
• Java app framework for building AI-related applications. It encapsulates many
of the complexities required to interact with a number of different GenAI
providers such as OpenAI by providing a robust abstraction layer.
• API support across AI providers for chat, text-to-image, and embedding
models.
• Its goal is to apply to the AI domain Spring ecosystem design principles such
as portability and modular design and promote using POJOs as the building
blocks of an application to the AI domain.
• https://spring.io/projects/spring-ai

LangChain4J
• LangChain4J
• Easy interaction with LLMs and Vector Stores
• All major commercial and open-source LLMs and Vector Stores are easily
accessible through a unified API
• Easy integration with Java frameworks – Quarkus and Spring Boot
• AI Services, RAG, Tools, Chains
• https://docs.langchain4j.dev/

Enabling Generative AI
with AI Vector Search in
Oracle Database
Recap on background behind semantic search and
Generative AI leveraging Oracle AI Vector Search
9 Copyright © 2024, Oracle and/or its affiliates

The following is intended to outline our general product
direction. It is intended for information purposes only, and may
not be incorporated into any contract. It is not a commitment to
deliver any material, code, or functionality, and should not be
relied upon in making purchasing decisions. The development,
release, timing, and pricing of any features or functionality
described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.
The materials in this presentation pertain to Oracle Health, Oracle, Oracle Cerner, and Cerner Enviza which are all wholly owned
subsidiaries of Oracle Corporation. Nothing in this presentation should be taken as indicating that any decisions regarding the
integration of any EMEA Cerner and/or Enviza entities have been made where an integration has not already occurred.

Generative AI and AI Vector Search Democratizes AI
Using Vector Databases plus LLMs is SIMPLE
• No longer requires data scientists to build AI solution
• Use standard pre trained embedding models
• Generate vectors, add to private Vector Database
• Augment LLM search with private knowledge
AI vector search is SUPERFAST
• In order to provide AI responses in real time

Finding what you’re looking for can be challenging…
12
I have a photo of an
artwork, but don’t know
the artist or work’s name
I heard a catchy tune
and can even hum a
few bars, but don’t
know the title or artist
I got this error message
ORA-XXXX, but don’t
know how to resolve it

A technology called Vector Search
enables end-user questions
to be mapped to relevant data
in your database
13

Vector search continues to evolve…
14
Vector Space Model
1960s – 1970s
Word Embeddings
e.g., LSA and LDA
2000s
Deep Learning Era
e.g., Word2Vec and GloVe
2010s
Efficient Search Algorithms
e.g., ANN, HNSW, PQ, Annoy
2010s
Transformers and Contextual Embeddings
e.g., BERT, GPT, etc.
late 2010s
33
42
16
21
50
Early Multidimensional Indexing
e.g., KD trees, R-trees, Ball trees
1970s – 1990s
Dedicated Vector Databases
e.g., Milvus, Pinecone, Vespa
late 2010s …
Relational Database
Integration
e.g., Oracle
late 2010s – 2020s

Vectors in AI represent semantics of
unstructured data such as images, documents, videos, etc.
A vector is a sequence of
numbers, called dimensions,
used to capture the important
"features" of the data
Vectors represent the semantic
content of data, not the
underlying words or pixels
Vectors generated using deep
learning embedding models
Vector
33
42
16
21
50
15

Word relatedness in two dimensions
An intuitive example
16
blackberry
strawberry
apple
raspberry
pear
plum
cat
kitten
dog
puppy
wolf
lion
Maine
Vermont
New York
kiwi
d1
d2
elephant
tiger

Convert an image to one or more vector embeddings and search vector database
Embeddings for images
17
0.5, 1.5, 2.6, -1.1, …
Compare these against other
vectors in the database to find
similar content

Generating embeddings
Use an embedding model
Text documents
Natural Language
Embedding
Model
Images
Image
Embedding
Model
Text Vector Table
id vector Image
1 [0.5, 1.5, 2.6, -1.1, …]
2 [1.0, 0.9, 1.6, -1.3, …]
3 [0.6, 1.1, 1.3, -0.9, …]
… … …
Image Vector Table
id vector text
1 [0.8, 0.5, 1.6, -2.5, …] “It was the best of times, it was the
worst of times, it was..”
2 [1.1, 0.3, 0.6, -1.3, …] “It is a truth universally
acknowledged, that a single man..”
3 [1.3, 0.1, 0.2, -1.1, …] “It was a bright cold day in April,
and the clocks were striking..”
… … …

Simple vector search
Pipeline
Embedding
Model
Input Data
Object
Data Object Retrieval
Vector ID
Matches
Relevant
content
Vector Search
embedding
What we’re looking for
Text description, image, etc.
compare
…

Role of vector databases with LLMs
Address the hallucination problem inherent in LLM responses
Augment prompt with enterprise-specific content to produce better responses
Avoid exceeding LLM token limits by using most relevant content
Broad range of data
from the internet
Snapshot of data from a point in time
Private enterprise data
Current data - frequently updated
LLM
Vector Database

Role of vector databases with LLMs
Avoid using sensitive customer data for LLM training and fine-tuning
Cheaper than fine-tuning LLMs, which can be expensive to update
Real-time updated knowledgebase
Cache previous LLM prompts/responses to improve performance and reduce costs
LLM
Better business outcomes
Vector Database

LLM-based chatbot with “enterprise knowledge”
Extended pipeline using retrieval architecture
Prompt
Chat History
Embedding
Model
”What’s the policy?”
”What’s my corporate
401k policy?”
Enhanced
Prompt
Similarity Search
Relational Database
Vector ID
Matches
Fetch docs for matching IDs
Augmented
Prompt
Enhanced prompt + policy details +
scenario examples + known concerns +
expected policy changes…
LLM
Private
content
RESPONSE
Vector Database
embedding
embedding

Use cases
What do embeddings and vector databases enable?
25
Similarity Search
Match candidates with jobs
Image and video retrieval
Find patients with similar issues
Legal e-discovery
Content-Based
Filtering
Personalized recommendation
Find retail items from image
Natural Language
Processing
Text Classification and Clustering
SQL generation
Data Analytics
Anomaly detection
Pattern recognition
Computer Vision
Face recognition
Biometric identification
Object detection
Biomedical Research
Gene/DNA similarity analysis
Molecular structure search
Geographic
Information Systems
Spatial analysis
Map rendering
Industrial Applications
Quality control
Predictive maintenance
Machinery malfunction

AI Vector Search can augment your prompt
to enable accuracy with Generative AI.

Java App
Dev with
Oracle
Database

Oracle JDBC - Support for the Latest Java
Versions
• Support to the latest Java versions
(11, 17, 21) and GraalVM
• Project Loom - Virtual Threads
support
• Java Virtual Threads and Pipelined
Database Operations
• JDBC Standards - 4.2 and 4.3
• Reactive Streams - Java Flow API
support
• R2DBC, Reactive Streams Ingestion
(RSI)
• Pipelined Database Operations
Data access is critical in
mission-critical applications!

Overview of Oracle DB Access with Java

Technical References
• Spring AI - https://spring.io/projects/spring-ai
• LangChain4J - https://github.com/langchain4j
• OpenAI - https://openai.com/
• Oracle AI Vector Search - https://rb.gy/gqhw4m
• SD4J by Oracle Labs - https://github.com/oracle/sd4j
• Java / JDBC – https://www.oracle.com/database/technologies/appdev/jdbc.html
• Introduction to Oracle JDBC Driver Support for Virtual Threads -
https://bit.ly/3UlNJWP
• Developing an Oracle JDBC app with GraalVM Native Image - https://rb.gy/iy3sgh
• Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver - https://rb.gy/42dnz5
• Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.ly/3rEiRnC
• Introduction to JDBC Reactive Extensions with the Oracle Database 23c Free —
Developer Release – https://rb.gy/qxlrbx
• Pipelined Database Operations - https://rb.gy/iy3sgh

The Trinity in GenAI - Spring AI, LangChain4J and OpenAI

New | Oracle Database 23ai
Next-Generation Converged Database Services
Over 300 major new features plus thousands of enhancements
Available on OCI and Azure
Major focus on
• AI for Data features
• Developer/Analyst features
• Mission-Critical features
Addresses data management pain points that have frustrated customers forever

Oracle LiveLabs
Showcasing how Oracle’s solutions can
solve your business problems
500+
free workshops,
available or in
development
3.5 million
people have already visited
LiveLabs
developer.oracle.com/livelabs
learn something new …at your pace!
600+
events run
using LiveLabs
workshops

3 membership tiers
Connect: @oracleace facebook.com/OracleACEs
aceprogram_ww@oracle.com
500+ technical experts &
community leaders helping peers globally
The Oracle ACE Program recognizes & rewards individuals for
their technical & community contributions to the Oracle community
Nominate
yourself or a candidate:
ace.oracle.com/nominate
Learn more - ace.oracle.com
blogs.oracle.com/ace

Create your FREE
Cloud Account
• Go to
https://signup.cloud.oracle.com/

SD4J – Demo
• Creating GenAI Apps in Java with SD4J (Stable
Diffusion for Java) and the ONNX Runtime
• https://juarezjunior.medium.com/b72385c59329

Spring AI with Open AI
• Code walkthrough of a sample Spring AI application
that uses Open AI

The Trinity in GenAI - Spring AI, LangChain4J and OpenAI

More Related Content

Similar to The Trinity in GenAI - Spring AI, LangChain4J and OpenAI (20)

More from Juarez Junior (20)

Recently uploaded (20)

The Trinity in GenAI - Spring AI, LangChain4J and OpenAI