Multi-agent Systems with Mistral AI, Milvus and Llama-agents

1 | © Copyright 8/16/23 Zilliz
Stephen Batifol | Zilliz
Unstructured Data Meetup, Sept 5th
Multi-agent Systems with
Mistral AI, Milvus and
Llama-agents

Stephen Batifol
Developer Advocate, Zilliz/ Milvus
stephen.batifol@zilliz.com
linkedin.com/in/stephen-batifol/
@stephenbtl
Speaker

28K
GitHub
Stars
25M
Downloads
250
Contributors
2,600
+
Forks
Milvus is an open-source vector database for GenAI projects. pip install on your
laptop, plug into popular AI dev tools, and push to production with a single line of
code.
Easy Setup
Pip-install to start
coding in a notebook
within seconds.
Reusable Code
Write once, and
deploy with one line
of code into the
production
environment
Integration
Plug into OpenAI,
Langchain,
LlmaIndex, and
many more
Feature-rich
Dense & sparse
embeddings,
filtering, reranking
and beyond

Seamless integration with all popular AI toolkits

01 Key Components

pip install pymilvus
Milvus Lite

• Framework for building LLM Applications
• Focus on retrieving data and integrating with
LLMs
• Integrations with most AI popular tools
🦙 llama-index

🦙 llama-agents 🤖 by llama-index
• Build Stateful apps with LLMs
and Multi-Agents workflow
• Cycles and Branching
• Human-in-the-Loop
• Persistence

Mistral AI
• Mistral Embed
• Embedding Model focused on Retrieval, very useful
for RAG
• English only
• Mistral Nemo
• 12B model with 128k context length
• Strong Function Calling and Retrieval for its size
• Run Locally
• Mistral Large 2
• 123 Billions parameters with 128K context length
• Very strong Function Calling and Retrieval skills

Tavily
● Web search API for up-to-date information
● Tavily Search API is a search engine
optimized for LLMs
● Expands agent's knowledge base

| © Copyright 8/16/23 Zilliz
11
RAG
Retrieval Augmented Generation)

Basic Idea
Use RAG to force the LLM to work with your data
by injecting it via a vector database like Milvus

Basic RAG Architecture

5 lines starter

Naive RAG is limited

Naive RAG Pipeline
⚠ Single-shot
⚠ No query understanding/planning
⚠ No tool use
⚠ No reflection, error correction
⚠ No memory (stateless)

Naive RAG failure mode
Summarization

Implicit data

Multi-part questions

RAG is necessary but
not sufficient

01 Agentic RAG

Agentic RAG
✅ Multi-turn
✅ Query / task planning layer
✅ Tool interface for external environment
✅ Reflection
✅ Memory for personalization

● Self-Reflection 🛜
○ Check the internet to verify
● Query Routing
○ Whether to check our RAG system
● Query Routing with Subquery
○ Whether to check our RAG system for different queries
● Conversation Memory
● Tool Use
Agentic RAG

milvus.io
github.com/milvus-io/
@milvusio
@stephenbtl
/in/stephen-batifol
Thank you

Meta Storage
Root Query Data Index
Coordinator Service
Proxy
Proxy
etcd
Log Broker
SDK
Load Balancer
DDL/DCL
DML
NOTIFICATION
CONTROL SIGNAL
Object Storage
Minio / S3 / AzureBlob
Log Snapshot Delta File Index File
Worker Node QUERY DATA DATA
Message Storage
VECTOR
DATABASE
Access Layer
Query Node Data Node Index Node
Milvus Architecture

Multi-agent Systems with Mistral AI, Milvus and Llama-agents

More Related Content

Similar to Multi-agent Systems with Mistral AI, Milvus and Llama-agents (20)

More from Zilliz (20)

Recently uploaded (20)

Multi-agent Systems with Mistral AI, Milvus and Llama-agents