Scaling Vector Search: How Milvus Handles Billions+

1 | © Copyright 8/16/23 Zilliz
Stephen Batifol | Zilliz
Unstructured Data Meetup, July 16th
Scaling Vector Search: How
Milvus Handles Billions+

Stephen Batifol
Developer Advocate, Zilliz/ Milvus
stephen.batifol@zilliz.com
linkedin.com/in/stephen-batifol/
@stephenbtl
Speaker

27K+
GitHub
Stars
25M+
Downloads
250+
Contributors
2,600
+
Forks
Milvus is an open-source vector database for GenAI projects. pip install on your
laptop, plug into popular AI dev tools, and push to production with a single line of
code.
Easy Setup
pip install
pymilvus to start
coding in a notebook
within seconds.
Reusable Code
Write once, and
deploy with one line
of code into the
production
environment
Integration
Plug into OpenAI,
Langchain,
LlamaIndex, and
many more
Feature-rich
Dense & sparse
embeddings,
filtering, reranking
and beyond

Seamless integration with all popular AI toolkits

Well-connected in LLM infrastructure to enable RAG
use cases
Framework
Hardware
Infrastructure
Embedding Models LLMs
Software Infrastructure
Vector Database

Retrieval Augmented
Generation (RAG)
Expand LLMs' knowledge by
incorporating external data sources
into LLMs and your AI applications.
Match user behavior or content
features with other similar ones to
make effective recommendations.
Recommender System
Search for semantically similar
texts across vast amounts of
natural language documents.
Text/ Semantic Search
Image Similarity Search
Identify and search for visually
similar images or objects from a
vast collection of image libraries.
Video Similarity Search
Search for similar videos, scenes,
or objects from extensive
collections of video libraries.
Audio Similarity Search
Find similar audios in large datasets
for tasks like genre classification or
speech recognition
Molecular Similarity Search
Search for similar substructures,
superstructures, and other
structures for a specific molecule.
Anomaly Detection
Detect data points, events, and
observations that deviate
significantly from the usual pattern
Multimodal Similarity Search
Search over multiple types of data
simultaneously, e.g. text and
images
Common AI Use Cases

01
Introduction to Vector DB
and Vector Search

Traditional database was built upon exact search

…which misses context, semantic meaning, and user intent
VS.
Apple
VS.
Rising dough
VS.
Change car tire
Rising Dough
Proofing Bread
✔
❌

…and cannot process increasingly growing unstructured data
*Data Source: The Digitization of the World by IDC
20%
Other
newly generated data in 2025
will be unstructured data
80%

Vector
Databases
Where do Vectors Come From?

Embeddings Models

Vector Embedding

Vector Space

02
How do Vector Databases
Work?

How Similarity Search Works
Vn, 1
…
…
…
1
2
3
4
5
Transform into
Vectors
Unstructured Data
Images
User Generated
Content
Video
Documents
Audio
Vector Embeddings
Perform Approximate
Nearest Neighbor
Similarity Search
Perform Query
Get Results
Store in Vector Database

02
How does Milvus do it at
Billions+ Scale?

Design Principles
• Disaggregate storage and computation
• Fully depends on mature storage
systems
• Micro Service - scale by functionality
• Separate Streaming and historical data
• Pluggable engine, storage and index
• Log As data

Meta Storage
Root Query Data Index
Coordinator Service
Proxy
Proxy
etcd
Log Broker
SDK
Load Balancer
DDL/DCL
DML
NOTIFICATION
CONTROL SIGNAL
Object Storage
Minio / S3 / AzureBlob
Log Snapshot Delta File Index File
Worker Node QUERY DATA DATA
Message Storage
VECTOR
DATABASE
Access Layer
Query Node Data Node Index Node
Fully Distributed Architecture

Each shard is managed by a supervisor
(shard leader). This supervisor is
responsible for:
• Adding new information to the shard.
• Regularly storing the data in a safe
place (object storage).
• Serving the latest information for
search requests.
• Forwarding historical data requests to
other cabinets (query nodes) if
needed.
Milvus Data Layout - Shard

Growing Segment:
• In-memory segment replaying data
from the Log Broker.
• Uses a FLAT index to ensure data is
fresh and appendable.
Sealed Segment:
• Immutable segment using
alternative indexing methods for
efficiency.
Milvus Data Layout - Segments

Behind the Scenes: How Data Gets Added and
Accessed
• Sharding: Large datasets are
divided into smaller,
manageable sections called
shards. Each shard is handled
by a dedicated datanode.
• Write-Ahead Log (WAL):
When you add new data, a
proxy service writes it to a
temporary log called a WAL
(e.g., Kafka, Pulsar). Think of it
as a to-do list for the
datanodes.

Behind the Scenes: How Data Gets Added and
Accessed
• Datanodes subscribe to the WAL
and:
• Add new data to their assigned shard.
• Remove outdated data (if needed)
• Flush accumulated data to permanent
storage.
• Query Nodes also subscribe to the
WAL but focus on:
• Creating and managing Segments
within each shard for fast searching.
• Ensuring searches access the latest
information.

Index Building
To avoid frequent index building
for data updates.
A collection in Milvus is divided
further into segments, each with
its own index.

Data query refers to:
• Searching a specified
collection for k number of
vectors nearest to a target
vector or for all vectors within
a specified distance range to
the vector.
Data query

Some customers with Millions/ Billions Scale

09
RAG in action with Milvus
Lite

milvus.io
github.com/milvus-io/
@milvusio
@stephenbtl
/in/stephen-batifol
Thank you

Scaling Vector Search: How Milvus Handles Billions+

More Related Content

Similar to Scaling Vector Search: How Milvus Handles Billions+ (20)

More from Zilliz (20)

Recently uploaded (20)

Scaling Vector Search: How Milvus Handles Billions+