Beyond Retrieval Augmented Generation (RAG): Vector Databases
0 likes586 views
The document discusses the significance and workings of vector databases, highlighting their ability to analyze vast amounts of unstructured data, which constitutes 80% of global data by 2025. It explains various vector similarity metrics, provides examples of use cases such as semantic search and recommendation systems, and outlines the architecture and advantages of using Milvus/Zilliz for large-scale data management. Vector databases are promoted as a solution for efficient storage, indexing, and querying of vector embeddings from unstructured data.
![Semantic Similarity
Image from Sutor et al
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Man = [0.5, 0.2]
Queen - Woman + Man = King
Queen = [0.3, 0.9]
- Woman = [0.3, 0.4]
[0.0, 0.5]
+ Man = [0.5, 0.2]
King = [0.5, 0.7]
Man = [0.5, 0.2]](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-13-320.jpg)
![Vector Similarity Metric: L2 (Euclidean)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
d(Queen, King) = √(0.3-0.5)2
+ (0.9-0.7)2
= √(0.2)2
+ (0.2)2
= √0.04 + 0.04
= √0.08 ≅ 0.28](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-15-320.jpg)
![Vector Similarity Metric: Inner Product (IP)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Queen · King = (0.3*0.5) + (0.9*0.7)
= 0.15 + 0.63 = 0.78](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-16-320.jpg)
![Queen = [0.3, 0.9]
King = [0.5, 0.7]
Vector Similarity Metric: Cosine
𝚹
cos(Queen, King) = (0.3*0.5)+(0.9*0.7)
√0.32
+0.92
* √0.52
+0.72
= 0.15+0.63 _
√0.9 * √0.74
= 0.78 _
√0.666
≅ 0.03](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-17-320.jpg)
Beyond Retrieval Augmented Generation (RAG): Vector Databases
- 1. Yujian Tang | Zilliz Beyond RAG: Vector Databases
- 2. Yujian Tang Senior Developer Advocate, Zilliz yujian@zilliz.com https://www.linkedin.com/in/yujiantang https://www.twitter.com/yujian_tang Speaker
- 3. 01 Why Vector Databases? CONTENTS 03 04 Vector Database Architecture 02 How Do Vector Databases Work? Use Cases
- 4. 01 Why Vector Databases?
- 5. Compare data that you couldn’t compare before
- 6. Unstructured Data is Everywhere Unstructured data is any data that does not conform to a predefined data model. By 2025, IDC estimates there will be 175 zettabytes of data globally (that's 175 with 21 zeros), with 80% of that data being unstructured. Currently, 90% of unstructured data is never analyzed. Text Images Video and more!
- 7. Vector Databases Unstructured Data + ML = Vector Magic
- 8. Find Semantically Similar Data Apple made profits of $97 Billion in 2023 I like to eat apple pie for profit in 2023 Apple’s bottom line increased by record numbers in 2023
- 9. But wait! There’s more!
- 10. Use math to quantify relationships between entities
- 11. 02 How Do Vector Databases Work?
- 12. Vector similarity is a mathematical measure of how close two vectors are
- 13. Semantic Similarity Image from Sutor et al Woman = [0.3, 0.4] Queen = [0.3, 0.9] King = [0.5, 0.7] Woman = [0.3, 0.4] Queen = [0.3, 0.9] King = [0.5, 0.7] Man = [0.5, 0.2] Queen - Woman + Man = King Queen = [0.3, 0.9] - Woman = [0.3, 0.4] [0.0, 0.5] + Man = [0.5, 0.2] King = [0.5, 0.7] Man = [0.5, 0.2]
- 14. Similarity metrics are ways to measure distance in vector space
- 15. Vector Similarity Metric: L2 (Euclidean) Queen = [0.3, 0.9] King = [0.5, 0.7] d(Queen, King) = √(0.3-0.5)2 + (0.9-0.7)2 = √(0.2)2 + (0.2)2 = √0.04 + 0.04 = √0.08 ≅ 0.28
- 16. Vector Similarity Metric: Inner Product (IP) Queen = [0.3, 0.9] King = [0.5, 0.7] Queen · King = (0.3*0.5) + (0.9*0.7) = 0.15 + 0.63 = 0.78
- 17. Queen = [0.3, 0.9] King = [0.5, 0.7] Vector Similarity Metric: Cosine 𝚹 cos(Queen, King) = (0.3*0.5)+(0.9*0.7) √0.32 +0.92 * √0.52 +0.72 = 0.15+0.63 _ √0.9 * √0.74 = 0.78 _ √0.666 ≅ 0.03
- 18. Vector Similarity Metrics Euclidean - Spatial distance Cosine - Orientational distance Inner Product - Both With normalized vectors, IP = Cosine
- 19. Indexes organize the way we access our data
- 21. Hierarchical Navigable Small Worlds (HNSW) Source: https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf
- 24. Indexes Overview - IVF = Intuitive, medium memory, performant - HNSW = Graph based, high memory, highly performant - Flat = brute force - SQ = bucketize across one dimension, accuracy x memory tradeoff - PQ = bucketize across two dimensions, more accuracy x memory tradeoff
- 25. Vector databases efficiently store, index, and relate entities by a quantitative value
- 26. 03 Use Cases
- 27. What Does Vector Data Look Like?
- 28. RAG RAG Inject your data via a vector database like Milvus/Zilliz Query LLM Milvus Your Data Primary Use Case ● Factual Recall ● Forced Data Injection ● Cost Optimization
- 29. Common AI Use Cases LLM Augmented Retrieval Expand LLMs' knowledge by incorporating external data sources into LLMs and your AI applications. Match user behavior or content features with other similar behaviors or features to make effective recommendations. Recommender System Search for semantically similar texts across vast amounts of natural language documents. Text/ Semantic Search Image Similarity Search Identify and search for visually similar images or objects from a vast collection of image libraries. Video Similarity Search Search for similar videos, scenes, or objects from extensive collections of video libraries. Audio Similarity Search Find similar audios from massive amounts of audio data to perform tasks such as genre classification, or recognize speech. Molecular Similarity Search Search for similar substructures, superstructures, and other structures for a specific molecule. Question Answering System Interactive QA chatbot that automatically answers user questions Multimodal Similarity Search Search over multiple types of data simultaneously, e.g. text and images
- 30. Example Use Case
- 31. Example Use Case
- 32. Example Use Case
- 33. 04 Vector Database Architecture
- 34. Why Not Use a SQL/NoSQL Database? ● Inefficiency in High-dimensional spaces ● Suboptimal Indexing ● Inadequate query support ● Lack of scalability ● Limited analytics capabilities ● Data conversion issues TL;DR: Vector operations are too computationally intensive for traditional database infrastructures
- 35. Why Not Use a Vector Search Library? ● Have to manually implement filtering ● Not optimized to take advantage of the latest hardware ● Unable to handle large scale data ● Lack of lifecycle management ● Inefficient indexing capabilities ● No built in safety mechanisms TL;DR: Vector search libraries lack the infrastructure to help you scale, deploy, and manage your apps in production.
- 36. What is Milvus/Zilliz ideal for? ○ Advanced filtering ○ Hybrid search ○ Durability and backups ○ Replications/High Availability ○ Sharding ○ Aggregations ○ Lifecycle management ○ Multi-tenancy ○ High query load ○ High insertion/deletion ○ Full precision/recall ○ Accelerator support (GPU, FPGA) ○ Billion-scale storage Purpose-built to store, index and query vector embeddings from unstructured data at scale.
- 37. Meta Storage Root Query Data Index Coordinator Service Proxy Proxy etcd Log Broker SDK Load Balancer DDL/DCL DML NOTIFICATION CONTROL SIGNAL Object Storage Minio / S3 / AzureBlob Log Snapshot Delta File Index File Worker Node QUERY DATA DATA Message Storage Access Layer Query Node Data Node Index Node High-level overview of Milvus’ Architecture
- 38. Start building with Zilliz Cloud today! zilliz.com/cloud
- 39. | © Copyright 9/25/23 Zilliz 39 Appendix
- 40. Important Notes - Cosine, IP, and L2 are all the SAME rank order. - They differ in use case - L2 for when you need magnitude - Cosine for orientation - IP for magnitude and orientation - OR - Cosine = IP for normalized vectors
- 42. Basic Idea You want to use your data with a large language model
- 43. RAG vs Fine Tuning LLM Fine Tuning Augment an LLM by training it on your data Your Data “New” LLM Query Primary Use Case ● Style transfer
- 44. Takeaway Use RAG to force the LLM to work with your data by injecting it via a vector database like Milvus or Zilliz
- 45. Chunking Considerations Chunk Size Chunk Overlap Character Splitters
- 46. How Does Your Data Look? Conversation Data Documentation Data Lecture or Q/A Data
- 47. Examples
- 48. Examples
- 49. Examples
- 50. Examples
- 51. Examples
- 52. Examples
- 53. Your chunking strategy depends on what your data looks like and what you need from it. Takeaway:
- 54. Examining Embeddings Picking a model What to embed Metadata
- 55. Embeddings Strategies Level 1: Embedding Chunks Directly Level 2: Embedding Sub and Super Chunks Level 3: Incorporating Chunking and Non-Chunking Metadata
- 56. Metadata Examples Chunking ● Paragraph position ● Section header ● Larger paragraph ● Sentence Number ● … Non-Chunking ● Author ● Publisher ● Organization ● Role Based Access Control ● …
- 57. Your embeddings strategy depends on your accuracy, cost, and use case needs Takeaway:
- 58. Basic Idea Vector Databases provide the ability to inject your data via semantic similarity Considerations include: scale, performance, and flexibility
- 59. Milvus Architecture: Differentiation 1. Cloud Native, Distributed System Architecture 2. True Separation of Concerns 3. Scalable Index Creation Strategy with 512 MB Segments
- 60. Vector Databases are purpose-built to handle indexing, storing, and querying vector data. Milvus & Zilliz are specifically designed for high performance and billion+ scale use cases. Takeaway:
- 61. Vector Database Resources Give Milvus a Star! Chat with me on Discord!
- 62. Get Started Free Got questions? Stop by our booth! Milvus Open Source Self-Managed github.com/milvus-io/milvus Zilliz Cloud SaaS Fully-Managed zilliz.com/cloud