NYCMeetup07-25-2024-Unstructured Data Processing From Cloud to Edge
0 likes277 views
The document outlines an upcoming presentation on the benefits of integrating a cloud-native vector database into data and AI platforms, focusing on the Milvus database and its capabilities in handling unstructured data. It highlights the importance of vector databases in modern applications, includes information about community events, hiring announcements, and encourages engagement through meetups and storytelling. Various use cases and operational advantages of vector databases, particularly Milvus, are also elaborated upon.
![68 | © Copyright Zilliz
68
Semantic Similarity
Image from Sutor et al
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Man = [0.5, 0.2]
Queen - Woman + Man = King
Queen = [0.3, 0.9]
- Woman = [0.3, 0.4]
[0.0, 0.5]
+ Man = [0.5, 0.2]
King = [0.5, 0.7]
Man = [0.5, 0.2]
Neural Word Embeddings](https://image.slidesharecdn.com/nycmeetup07-25-2024-unstructureddataprocessingfromcloudtoedge-240725135315-c8a8acd6/85/NYCMeetup07-25-2024-Unstructured-Data-Processing-From-Cloud-to-Edge-68-320.jpg)
![69 | © Copyright Zilliz
69
Vector Similarity Measures: L2 (Euclidean)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
d(Queen, King) = √(0.3-0.5)2
+ (0.9-0.7)2
= √(0.2)2
+ (0.2)2
= √0.04 + 0.04
= √0.08 ≅ 0.28](https://image.slidesharecdn.com/nycmeetup07-25-2024-unstructureddataprocessingfromcloudtoedge-240725135315-c8a8acd6/85/NYCMeetup07-25-2024-Unstructured-Data-Processing-From-Cloud-to-Edge-69-320.jpg)
![70 | © Copyright Zilliz
70
Vector Similarity Measures: Inner Product
(IP)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Queen · King = (0.3*0.5) + (0.9*0.7)
= 0.15 + 0.63 = 0.78](https://image.slidesharecdn.com/nycmeetup07-25-2024-unstructureddataprocessingfromcloudtoedge-240725135315-c8a8acd6/85/NYCMeetup07-25-2024-Unstructured-Data-Processing-From-Cloud-to-Edge-70-320.jpg)
![71 | © Copyright Zilliz
71
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Vector Similarity Measures: Cosine
𝚹
cos(Queen, King) = (0.3*0.5)+(0.9*0.7)
√0.32
+0.92
* √0.52
+0.72
= 0.15+0.63 _
√0.9 * √0.74
= 0.78 _
√0.666
≅ 0.03](https://image.slidesharecdn.com/nycmeetup07-25-2024-unstructureddataprocessingfromcloudtoedge-240725135315-c8a8acd6/85/NYCMeetup07-25-2024-Unstructured-Data-Processing-From-Cloud-to-Edge-71-320.jpg)
NYCMeetup07-25-2024-Unstructured Data Processing From Cloud to Edge
- 1. 1 | © Copyright 10/22/23 Zilliz 1 | © Copyright 10/22/23 Zilliz Presented by: New York Unstructured Data Meetup
- 2. 2 | © Copyright 10/22/23 Zilliz 2 | © Copyright 10/22/23 Zilliz 2 | © Copyright 10/22/23 Zilliz 2 | © Copyright 10/22/23 Zilliz Tim Spann Principal Developer Advocate, Zilliz tim.spann@zilliz.com https://www.linkedin.com/in/timothyspann/ https://x.com/PaaSDev Unstructured Data Meetup | Host
- 3. 3 | © Copyright 10/22/23 Zilliz 3 | © Copyright 10/22/23 Zilliz Code of Conduct Be respectful and kind When communicating with all event participants, speakers, and hosts. Be considerate All ideas are welcome Be present and participate actively in discussions. Ask questions and reach out for help when needed. Report inappropriate behavior Any inappropriate behavior is not tolerated at this event. Inform a Zilliz team member immediately if you see any behavior deemed inappropriate
- 4. 4 | © Copyright 10/22/23 Zilliz 4 | © Copyright 10/22/23 Zilliz 4 | © Copyright 10/22/23 Zilliz 4 | © Copyright 10/22/23 Zilliz Milvus Open Source Self-Managed Zilliz Cloud SaaS Fully-Managed github.com/milvus-io/milvus Getting Started with Vector Databases zilliz.com/cloud
- 5. 5 | © Copyright 10/22/23 Zilliz 5 | © Copyright 10/22/23 Zilliz Zilliz is Hiring! Join our Team Zilliz.com/careers • Developer Advocate • Senior Software Engineer • Staff Software Engineer • Solutions Architect
- 6. 6 | © Copyright 10/22/23 Zilliz 6 | © Copyright 10/22/23 Zilliz Join the Milvus Discord!
- 7. 7 | © Copyright 10/22/23 Zilliz 7 | © Copyright 10/22/23 Zilliz Become a Speaker! Interesting in speaking at and/or sponsoring a Zilliz Unstructured Data Meetup? Fill out this form! 🎤🎤🎤
- 8. 8 | © Copyright 10/22/23 Zilliz 8 | © Copyright 10/22/23 Zilliz Have you built something cool using Milvus or Zilliz? We want to hear all about it. Share Your Story
- 9. 9 | © Copyright 10/22/23 Zilliz 9 | © Copyright 10/22/23 Zilliz Community Day Event September 13, 2024 Computer History Museum, Mountain View This event celebrates the diverse capabilities of AWS, showcasing cutting-edge technologies and practical applications across various domains.
- 10. 10 | © Copyright 10/22/23 Zilliz 10 | © Copyright 10/22/23 Zilliz Star Milvus for a chance to win a prize tonight!
- 11. 11 | © Copyright 10/22/23 Zilliz 11 | © Copyright 10/22/23 Zilliz Share your photos! #ZillizUnstructuredData @zilliz_universe, @milvusio @Zilliz, @Milvus
- 12. 12 | © Copyright 10/22/23 Zilliz 12 | © Copyright 10/22/23 Zilliz 12 | © Copyright 10/22/23 Zilliz 12 | © Copyright 10/22/23 Zilliz Welcome Speakers Unstructured Data Processing From Cloud to Edge RAG Pipelines with Apache NiFi Metadata Lakes for Next-Gen AI/ML TECH TALK 1 TECH TALK 2 TECH TALK 3 Tim Spann Principal Developer Advocate, Zilliz Chris Joynt Senior PMM, Cloudera Lisa N Cao Product Manager, Datastrato
- 13. 13 | © Copyright 10/22/23 Zilliz 13 | © Copyright 10/22/23 Zilliz 13 | © Copyright 10/22/23 Zilliz 13 | © Copyright 10/22/23 Zilliz
- 14. 14 | © Copyright 10/22/23 Zilliz 14 | © Copyright 10/22/23 Zilliz Join us at our next meetup! lu.ma/unstructured-data-meetup
- 15. 15 | © Copyright 2024 Zilliz 15 Unstructured Data Processing From Cloud to Edge Tim Spann @ Zilliz
- 16. Slides
- 17. Agenda In this talk I will do a presentation on why you should add a Cloud Native vector database to your Data and AI platform. He will also cover a quick introduction to Milvus, Vector Databases and unstructured data processing. By adding Milvus to your architecture you can scale out and improve your AI use cases through RAG, Real-Time Search, Multimodal Search, Recommendations Engines, fraud detection and many more emerging use cases. As I will show, Edge devices even as small and inexpensive as a Raspberry Pi 5 can work in machine learning, deep learning and AI use cases and be enhanced with a vector database.
- 18. 18 | © Copyright 2024 Zilliz 18 01 Introduction CONTENTS 02 AI Use Cases 03 Edge Devices Unstructured data, vector databases, traditional databases, similarity search Why Vector Database, Milvus, Use Cases, Infrastructure. Demos. How Milvus and AI are at the edge powering the future. Demos.
- 19. 19 | © Copyright Zilliz 19 01 Introduction
- 20. 20 | © Copyright Zilliz 20 - Unstructured Data is 80% of data - Vector Databases are the only type of database that can work with unstructured data - Examples of Unstructured Data include text, images, videos, audio, etc Why Vector Databases?
- 21. 21 | © Copyright Zilliz 21 Traditional databases were built on exact search
- 22. 22 | © Copyright Zilliz 22 …which misses context, semantic meaning, and user intent VS. Apple VS. Rising dough VS. Change car tire Rising Dough Proofing Bread ✔ ❌
- 23. 23 | © Copyright Zilliz 23 Vector Databases Where do Vectors Come From?
- 24. 24 | © Copyright Zilliz 24 The evolution of AI made the semantic search of unstructured data possible Search by Probability Statistical analyses of common datasets established the foundation for processing unstructured data, e.g. NLP, and image classification AI Model Breakthrough The advancements in BERT, ViT, CBT etc. have revolutionized semantic analysis across unstructured data Vectorization Word2Vec, CNNs, Deep Speech pioneered unstructured data embeddings, mapping the words, images, videos into high-dimensional vectors
- 25. 25 | © Copyright Zilliz 25 This new AI breakthrough requires new databases to fully unleash its potential Support multiple use case types Accommodate diverse data requirements, enhancing flexibility and effectiveness in varied operational contexts Scale as needed Enable robust handling of expanding data volumes and search demands Highly performant Ensures swift and accurate query responses, crucial for optimal user experience
- 26. 26 | © Copyright Zilliz 26 https://milvus.io/milvus-demos/reverse-image-search Show Me
- 27. 27 | © Copyright Zilliz 27 | © Copyright Zilliz 27 Introducing Milvus
- 28. 28 | © Copyright Zilliz 28 Why Even Use a Vector DB? Beyond High-Performance Search • CRUD Operations: Just like traditional databases, vector databases allow you to Create, Read, Update, and Delete data. • Data Freshness: Vector databases ensure your data remains up-to-date, reflecting the latest information for accurate searches. • Persistence: Your data is securely stored and persists even if the system restarts. • Availability: Your data is readily accessible for search and retrieval operations. • Scalability: Vector databases can handle growing data volumes efficiently.
- 29. 29 | © Copyright Zilliz 29 Complete Data Management • Data Management: Vector databases provide tools to manage your data effectively, including data ingestion, indexing, and querying. • Backup and Migration: Create backups of your data for disaster recovery and easily migrate your data between different systems. Why Even Use a Vector DB?
- 30. 30 | © Copyright Zilliz 30 Operational ease • Cloud or On-Premise Deployment: Vector databases can be deployed easily on various platforms, including cloud and on-premise environments. • Observability: Monitor the health and performance of your vector database to ensure optimal operation. • Multi-tenancy: Support multiple users or applications accessing the same database instance securely. Why Even Use a Vector DB?
- 31. 31 | © Copyright 8/16/23 Zilliz 31 | © Copyright 8/16/23 Zilliz 27K+ GitHub Stars 25M+ Downloads 250+ Contributors 2,600 + Forks Milvus is an open-source vector database for GenAI projects. pip install on your laptop, plug into popular AI dev tools, and push to production with a single line of code. Easy Setup Pip-install to start coding in a notebook within seconds. Reusable Code Write once, and deploy with one line of code into the production environment Integration Plug into OpenAI, Langchain, LlmaIndex, and many more Feature-rich Dense & sparse embeddings, filtering, reranking and beyond
- 32. 32 | © Copyright Zilliz 32 Why Not Use a SQL/NoSQL Database? • Inefficiency in High-dimensional spaces • Suboptimal Indexing • Inadequate query support • Lack of scalability • Limited analytics capabilities • Data conversion issues TL;DR: Vector operations are too computationally intensive for traditional database infrastructures
- 33. 33 | © Copyright Zilliz 33 Why Not Use a Vector Search Library? • Have to manually implement filtering • Not optimized to take advantage of the latest hardware • Unable to handle large scale data • Lack of lifecycle management • Inefficient indexing capabilities • No built in safety mechanisms TL;DR: Vector search libraries lack the infrastructure to help you scale, deploy, and manage your apps in production.
- 34. 34 | © Copyright Zilliz 34 What is Milvus ideal for? • Advanced filtering • Hybrid search • Durability and backups • Replications/High Availability • Sharding • Aggregations • Lifecycle management • Multi-tenancy • High query load • High insertion/deletion • Full precision/recall • Accelerator support (GPU, FPGA) • Billion-scale storage Purpose-built to store, index and query vector embeddings from unstructured data at scale.
- 35. 35 | © Copyright Zilliz 35 We’ve built technologies for various types of use cases Compute Types Designed for various compute powers, such as AVX512, Neon for SIMD, quantization cache-aware optimization and GPU Leverage strengths of each hardware type, ensuring high-speed processing and cost-effective scalability for different application needs Search Types Support multiple types such as top-K ANN, Range ANN, sparse & dense, multi-vector, grouping, and metadata filtering Enable query flexibility and accuracy, allowing developers to tailor their information retrieval needs Multi-tenancy Enable multi-tenancy through collection and partition management Allow for efficient resource utilization and customizable data segregation, ensuring secure and isolated data handling for each tenant Index Types Offer a wide range of 15 indexes support, including popular ones like HNSW, PQ, Binary, Sparse, DiskANN and GPU index Empower developers with tailored search optimizations, catering to performance, accuracy and cost needs
- 36. 36 | © Copyright Zilliz 36 Meta Storage Root Query Data Index Coordinator Service Proxy Proxy etcd Log Broker SDK Load Balancer DDL/DCL DML NOTIFICATION CONTROL SIGNAL Object Storage Minio / S3 / AzureBlob Log Snapshot Delta File Index File Worker Node QUERY DATA DATA Message Storage Access Layer Query Node Data Node Index Node Milvus’ fully distributed architecture is designed scalability and performance
- 37. 37 | © Copyright Zilliz 37 Milvus: From Dev to Prod AI Powered Search made easy Milvus is an Open-Source Vector Database to store, index, manage, and use the massive number of embedding vectors generated by deep neural networks and LLMs. contributors 267+ stars 27K+ downloads 25M+ forks 2K+
- 38. 38 | © Copyright Zilliz 38 Retrieval Augmented Generation (RAG) Expand LLMs' knowledge by incorporating external data sources into LLMs and your AI applications. Match user behavior or content features with other similar ones to make effective recommendations. Recommender System Search for semantically similar texts across vast amounts of natural language documents. Text/ Semantic Search Image Similarity Search Identify and search for visually similar images or objects from a vast collection of image libraries. Video Similarity Search Search for similar videos, scenes, or objects from extensive collections of video libraries. Audio Similarity Search Find similar audios in large datasets for tasks like genre classification or speech recognition Molecular Similarity Search Search for similar substructures, superstructures, and other structures for a specific molecule. Anomaly Detection Detect data points, events, and observations that deviate significantly from the usual pattern Multimodal Similarity Search Search over multiple types of data simultaneously, e.g. text and images …powers searches across various types of unstructured data
- 39. 39 | © Copyright Zilliz 39 Up to 100 billion vectors with K8s!
- 40. 40 | © Copyright Zilliz 40
- 41. 41 | © Copyright Zilliz 41 Entity
- 42. 42 | © Copyright Zilliz 42 Vector Space
- 43. 43 | © Copyright Zilliz 43 03 Edge Devices
- 44. 44 | © Copyright Zilliz 44 Milvus Lite pip install pymilvus
- 45. 45 | © Copyright Zilliz 45 Edge AI + Edge Vector Database Retrieval Augmented Generation (RAG) Run local LLM like OLLAMA Image Similarity Search Capture and search images at the edge for no network, local robotics, remote and secure. Video Similarity Search Search for similar videos, scenes, or objects from local videos. Audio Similarity Search Find similar audios in local audio for tasks like genre classification or speech recognition for robotics and sensing Anomaly Detection Detect data points, events, audio, images and observations that deviate significantly from the usual pattern at the edge Facial Recognition For security applications Customization Robots Benefits Lower latency Offline Security Localized storage
- 46. https://medium.com/@tspann/unstructured-data-processing-with-a-raspberry-pi-ai-kit-c959dd7fff47 Raspberry Pi AI Kit Hailo Edge AI
- 48. 48 | © Copyright Zilliz 48 | © Copyright Zilliz 48 RESOURCES
- 49. 49 | © Copyright Zilliz 49 Vector Database Resources Give Milvus a Star! Chat with me on Discord! https://github.com/milvus-io/milvus
- 50. 50 Unstructured Data Meetup https://www.meetup.com/unstructured-data-meetup-new-york/ This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs. This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
- 54. https://medium.com/@tspann/shining-some-light-on-the-new-milvus-lite-5a0565eb5dd9
- 55. Extracting Value from Unstructured Data Example • A company has 100,000s+ pages of proprietary documentation to enable their staff to service customers. Problem • Searching can be slow, inefficient, or lack context. Solution • Create internal chatbot with ChatGPT and a vector database enriched with company documentation to provide direction and support to employees and customers. https://osschat.io/chat
- 56. We provide deployment flexibility for different operational, security and compliance requirements BRING YOUR OWN CLOUD Zilliz BYOC Enterprise-ready Milvus for Private VPCs Deploy in your virtual private cloud Zilliz Cloud Milvus Re-engineered for the Cloud Available on the leading public clouds FULLY MANAGED SERVICE Coming Soon! Coming Soon! Milvus Most widely-adopted open source vector database Self hosted on any machine with community support SELF MANAGED SOFTWARE Local Docker K8s
- 57. 57 | © Copyright Zilliz 57 Well-connected in LLM infrastructure to enable RAG use cases Framework Hardware Infrastructure Embedding Models LLMs Software Infrastructure Vector Database
- 58. 58 | © Copyright 10/22/23 Zilliz 58 | © Copyright 10/22/23 Zilliz 58 This week in Milvus, Towhee, Attu, GPT Cache, Gen AI, LLM, Apache NiFi, Apache Flink, Apache Kafka, ML, AI, Apache Spark, Apache Iceberg, Python, Java, Vector DB and Open Source friends. https://bit.ly/32dAJft https://github.com/milvus-io/milvus AIM Weekly by Tim Spann
- 59. 59 | © Copyright 10/22/23 Zilliz 59 | © Copyright 10/22/23 Zilliz 59 https://flankworkspace.slack.com/ https://join.slack.com/t/flankworkspac e/shared_invite/zt-2fycjv241-~NRHZDt dfwDjlfvXK_Bz0A Join Our Slack and Interact with LLM
- 60. 60 | © Copyright 10/22/23 Zilliz 60 | © Copyright 10/22/23 Zilliz milvus.io github.com/milvus-io/ @milvusio @paasDev /in/timothyspann Connect with me! Thank you!
- 61. 61 | © Copyright 10/22/23 Zilliz 61 | © Copyright 10/22/23 Zilliz Join us at our next meetup! meetup.com/unstructured-data-meetup- new-york/
- 62. 62 | © Copyright Zilliz 62 T H A N K Y O U
- 63. 63 | © Copyright 10/22/23 Zilliz 63 | © Copyright 10/22/23 Zilliz Vector Database Resources Give Milvus a Star! Chat with me on Discord! https://github.com/milvus-io/milvus
- 64. 64 | © Copyright 10/22/23 Zilliz 64 | © Copyright 10/22/23 Zilliz Separation of storage and compute Yes Separation of query, insertion, index creation, and coordination. Yes. At the component level (which provides more fine-grained scalability) Multi-replication Yes Dynamic segment placement vs. static data sharding Dynamic segment placement Cloud-native Yes Regarding scalability, Milvus uses worker nodes for each type of action (components to handle connections, data nodes to handle ingestion, index nodes to index, and query nodes to search). Each node has its own assigned CPU and memory resources. Milvus can dynamically allocate new nodes to an action group, speeding up operations or reducing the number of nodes, thus freeing resources for other actions. Dynamically allocating nodes allows for easier scaling and resource planning and guarantees latency and throughput. Billion-scale vector support Yes
- 65. 65 | © Copyright 10/22/23 Zilliz 65 | © Copyright 10/22/23 Zilliz Roll-based Access Control (RBAC) Yes Disk Index support Yes Support for multi-vector/ multimodal Yes Support Search Types ANN, Range, Grouping Table-level partitions Yes Milvus is the fastest regarding search latency and throughput, supporting a billion scale-dataset. In addition, its exceptional approach to supporting multiple in-memory indexes and table-level partitions results in the high performance required for real-time information retrieval systems. Hybrid Search Yes with Scalar filtering and combined Sparse and Dense Vectors Index type supported 9 (FLAT, IVS_FLAT, IVF_SQ8, IVF_PQ, HNSW, ANNOY, BIN_FLAT, and BIN_IVF_FLAT) Purpose-built for Vectors Yes Database rollback Yes Data Consistency settings Yes
- 66. 66 | © Copyright 10/22/23 Zilliz 66 | © Copyright 10/22/23 Zilliz Support for both stream and batch of vector data Yes Binary Vector support Yes Multiple SDKs Python, Java, Go, C++, Node.js Milvus has an efficent Database rollback mechanism to ensure that... Milvus is a fully open source and independent project, maintained by a number of companies and individuals, some of whom also offer commercial services and support. Graduate of LF AI Data. License: Apache-2.0 license
- 67. 67 | © Copyright Zilliz 67 Choosing Vector Embedding Types https://milvus.io/docs/metric.md
- 68. 68 | © Copyright Zilliz 68 Semantic Similarity Image from Sutor et al Woman = [0.3, 0.4] Queen = [0.3, 0.9] King = [0.5, 0.7] Woman = [0.3, 0.4] Queen = [0.3, 0.9] King = [0.5, 0.7] Man = [0.5, 0.2] Queen - Woman + Man = King Queen = [0.3, 0.9] - Woman = [0.3, 0.4] [0.0, 0.5] + Man = [0.5, 0.2] King = [0.5, 0.7] Man = [0.5, 0.2] Neural Word Embeddings
- 69. 69 | © Copyright Zilliz 69 Vector Similarity Measures: L2 (Euclidean) Queen = [0.3, 0.9] King = [0.5, 0.7] d(Queen, King) = √(0.3-0.5)2 + (0.9-0.7)2 = √(0.2)2 + (0.2)2 = √0.04 + 0.04 = √0.08 ≅ 0.28
- 70. 70 | © Copyright Zilliz 70 Vector Similarity Measures: Inner Product (IP) Queen = [0.3, 0.9] King = [0.5, 0.7] Queen · King = (0.3*0.5) + (0.9*0.7) = 0.15 + 0.63 = 0.78
- 71. 71 | © Copyright Zilliz 71 Queen = [0.3, 0.9] King = [0.5, 0.7] Vector Similarity Measures: Cosine 𝚹 cos(Queen, King) = (0.3*0.5)+(0.9*0.7) √0.32 +0.92 * √0.52 +0.72 = 0.15+0.63 _ √0.9 * √0.74 = 0.78 _ √0.666 ≅ 0.03
- 72. 72 | © Copyright Zilliz 72 Why RAG? RAG vs. LLM - Knowledge of LLM is out-of-date - LLM can not get your private knowledge - Hallucinations - Transparency and interpretability RAG vs. Fine-tune - Fine-tune is expensive - Fine-tune spent much time - RAG is pluggable
- 74. …and cannot process increasingly growing unstructured data *Data Source: The Digitization of the World by IDC 20% Other newly generated data in 2025 will be unstructured data 80%
- 75. 75 | © Copyright Zilliz 75 Milvus Dependencies https://zilliz.com/blog/Milvus-server-docker-installation-and-packaging-dependencies 🧱 Main Dependencies: ● FAISS 🔍 (vector search) ● etcd 🔑 (metadata store) ● Pulsar/Kafka 📢 (messaging) ● Tantivy 🔎 (text search) ● RocksDB 💾 (storage) ● Object Storage 🗄 (Minio/S3/GCS/Azure Blob Storage) ● Kubernetes 🐳 (containerization) ● StorageClass & Persistent Volumes 🗄(Storage Management for etcd and Pulsar) ● Prometheus & Grafana 📈 (monitoring) 📦 Docker Image Size: ~500MB 🆕 Release Frequency: ~1x per month, with frequent minor releases 🛠 SDKs Available: Python 🐍, Node 🌐, Go 🐹, C# 💻, Java ☕, Ruby 💎 💻 Python SDK Installation: pip install pymilvus ✅ Version Compatibility: Ensure SDK and Milvus server versions match (major.minor)
- 76. …different types of data and schemas needs to be thoroughly planned ahead of time
- 77. 77 | © Copyright Zilliz 77 • Search Quality - Hybrid Search? Filtering? • Scalability - Billions of vectors? • Multi tenancy - Isolating Multi-Tenant data • Cost - Memory, disk, S3? • Security - Data Safety and Privacy TL;DR: Vector search libraries lack the infrastructure to help you scale, deploy, and manage your apps in production. Why Not Vector Search Libraries?
- 78. 78 | © Copyright Zilliz 78 Why Not Use a SQL/NoSQL Database? • Inefficiency in High-dimensional spaces • Suboptimal Indexing • Inadequate query support • Lack of scalability • Limited analytics capabilities • Data conversion issues TL;DR: Vector operations are too computationally intensive for traditional database infrastructures
- 79. 79 | © Copyright Zilliz 79 What is Milvus/Zilliz ideal for? • Advanced filtering • Hybrid search • Multi-vector Search • Durability and backups • Replications/High Availability • Sharding • Aggregations • Lifecycle management • Multi-tenancy • High query load • High insertion/deletion • Full precision/recall • Accelerator support (GPU, FPGA) • Billion-scale storage Purpose-built to store, index and query vector embeddings from unstructured data at scale.
- 80. 80 | © Copyright Zilliz 80 Vector Databases are purpose-built to handle indexing, storing, and querying vector data. Milvus & Zilliz are specifically designed for high performance and billion+ scale use cases. Takeaway:
- 81. 81 | © Copyright Zilliz 81 Inverted File Index Source: https://towardsdatascience.com/similarity-search-with-ivfpq-9c6348fd4db3
- 82. 82 | © Copyright Zilliz 82 HNSW Source: https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf
- 83. 83 | © Copyright Zilliz 83 SQ
- 84. 84 | © Copyright Zilliz 84 Open Source Deploy fully managed or “Bring Your Own Cloud” (BYOC) Commercial Offerings Zilliz Cloud Optimized Milvus with essential data and security tools for a high-performing vector search platform VECTOR SEARCH ENGINE VECTORDB BENCHMARK TOOL VECTOR DATABASE SEMANTIC CACHE FOR LLM QUERIES GPT-Cache Product Portfolio GUI for Milvus
- 85. 85 | © Copyright Zilliz 85 Couple of Customers
- 86. 86 | © Copyright Zilliz 86 Basic Idea You want to use your data with a large language model
- 87. 87 | © Copyright Zilliz 87 Cloud Service Provider Data Platform GenAI Tooling Chip Manufacturer Partner with Industry Leaders
- 88. 88 | © Copyright Zilliz 88 Composite Identifiers • Combine multiple fields to create a unique identifier. Example: Combine user_id and timestamp to create unique IDs for user-generated content. E.g. {"id": "user123_20240606T123000"} Hierarchical IDs • Use hierarchical structures for complex data sets. Example: For a hierarchical document system, use IDs like {"id": "projectA_chapter1_section2"} Identifier Strategies