Boosting MySQL with Vector Search -THE VECTOR SEARCH CONFERENCE 2025 .pdf
0 likes118 views
As the demand for vector databases and Generative AI continues to rise, integrating vector storage and search capabilities into traditional databases has become increasingly important. This session introduces the *MyVector Plugin*, a project that brings native vector storage and similarity search to MySQL. Unlike PostgreSQL, which offers interfaces for adding new data types and index methods, MySQL lacks such extensibility. However, by utilizing MySQL's server component plugin and UDF, the *MyVector Plugin* successfully adds a fully functional vector search feature within the existing MySQL + InnoDB infrastructure, eliminating the need for a separate vector database. The session explains the technical aspects of integrating vector support into MySQL, the challenges posed by its architecture, and real-world use cases that showcase the advantages of combining vector search with MySQL's robust features. Attendees will leave with practical insights on how to add vector search capabilities to their MySQL systems.
Boosting MySQL with Vector Search -THE VECTOR SEARCH CONFERENCE 2025 .pdf
- 1. Boosting MySQL with Vector Search MyVector Alkin Tezuysal THE VECTOR SEARCH CONFERENCE June 2025 1
- 2. Let’s get connected! Alkin Tezuysal - Director of Services @AltinityDB ● Linkedin : https://www.linkedin.com/in/askdba/ Open Source Database Evangelist ● Previously ChistaDATA, PlanetScale, Percona and Pythian as Senior Technical Manager, SRE, DBA ● Earlier in life Enterprise DBA , Informix, Oracle, DB2 , SQL Server ● Recent Recognitions: ○ Most Influential in Database Community 2022 - The Redgate 100 ○ MySQL Cookbook, 4th Edition 2022 - O'Reilly Media, Inc. ○ MySQL Rockstar 2023 - Oracle (MySQL Community) ○ Database Design and Modeling with PostgreSQL and MySQL 2024 - <Packt> ○ Oracle ACE Pro 2025 - Oracle @ask_dba @2025 Altinity, Inc. Born to Sail, Forced to Work! 2
- 3. What is Artificial Intelligence? Artificial Intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. AI systems are capable of performing tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. 3
- 4. Vectors and Dimensions Objects Dimensions Data Points Mathematical objects representing magnitude and direction (velocity, force) When dimensions increase data points increase and they turn into embeddings Represented as arrays or ordered lists of numbers MyVector 4
- 5. Vector Embeddings Mathematical representations of data in a high-dimensional space. Semantic Search Pre-Trained Vector Search Capture semantic meaning and relationships between words, phrases, or other data. BERT, OpenAI, Word2Vec, CLIP for images It is a technique that retrieves the most similar items by comparing vector embeddings using distance or similarity metrics in high-dimensional space. MyVector 5
- 6. Pre-Trained Models • Natural Language (Text) Embeddings • Multilingual Embeddings • Image & Vision Embeddings MyVector • Audio / Video Embeddings • Cross-modal / Multimodal 6
- 7. Benefits of Vector Search ● Efficient and relevant search results. ● Improved relevance for text and image search. ● Semantic understanding. ● Supports AI-based applications. ● Leverage and unlock value from live and historical business data in the MySQL database : e.g conversation logs, clinical records, support tickets, persona descriptions, medical images, product catalog, legal filings etc. MyVector 7
- 8. Vector Support Helps ● Traditional search struggles with high-dimensional data and semantic understanding. ● Inefficient for AI and ML applications that rely on vector embeddings. ● Vector support enhances performance and relevance for similarity searches. ● Enables new AI-driven applications directly within MySQL. ● Unlocks advanced data analysis techniques using vector embeddings. MyVector 8
- 9. Why Vector Search in MySQL? ● MySQL World’s number one open-source relational database ● Vector search allows for more efficient and relevant search results. ● Improved relevance for text and image search with semantic understanding. ● MyVector plugin extends MySQL with vector search capabilities. ● Supports a wide range of AI-based applications. P.S: MySQL does not come with extensibility interfaces to add new data types, and no support to add new index types and to add new access methods 9
- 10. Vector Search Use Cases Use Cases: Natural Language Processing (NLP): Words or sentences are converted into vectors where similar words or phrases are closer in vector space. Recommender Systems: Items or users can be embedded to find similarities or predict preferences. Image Recognition: Images can be encoded into vectors for tasks like similarity search or classification. 10
- 11. Vector Databases Vector Database Providers ● Pinecone ● Qdrant ● Milvus ● Weavite ● Chroma Vector Store Providers ● ClickHouse ● Elasticsearch ● Singlestore ● Typesense ● Others ○ MySQL ○ MariaDB ○ PostgreSQL ○ TiDB 11
- 12. Vector Databases vs Traditional Databases 12
- 14. Model Context Protocol (MCP) 14
- 15. MySQL Vector Plugin Architecture The MyVector Plugin, discussed earlier, leverages this architecture. Here's how it fits: ● UDFs in the plugin: Adds vector-related SQL functions. ● Data storage : Use varbinary in MySQL 8.x and VECTOR in MySQL 9.x. ● Indexing: Integrates vector similarity search via HNSW indexing. ● Similarity Search - Intuitive syntax for search using SQL ● Administration - Simple MySQL stored procedures 15
- 16. MySQL Vector Plugin Build /// get the MyVector sources $ cd mysql-server/src/plugin $ git clone https://github.com/p3io/myvector-dev/ ./myvector /// Generate makefile for the new plugin $ cd mysql-server/bld $ cmake .. <other options used for this build > /// Build the plugin $ cd mysql-server/bld/plugin/myvector $ make 16
- 17. MySQL Vector Plugin Install /// Copy the MyVector plugin shared library to the MySQL installation plugins $ cp mysql-server/bld/plugin_output_directory/myvector.so /usr/local/mysql/lib/plugin/ /// Register the MyVector plugin and create MyVector stored procedures. $ cd mysql-server/plugin/myvector /// Connect to 'mysql' database as 'root' $ mysql -u root -p mysql mysql> source myvectorplugin.sql 17
- 18. MySQL Vector Plugin Architecture ● MySQL V8.X - InnoDB + Plugin + Varbinary Data Type ● MySQL V9.X - InnoDB + Plugin + Vector Data Type mysql> select udf_name from performance_schema.user_defined_functions where udf_name like "myvector%" ; +-----------------------------+ | udf_name | +-----------------------------+ | | myvector_is_valid | | myvector_distance | | myvector_display | | myvector_ann_set | | myvector_row_distance | | myvector_construct | +-----------------------------+ 18
- 19. MyVector Features High speed, parallel build of HNSW index Specify recall v/s latency parameters : M, ef, ef_search HNSW index is incrementally persisted after initial build Online update of index via binlog read & parse of DMLs HNSW Index is crash-safe and is recovered on MySQL instance crash Write amplification to redo log/undo log avoided 19
- 20. MySQL Vector Plugin Examples MySQL 8.X Ref: https://nlp.stanford.edu/projects/glove/ mysql > create table words50d( wordid int primary key, word varchar(200), wordvec MYVECTOR(type=HNSW, dim=50,size=400000, M=64,ef=100) ); – Load the Data (insert …) mysql> call myvector_index_build ('test.words50d.wordvec','wordid'); 20
- 21. MySQL Vector Plugin Examples MySQL 9.1 mysql > create table words50d( wordid int primary key, word varchar(200), wordvec MYVECTOR(type=HNSW, dim=50,size=400000, M=64,ef=100) ); – Load the Data (insert …) mysql> call myvector_index_build ('test.words50d.wordvec','wordid'); 21
- 23. Future Trends Self-driving cars are becoming more sophisticated. AI-driven adaptive learning platforms tailor education to individual student needs. AI is improving in understanding and generating human language. Autonomous Vehicles Personalized Education Natural Language Processing 23
- 24. Conclusion Vector search technologies like MyVector are poised to transform how organizations manage and retrieve unstructured data, enabling smarter AI-driven applications. As we unlock new capabilities in semantic understanding and real-time relevance, it’s essential to balance technical innovation with ethical data handling and responsible AI integration—ensuring we realize the full potential of vector-powered intelligence safely and effectively. MyVector 24
- 26. References Introducing MyVector Plugin — Vector Storage & Similarity Search in MySQL - HNSW, ANN | Medium Introducing MyVector Plugin — Vector Storage & Similarity Search in MySQL — Part 2 | by Shankar Iyer | Medium MyVector Plugin — Part 3 — Nearest Neighbour Search with other predicates and the SQL Optimizer | by Shankar Iyer | Medium MySQL Vector Datatype: create your operations (part 1) MySQL Vector Datatype: create your operations (part 2) - lefred blog O'Reilly/Safari Books https://asciinema.org/a/QuRroM7n06vxvK78pYYKoCWNG https://asciinema.org/a/KZeSifbD0QpHYmYUep5SiaCMA https://asciinema.org/a/YTIg12ClZAh38qUcgsBUAdSA7 https://asciinema.org/a/4uro9wYgFijuwP3EdJvp0tRMj https://asciinema.org/a/O7rNs2OzLXyUja0bwWcldXsX9 https://asciinema.org/a/RJs3CFbuShQGeXkCSxDDG0n3i https://asciinema.org/a/6F0hDcnFT4XMqi5ALSeTNkCbO FOSDEM 2025 - Boosting MySQL with Vector Search: Introducing the MyVector Plugin 26