Beyond Limits: How GraphRAG Revolutionises Data Interaction
0 likes385 views
The document discusses the evolving landscape of generative AI and the role of retrieval augmented generation (RAG) and knowledge graphs in enhancing AI applications. It highlights the importance of integrating domain-specific knowledge to improve accuracy, explainability, and reliability of AI-generated responses, particularly in enterprise contexts. Additionally, it introduces 'graphrag' as a composite approach that merges graph analytics with generative AI to enrich data retrieval and contextual understanding.
Beyond Limits: How GraphRAG Revolutionises Data Interaction
- 1. Beyond Limits: GraphRAG Kristof Neys, Neo4j Field Engineering
- 2. Outline 1) Brave New World… 2) Why RAG? 3) Can Knowledge Graphs help? 4) From RAG to GraphRAG 5) Improve your R 6) An example…. 7) Does it work? Neo4j Inc. All rights reserved 2023 2
- 3. But First - A Word from our sponsor…
- 5. The State of Generative AI GENAI GENAI
- 6. GENAI GENAI The Good 💪 The State of Generative AI
- 7. GENAI GENAI The Good 💪 The State of Generative AI
- 8. GENAI GENAI The Good 💪 The State of Generative AI The Bad 👎
- 9. GENAI GENAI The Good 💪 The State of Generative AI The Bad 👎 GENAI
- 10. GENAI GENAI The Good 💪 The Bad 👎 The State of Generative AI The Ugly 😱 GENAI
- 11. GENAI GENAI The State of Generative AI The Good 💪 The Bad 👎 The Ugly 😱 GenAI Alone != Right Outcomes 🤯 GENAI
- 12. Challenges with GenAI: Stochastic Parrot? ● Lack of enterprise domain knowledge ● Inability to verify answers ● Hallucination ● Ethical and data bias concerns ● and more 12 Neo4j Inc. All rights reserved 2024 GenAI PARROT
- 13. 13 Managing AI risk is the biggest barrier to scaling AI initiatives1 Skepticism: Over half of business leaders are skeptical in adopting GenAI.2 Neo4j Inc. All rights reserved 2024 Explainability: Over 80% of executives worry about non-transparent nature of GenAI could result in poor or unlawful decisions.2 Reliability: Inaccuracy and hallucination are two of the most-cited risks of adopting GenAI technology at all levels of an organisation.3 1. Deloitte’s State of AI in the Enterprise 2. BCG’s Digital Acceleration Index Study 2023 3. McKinsey: The state of AI in 2023
- 14. 14 Neo4j Inc. All rights reserved 2024 How can enterprises use domain-specific knowledge to rapidly build accurate, contextual, and explainable GenAI applications? Problem Statement
- 15. Why RAG? And what is it anyway…
- 16. Retrieval Augmented Generation: The ability to dynamically query a large text corpus to incorporate relevant factual knowledge into the responses generated by the underlying language model
- 17. Neo4j Inc. All rights reserved 2023 17 RAG augments LLMs by retrieving up-to-date, contextual external data to inform responses: Retrieve - Find documents of interest for the user question Augment: Combine the user question with the relevant documents Generate: Feed enhanced prompt to an LLM and obtain answer Retrieval Augmented Generation Database of Truth RAG is becoming an industry standard
- 18. Why RAG With Vector Databases Fall Short Similarity is insufficient for rich enterprise reasoning Neo4j Inc. All rights reserved 2024 18 1 3 2 4 Only leverage a fraction of your data: Beyond simple “metadata”, vector databases alone fail to capture relationships from structured data Miss critical context: Struggle to capture connections across nuanced facts, making it challenging to answer multi-step, domain-specific, questions Vector Similarity ≠ Relevance: Vector search uses an incomplete measure of similarity. Relying on it solely can result in irrelevant and duplicative results Lack explainability: The black-box nature of vectors lacks transparency and explainability
- 20. Recap a Knowledge Graph A knowledge graph is a structured representation of facts, consisting of entities, relationships and semantic descriptions 20 Neo4j Inc. All rights reserved 2024 MICA ANDRE Name: “Andre” Born: May 29, 1970 Twitter: “@andre123” Name: “Mica” Born: Dec 5, 1975 CAR Brand “Volvo” Model: “V70” Description: ‘Blue external, red seats’ #days: 5/7 LOVES LOVES LIVES WITH O W N S D R I V E S Since: Jan 10, 2011
- 21. Knowledge Graphs – New & Improved! NOW WITH VECTORS!
- 22. Now with Vectors! Vectors as Node properties = Vector Search + Graph Traversal 22 Neo4j Inc. All rights reserved 2024 MICA ANDRE Name: “Andre” Born: May 29, 1970 Twitter: “@andre123” Name: “Mica” Born: Dec 5, 1975 CAR Brand “Volvo” Model: “V70” Description: #days: 5/7 LOVES LOVES LIVES WITH O W N S D R I V E S Since: Jan 10, 2011
- 23. Neo4j Inc. All rights reserved 2023 23 Neo4j - Vector Database Capabilities Vector Search Data Science Knowledge Graph ● Find nodes using an implicit similarity search in the vector index* and enrich with additional explicit relationships from the knowledge graph ● Hybrid Search with text ● Create vectors of network information using node embeddings Now a top 10 vector database on LangChain.
- 24. Neo4j Inc. All rights reserved 2023 24 By 2025, 50% of generative AI initiatives will have improved reliability and transparency by combining deep learning foundation models with knowledge graphs or other composite AI elements. Technological Implications of Generative AI, August 2023 Impact Radar for GenAI (2024)
- 25. From RAG to GraphRAG
- 26. GraphRAG Technique for richly understanding text datasets by combining text extraction, network analysis, LLM prompting and summarization into a single end-to-end system
- 27. Neo4j Inc. All rights reserved 2024 A Neo4j Knowledge Graph combined with LLM’s obtains some unique improvements: Accuracy - Obtain better answers compared to plain vector searches Specificity: domain specific, factual knowledge on your subject Explainability: Provide the user with more reasoning on how the results were obtained. Security: Role Based Access Control Retrieval Augmented Generation Evolving From RAG to GraphRAG
- 28. We are not making this up… Neo4j Inc. All rights reserved 2023 28
- 29. You need a better R…
- 30. Quick quiz for SWAG Neo4j Inc. All rights reserved 2023 30
- 31. Neo4j Inc. All rights reserved 2023 31
- 32. Neo4j Inc. All rights reserved 2023 32 Data Science on Graphs: Graph Data Science… Vector Search Graph Data Science Knowledge Graph Bring the context of your connected data into a format that other pipelines can ingest. The Largest Catalog of Graph Algorithms Graph Vector Embeddings for Machine Learning
- 34. Neo4j Inc. All rights reserved 2023 34 GraphRAG with Neo4j Find similar documents and content Identify entities associated to content and patterns in connected data Improve GenAI inferences and insights. Discover new relationships and entities Unify vector search, knowledge graph and data science capabilities to improve RAG quality and effectiveness Vector Search Graph Data Science Knowledge Graph
- 35. An example… RFP Generation GenAI App 35 Neo4j Inc. All rights reserved 2024
- 36. Why a KG Matters in RFP GenAI App? 36 Challenges Outcomes Time consuming to read previous RFP across multiple repositories Knowledge base to collect, store and retrieve domain-specific information Repetitive and manual tasks to synthesise the content Drive efficient, accurate, contextual and explainable way to streamline RFP responses Non-standard structure of RFP making it difficult to do data modelling Flexible storage that’s adoptable to the varying structure of an RFP Neo4j Inc. All rights reserved 2024
- 37. Anatomy of an RFP Document 37 AWS RFP Intro Objectives Proposal About the Company Financial Result Content Subsection 1 Subsection 2 Subsection 1.1 Content Content Content Content Content Content Content Content Content Content Neo4j Inc. All rights reserved 2024
- 38. Anatomy of an RFP Document 38 AWS RFP Intro Objectives Proposal About the Company Financial Result Content Subsection 1 Subsection 2 Subsection 1.1 Content Content Content Content Content Content Content Content Content Content Neo4j Inc. All rights reserved 2024
- 39. RFP Document as a Graph 39 AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Neo4j Inc. All rights reserved 2024
- 40. Why Neo4j KG Matters in RFP GenAI App? 40 Challenges Outcomes Time consuming to read previous RFP across multiple repositories Knowledge base to collect, store and retrieve domain-specific information Repetitive and manual tasks to synthesise the content Drive efficient, accurate, contextual and explainable way to streamline RFP responses Non-standard structure of RFP making it difficult to do data modelling Flexible storage that’s adoptable to the varying structure of an RFP Neo4j Inc. All rights reserved 2024
- 41. Why Neo4j KG Matters in RFP GenAI App? 41 Challenges Outcomes Time consuming to read previous RFP across multiple repositories Knowledge base to collect, store and retrieve domain-specific information Repetitive and manual tasks to synthesise the content Drive efficient, accurate, contextual and explainable way to streamline RFP responses Non-standard structure of RFP making it difficult to do data modelling Flexible storage that’s adoptable to the varying structure of an RFP Neo4j Inc. All rights reserved 2024
- 42. AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Knowledge Graph as the Knowledge Base Document in a KG Knowledge Graph 42 Neo4j Inc. All rights reserved 2024
- 43. Knowledge Graph as the Knowledge Base Document in a KG Knowledge Graph 43 Neo4j Inc. All rights reserved 2024 AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding
- 44. Why Neo4j KG Matters in RFP GenAI App? 44 Challenges Outcomes Time consuming to read previous RFP across multiple repositories Knowledge base to collect, store and retrieve domain-specific information Repetitive and manual tasks to synthesise the content Drive efficient, accurate, contextual and explainable way to streamline RFP responses Non-standard structure of RFP making it difficult to do data modelling Flexible storage that’s adoptable to the varying structure of an RFP Neo4j Inc. All rights reserved 2024
- 45. Why Neo4j KG Matters in RFP GenAI App? 45 Challenges Outcomes Time consuming to read previous RFP across multiple repositories Knowledge base to collect, store and retrieve domain-specific information Repetitive and manual tasks to synthesise the content Drive efficient, accurate, contextual and explainable way to streamline RFP responses Non-standard structure of RFP making it difficult to do data modelling Flexible storage that’s adoptable to the varying structure of an RFP Neo4j Inc. All rights reserved 2024
- 46. Accurate, Contextual and Explainable 46 Neo4j Inc. All rights reserved 2024 GenAI App Who is the main respondent of the AWS RFP?
- 47. Accurate, Contextual and Explainable 47 Neo4j Inc. All rights reserved 2024 Who is the main respondent of the AWS RFP? GenAI App Embedding Model User Question
- 48. Accurate, Contextual and Explainable 48 Neo4j Inc. All rights reserved 2024 Who is the main respondent of the AWS RFP? GenAI App Embedding Model User Question Vector Embedding
- 49. Accurate, Contextual and Explainable 49 Neo4j Inc. All rights reserved 2024 Who is the main respondent of the AWS RFP? GenAI App Embedding Model User Question Similarity Search using Neo4j Vector Index Vector Embedding
- 50. Accurate, Contextual and Explainable 50 Neo4j Inc. All rights reserved 2024 AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Similarity Search using Neo4j Vector Index Response from ABC Company Signed by John Smith He’s the General Manager
- 51. Accurate, Contextual and Explainable 51 Neo4j Inc. All rights reserved 2024 AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Contextual Knowledge Retrieval within Neo4j KG Response from ABC Company Signed by John Smith He’s the General Manager
- 52. Accurate, Contextual and Explainable 52 Neo4j Inc. All rights reserved 2024 AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Knowledge Retrieval to aid in Explainability Response from ABC Company Signed by John Smith He’s the General Manager
- 53. Accurate, Contextual and Explainable 53 Neo4j Inc. All rights reserved 2024 AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Fine Grained Access Control to prevent unwarranted Knowledge Retrieval Response from ABC Company Signed by John Smith He’s the General Manager
- 54. Accurate, Contextual and Explainable 54 Neo4j Inc. All rights reserved 2024 Who is the main respondent of the AWS RFP? GenAI App Embedding Model User Question Similarity Search using Neo4j Vector Index Vector Embedding
- 55. Accurate, Contextual and Explainable 55 Neo4j Inc. All rights reserved 2024 Who is the main respondent of the AWS RFP? GenAI App Embedding Model User Question Vector Embedding AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Response from ABC Company Signed by John Smith He’s the General Manager Similarity Result Similarity Search using Neo4j Vector Index Bedrock LLM
- 56. Accurate, Contextual and Explainable 56 Neo4j Inc. All rights reserved 2024 Who is the main respondent of the AWS RFP? GenAI App Embedding Model User Question Vector Embedding AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Response from ABC Company Signed by John Smith He’s the General Manager Similarity Result Similarity Search using Neo4j Vector Index The main respondent of the AWS RFP is John Smith, General Manager XYZ Group representing ABC Company (source: RFP_AWS.pdf, page: 38) AWS RFP Intro Objectives Proposal Content Chunk Content Chunk Content Chunk Financial Result About the Company Content Chunk Content Chunk Content Chunk Subsection 2 Subsection 1 Content Chunk Content Chunk Subsection 1.1 Content Chunk Content Chunk Content Chunk Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Vector Embedding Response from ABC Company Signed by John Smith He’s the General Manager Similarity + Contextual Result Bedrock LLM
- 57. How about Graph Data Science? 57 Enrich the measure of relevancy using graph algorithms. ● Page Rank to understand the importance of parts of documents ● Link Prediction to find hidden relationships that further contextualise the results ● Community Detection to group related parts of documents for more focused knowledge retrieval Neo4j Inc. All rights reserved 2024
- 58. Try it yourself… - Neo4j Graphbuilder
- 59. But does it REALLY work??
- 60. 61 Neo4j Inc. All rights reserved 2024
- 61. Neo4j Inc. All rights reserved 2023 62
- 62. Let’s Wrap up…
- 63. GraphRAG enables you to..: ● To leverage structural information across entities to enable more precise and comprehensive retrieval ● To perform advanced Graph analytics to enhance retrieval ● To have an accurate conversation with your data that is explainable 64
- 64. Neo4j Inc. All rights reserved 2023 65 Thank You