Neo4j Graph DB & LLM.graphs & genAI introduction & cheatsheet.with comments.pdf
0 likes583 views
This document presents a comprehensive exploration of the intersection between graph technology and large language models (LLMs), highlighting the speaker's journey from linguistics and data processing to specializing in graph technologies. Key concepts covered include the structure of label property graph databases, vector similarity searches, and retrieval-augmented generation (RAG) methodologies that enhance LLM capabilities through structured databases. The document also addresses concerns regarding the societal implications of machine-generated text, particularly its effects on language perception and development in younger generations.
Neo4j Graph DB & LLM.graphs & genAI introduction & cheatsheet.with comments.pdf
- 1. - Thank you Amy Hodler, for the opportunity to present this exploration of the graph & LLM world at GraphGeeks. Also thank you for the feed back on the first version of the document that helped make the completed version I’m presenting today. - To make sure we understand each other, to left is what I would call charts, it is sometimes called « graph » but when I’m saying graph I mean what is graphically represented right of the slide: stuff related to other stuffs - A graph is a formal representation that explicitly represent relations 1
- 2. - Let me first say a few words about how I got here today, talking about graphs and LLMs - Languages in their different aspects are the common thread in my different experiences : foreign languages, linguistics, NLP and of course formal languages for computer programming - In the late 90s early 2000, NLP was not about LLM, we were working with linguistic formalisms. A lot of them where base on a tree structure. Since a tree structure is a specific kind of graph, I consider this the first step of my journey in the world of graphs. That is also when I started doing data processing. - In 2009, I started using TheBrain, which is a personal documentation tool that allows you to organize all your digital documents in a graph structure. It has a graphical interface that allows you to actually draw relations with the mouse and visualize them on screen. So this is how I started extensively using a graph structure, with an intuitive approach. The mathematical theory came in later, my first approach of graphs was intuitive - At the time I was also doing data processing and visualizations for companies - Then for 5 years I worked for a research team who was building a database for epidemiologists. My job was to create and adjust web applications to help them process different kinds of data. - I first heard of Neo4j graph db in a meetup in 2013 and rapidly became a huge fan of cypher’s, the graph query language. So in 2019, I quitted my job to train myself and specialize on graph technologies : both database and visualization - In November 2022, when the chatgpt meteorite hit the tech world, I first tried to convince myself that LLMs where just another Californian hype that would pass, and for a while, tried to remained focus on graph technologies. But in August last year Neo4j released a version that included vector index and vector similarity search and graph RAG started to be all over the graph tech world. Some presentation seemed to good to be true, but still there also seemed to be something to it, especially for leveraging documentation which is one of main subject of interest, so I decided to look into it. - What I’m sharing today is some landmark concepts that should help you navigate the field. I’ve represented them visually so anyone, even with no technical background can get a general idea of what can be done. 2
- 3. Here is the menu: We will be starting with an introduction to - Graphs - LPG - Vector search - Text generation with LLM, what is RAG and Graph RAG We will then see different kinds of implementation : with Neo4j core technology and pluggins, with the Neo4j Python module and with LangChain, the LLM orchestration platform We will talk about limitations And get an overview of use cases that leverage graph and LLM Before wrapping up and providing some resources for further exploration of the field 3
- 4. Label Property Graph Databases is a type od database that is based on a graph structure. Basic elements of a LPG db are Nodes with one or several Label(s), Relations and Properties, on nodes and relations. 4
- 5. What I like about cypher, the LPG DB query language is that it is visual : rounded parenthesis represent nodes and square parenthesis represent relationships. For visual thinkers, it makes it easy to model and think in graphs. NB: Links in slides are not active in this version with comments. You can get the slides with activated links here https://www.e- tissage.net/graphs-llm-an-introduction/ 5
- 6. 6
- 7. In a graph database, just like you can have text or number properties, embeddings can now be set as a property of nodes 7
- 8. From the embeddings set on nodes, you can build an vector index 8
- 9. With embeddings on nodes and a vector index, vector similarity can be done between embeddings in the database 9
- 10. Vector similarity can also be done between embeddings in the database and an external question. 10
- 11. - That was for semantic / vector similarity search: in that case, LLM are used to produce an embedding of text. You can then find similar embeddings in an index, by vector similarity search. - Let’s now see how LLM are used for text generation 11
- 12. - First you embbed the question (just like before), that is you turn text into a vector. - Then you use the LLM again, for the stochastic information it contains, meaning that the generated text is the most probable sequence, given the embedding of the question and the model - The important thing to realise here is that LLM encode 2 things : - Information contained in the data it’s been trained upon - Capacity to produce humain like sentences 12
- 13. If inputted with questions about a subject it has not been trained upon, like more recent or private information, LLM fail to return a significant response. But if together with the question you provide the LLM with the required information, the LLM will output it in a human like sentence 13
- 14. Providing context is what is used for Retrieval Augmented Generation (RAG) In an emerging field like the graph & LLM field a confusing aspect is that the terminology is not settled yet, you need to expect some variability in the way different authors use of some terms. Generation prompt = question + context + system prompt First time adjusting a system prompt has been my weirdest experience as a developper : the youngest need to realise that my generation of developper started out manually numbering lines, to get the thing to execute commands in the proper order. Take a second to realise the level of formalisation this is. I think over the years, I have develop a specific area in my brain that automatically switches to « hyper formal », as soon as I address a computer. So instructing an LLM in plain not only English but even in French, has felt very very weird… 14
- 15. - For me, graph RAG is a specific kind of RAG, in which the external specific, private / updated knowledge base used to provide the context is a db structured in graph. (Note that the term is sometimes used in a wider sense, including for designating things that do not involve RAG. Imho, this is when stretching the sense of the term – for the sake of hype ? - is going too far…) - With graph RAG (= RAG where the generation is augmented with retrieval in a graph structured DB ), 2 kinds of relations are used : - stochastic relations of vector similarity - implicit (properties of same node) or explicit (neighbouring nodes) deterministic relations to produce the text given as context to the generation prompt 15
- 16. The whole point of graph RAG is not just to use properties on the node returned by vector similarity search, you can leverage the power of any graph pattern matching around that node. In this example nodes representing movies are related to a node for the serie it is part of and all other episodes are also related to the :Serie, so when an episode is returned by the vector similarity search you can grab the plot of all episodes to feed the context sent to the generation LLM, This way, you can get any related information, that is you can get context of the node returned by vector similarity search, in a very detailed way. This example is detailed in Christopher Bergmans’ article that describes an implementation with Neo4j dabase engin and the genAI and the APOC plugins 16
- 17. The second way to do Graph RAG with Neo4j is with the Python module. It just changed its name from neo4j-genai to neo4j-graphrag. It is currently subject to frequent change. The LangChain integration that we will see next currently seems in a more stable state 17
- 18. To do graphRAG with Python, you will need retrievers 18
- 19. When searching a retriever by vector similarity search, you can add deterministic filters by filtering on properties. 19
- 20. Another kind of retriever you can use is by combining the results of a vector similarity search and a full text (Lucene) search. (Apache Lucene is a search algorithm that uses a relevance scoring algorithm to rank the documents based on how relevant they are to the search query. It supports advanced search queries like Boolean operators AND, OR, NOT, wildcard searches, proximity searches, and phrase searches) 20
- 21. Graph RAG can be done with the Python module, just like we have seen before. Here is the code and the link to a Will Tai article to get the details 21
- 22. - We have seen that Vector similarity search and graph RAG can be done with - genAI + APOC plugins on Neo4j core technology, - with a Python module that contains a baseline driver to a Neo4j graph db + graphRAG classes, including several types of retrievers - In the LLM world, you have what are called orchestration platforms that are meant for chaining several component with LLMs. It is the way to implement graph RAG that we will see now. 22
- 23. I will not go in too much details of the next 3 slides : they describes 3 ways to build a Neo4j based vector store in langchain. You can refer to the documents linked on the slide, to get the details. (NB: Links in slides are not active in this version with comments. You can get the slides with activated links here https://www.e-tissage.net/graphs-llm- an-introduction) Basically you can build the vector store upon an existing index in the database 23
- 24. The from_existing_graph method will build a vector store for nodes with a certain label, on the indicated list of properties 24
- 25. The from_document method uses the LangChain Document class that is meant for different kinds of processing with documents (chunking etc) If you have documents in such classes, you can directly build a Neo4j backed vector store 25
- 26. Once you have a Neo4j backed LangChain vector store, you can do in LangChain, a vector similarity search, just by invoking the similarity_search method on the store. It will get you the most similar embeddings and the corresponding node just like we have seen before. 26
- 27. Now retrieval query. This is how you can fine-tune the graph pattern matching around the node returned by the vector similarity search In this case, the purpose is to grab the previous and next chunks to the one that is most similiar to the question. The similarity search will return a node and a score variable (fixed names). So you can write any cypher query anchored on that node, and return any property of any node in the pattern as metadata Ex : source metadata + concatenation of the text property of the returned + previous and next nodes 27
- 28. When you have a vector store it is very straight forward to turn it into a retriever, with the as_retriever method. As a retriever, a vector store can be chained with other LangChain elements. 28
- 29. Instead of using vector similarity search, to match nodes in the graph database, there are other ways to grab complementary information to send as context to the generation prompt. In this example (see Tomaz Bratanic article and notebook), an entity extraction, with a Lucene full-text search build on extracted entities is used in combination with a vector similarity search. 29
- 30. In this example, the retrieval in the graph database is done without vector similarity search. With an adequate system prompt, the LLM is instructed to turn the user’s question into a cypher query. That is cypher generation. And then, the cypher query is run against graph database to return the context sent to the LLM in the generation prompt. 30
- 31. Here is one of my favorite feature of the combination of graphs with LLM: one of LLM limitation is that the response they generate is not explicable, you have no way to know where the produced sequence of words comes from. If you augment generation with a graph structured database, you can provide metadata about the piece of text that has been used, that is metadata about the source that the LLM reformulated. In this example, the text of the state of the Union is chopped into chunks, with an embedding for each chunk of text, so when the vector similarity search returns a node, you can provide the chunk id that will be returned with the answer. I think this is an awesome feature because it gives the user the possibility to check the human written text on which the generated response is based. 31
- 32. - So far we have seen how LLMs and graphs can be articulated for information retrieval - LLMs are also used to build graphs, often called KG. What the LLM does is entities and relationships extraction, that can then be « knitted » into a graph. - Neo4j has a graph builder that you can use to automatically extract entities and relation on your documents, with no code 32
- 33. If you want to have a hand on the parameters, you can try the LangChain LLMGraphTransformer class. The allowed _node allowed_relationship parameters restricts the list of nodes and relation that can be recognized. This helps prevent the default behavior that tends to over generate, especially entities. 33
- 34. I would not dare pretend this presentation of how graphs are used in articulation with LLM is exhaustive, since another usage is poping out every other morning,,, Let me just mention this lead. Have not looked into it but looks promising : graph can also be used to organise corrective RAG rules. 34
- 35. 35
- 36. - The 2 main domains for which I’ve seen use cases for graphs and LLM is recommendation systems and leveragong the content of different kinds of documentation - If you are aware of use cases that do not fit in one of those categories, please share in the chat - To be honest, I’m quite concerned about the effect, exposing us humans to large proportion of machine generated text will have in the long term. People my age will be ok, because our psychological building is pretty completed already, but what will happen to the young generation, if they are exposed to a large proportion of machine generated text ? We have seen how over exposure to screens hinder children developments. Since langage is the core of human psychee, I do believe we should be careful about how much we mess up with it by over exposing ourselves to machine generate text and speech. For example, I’ve been approached by I guy who created a start up to let people have a personnal bot answer emails for them. Spam is enough of a nuisance, I hate the idea of having to read machine generated rant espacially if I have no way to determine it is machine generate, because it is issued by the email address of someone I know. - This is to say email generation are really not my favorite use case and I believe, as a society, we should really think about what limit we want to put to this machine generated production of text and speech. Everyone sure needs to advertise for what we do or sell, and text generation will be used for that, but as professionnal of the field, it is also important that we give a thought to the limit we want to put to it, in order to get our social priorities right, - There are a lot of POC description out there, the use cases I’ve mentionned here are the ones that seems the closest to real life production deployment - The usecase I’m really excited about is how graph RAG and entity and relation extraction can leverage the content of all sorts of documentations 36
- 37. - By definition, structure is about the arrangement of and the relations between parts or elements of something complex. - So let’s keep in mind that structure is about elements related to other elements, which means building a graph is making a structure explicit and conversely, making a structure explicit can be done by building a graph. 37
- 38. - LLM produce the most probable words, given some statistics learned on a corpus and the embedding of a generation prompt (= question + context information + instructions about what is expected.) Those statistical production have proven astonishingly good at taking the form of what we all recognize as human language. LLM have something that has the form of a linguistic competence, but what they produce is a stochastic result. - Graphs on the other hand, represent information explicitly, in an accurate and deterministic way. They are a very good way to represent domain specific / updated information to feed LLM with. With graphs, you can get LLM to so to say focus on specific information that have been extracted based on explicit relations. The first release of ChatGPT was ranting about the difference between chicken and cows eggs. Now, when asked about the difference it says cow eggs do not exists, and it also says it uses KG… LLM are also bad at what involves mathematical logic, aggregating elements to count them. It is another area whe graph can help. Lack of explainability is another weakness of LLM : all the information is mixed together, when you get an answer, there is no way to figure where it comes from. If you have part of the generation processes base on deterministic relations, you can for example provide the piece of text the generated answer is based upon. The graph adds interpretability to the process 38
- 39. 39