MemGPT: Introduction to Memory Augmented Chat
0 likes203 views
The document discusses MemGPT, a memory-augmented language model that enhances long-term memory and contextual understanding for AI interactions. It emphasizes the importance of memory for LLMs, enabling them to manage user messages, external data, and system alerts effectively, while also facilitating self-editing capabilities. Additionally, it outlines the integration of MemGPT with Docker for deployment and supports both streaming and non-streaming API functionalities.
![MemGPT streaming API - token streaming
CLI: memgpt run --stream
REST API: use the stream_tokens flag [PR #1280 - staging]](https://image.slidesharecdn.com/copyofunstructureddatameetupmay212024-240606220847-0624cb0d/85/MemGPT-Introduction-to-Memory-Augmented-Chat-48-320.jpg)
MemGPT: Introduction to Memory Augmented Chat
- 1. MemGPT why we need memory-augmented LLMs
- 2. 👋 Charles Packer ● PhD candidate @ Sky / BAIR, focus in AI ● Author of MemGPT ○ First paper demonstrating how to give GPT-4 self-editing memory (AI that can learn over time) ● Working on agents since 2017 ○ “the dark ages” ○ 5 BC = Before ChatGPT 📧 cpacker@berkeley.edu 🐦 @charlespacker
- 3. Agents in 2017 🙈
- 4. For LLMs, “memory” is everything memory is context context includes long-term memory, tool use, ICL, RAG, …
- 5. For LLMs, “memory” is everything “memory” =
- 6. MemGPT - giving LLMs real “memory” GPT
- 7. Why is this the “best” AI product?
- 9. Search engine AI assistant
- 10. Search engine AI assistant
- 12. Search engine AI assistant
- 13. Search engine AI assistant
- 15. tl;dr LLMs doing constrained Q/A 🤩
- 16. tl;dr LLMs doing long-range, open-ended tasks 🤨
- 18. 90%+ of questions are related to one project No shared context! Why? We don’t know how to do it…
- 19. How to get an LLM to use ● hundreds of chats ● + code base (1M+ LoC) ● + …
- 20. ● …RAG? ● Lots of retrieval? ● Multi-step retrieval? ● Retrieval that works? ● What about writing?
- 21. …long-context LLMs? Cost + latency Context pollution
- 22. No shared context! Why? We don’t know how to do it…
- 23. Search engine AI assistant state management
- 24. MemGPT -> giving LLMs real “memory”
- 25. MemGPT -> memory via tools LLM tools �� Memory
- 26. Text User message �� GPT-4 Context window 8k max token limit ChatGPT Text Agent reply �� Standard LLM setup e.g., ChatGPT UI + GPT-4 model
- 27. Event User message �� Document upload �� System alert 🔔 Function Send message �� Query database Pause interrupts �� LLM Virtual context Main context External context ∞ tokens Max token limit MemGPT parse parse MemGPT LLM OS setup Event loop + functions + memory hierarchy
- 28. Event User message �� Document upload �� System alert 🔔 Function Send message �� Query database Pause interrupts �� LLM Virtual context Main context External context ∞ tokens 8k max token limit MemGPT parse parse Fixed-context LLM e.g., GPT-4 with 8k max tokens
- 29. Event User message �� Document upload �� System alert 🔔 Function Send message �� Query database Pause interrupts �� LLM Virtual context Main context External context ∞ tokens 8k max token limit MemGPT parse parse LLM inputs are “events” (JSON) System alerts help the LLM manage memory Virtual context Main context External context ∞ tokens Max token limit LLM
- 30. Event User message �� Document upload �� System alert 🔔 LLM inputs are “events” (JSON) System alerts help the LLM manage memory { “type”: “user_message”, “content”: “how to undo git commit -am?” } { “type”: “document_upload”, “info”: “9 page PDF”, “summary”: “MemGPT research paper” } { “type”: “system_alert”, “content”: “Memory warning: 75% of context used.” }
- 31. Event User message �� Document upload �� System alert 🔔 Function Send message �� Query database Pause interrupts �� LLM Virtual context Main context External context ∞ tokens Max token limit MemGPT parse parse LLM outputs are functions (JSON) Event loop + functions that allow editing memory
- 32. Function Send message �� Query database Pause interrupts �� Agent can query out-of-context information with functions { “function”: “ archival_memory_search”, “params”: { “query”: “Berkeley LLM Meetup”, “page”: “0” } }
- 33. Function Send message �� Query database Pause interrupts �� Pages into (finite) LLM context { “function”: “ archival_memory_search”, “params”: { “query”: “Berkeley LLM Meetup”, “page”: “0” } } LLM
- 34. Function Send message �� Edit context Pause interrupts �� Agent can edit their own memory including their own context { “function”: “ core_memory_replace”, “params”: { “old_content”: “OAI Assistants API”, “new_content”: “MemGPT API” } }
- 35. Function Send message �� Edit context Pause interrupts �� Core memory is a reserved block System prompt In-context memory block Working context queue { “function”: “ core_memory_replace”, “params”: { “old_content”: “OAI Assistants API”, “new_content”: “MemGPT API” } }
- 36. Function Send message �� Query database Pause interrupts �� { “function”: “ send_message”, “params”: { “message”: “How may I assist you?” } } User messages are a function Allows interacting with system autonomously w/o user inputs
- 37. { “type”: “user_message”, “content”: “ what’s happening on may 21 2024?” } { “function”: “archival_memory_search”, “params”: { “query”: “ may 21 2024”, } } { “function”: “send_message”, “params”: { “message”: “ Have you heard about Milvus?” } } 🧑 🤖
- 38. what’s happening on may 21 2024? Have you heard about Milvus? 🧑 🤖 (User’s POV)
- 39. Event User message �� Document upload �� System alert 🔔 Function Send message �� Query database Pause interrupts �� LLM Virtual context Main context External context ∞ tokens Max token limit MemGPT parse parse MemGPT LLM OS setup Event loop + functions + memory hierarchy
- 40. Calling & executing custom tools MemGPT -> Building LLM Agents Long-term memory management �� �� Loading external data sources (RAG) 🛠
- 41. MemGPT = the OSS platform for building 🛠 and hosting 🏠 LLM agents
- 42. Developer User MemGPT Dev Portal MemGPT CLI $ memgpt run
- 43. MemGPT server User-facing application REST API Users Agents Tools Sources user_id: … agent_id: … Personal Assistant State Memories Documents
- 44. MemGPT server User-facing application REST API Users Agents Tools Sources user_id: … agent_id: … Personal Assistant State Memories Documents Webhooks
- 45. MemGPT may 21 developer update 🎉
- 47. Docker integration - the fastest way to create a MemGPT server Step 1: docker compose up Step 2: create/edit/message agents using the MemGPT API MemGPT ❤
- 48. MemGPT streaming API - token streaming CLI: memgpt run --stream REST API: use the stream_tokens flag [PR #1280 - staging]
- 49. MemGPT streaming API - token streaming MemGPT API works with both non-streaming + streaming endpoints If the true LLM backend doesn’t support streaming, “fake streaming”
- 50. MemGPT /chat/completions proxy API Connect your MemGPT server to any /chat/completions service! For example - 📞 voice call your MemGPT agents using VAPI! MemGPT ��