From the course: Advanced LLMOps: Deploying and Managing LLMs in Production
Join today to access over 24,700 courses taught by industry experts.
Introduction to LLM performance monitoring
From the course: Advanced LLMOps: Deploying and Managing LLMs in Production
Introduction to LLM performance monitoring
- [Instructor] In the last chapter, we learned about generative AI deployment architectures and the complexities of integrating LLMs into production applications. However, deploying these models is only the first step. Ensuring that they operate effectively and safely in production is an ongoing challenge. This is where LLM monitoring comes into play. Monitoring these systems is not just about tracking their performance, it's about ensuring that they behave as expected, and mitigate any potential risk they might introduce. Even though LLMs are very capable models, deploying them to production comes with inherent risks and imperfections. They are susceptible to several issues like hallucination, prompt injection, and other security challenges that can affect their functionality and user experience. In this lesson, we will learn about monitoring LLM apps in production and how to address challenges like hallucination. Let's talk about the key components that we should monitor…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.