From the course: Advanced LLMOps: Deploying and Managing LLMs in Production
Join today to access over 24,700 courses taught by industry experts.
Solution: Evaluating LLM systems
From the course: Advanced LLMOps: Deploying and Managing LLMs in Production
Solution: Evaluating LLM systems
(smooth uptempo music) - [Instructor] Let's see how to use Ragas to evaluate a RAG application. We start by first installing the Ragas library. Then we import the necessary functions and classes that we will use. First, we'll import the datasets package. Next, from Ragas, we will import the four metrics, as well as the evaluate class. Next, we set up an OpenAI API key using the getpass model. Now we are ready to load our dataset. We will use the fiqa_eval dataset from the exploding gradients. This dataset contains questions, context, and answers from a RAG application. It also contains the ground truth we can compare with. Since this is a large data set, we will select a subset for evaluation. In this case, I am selecting the first three entries. Now we can pass the data to evaluate function along with four metrics. You can then print the results and see values of the four metrics. As a bonus, try to generate an answer using a different model, and see how the evaluation results change.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.