In this tutorial we will show how you can evaluate your RAG application with the help of LangEvals. Check RAGs Context Tracking to learn how to integrate your application with LangWatch and trace your RAG calls.

​
LangEvals RAG Evaluators

You can easily import RAG evaluators from the langevals_ragas module and quickly make use of their features.

from langevals_ragas.context_relevancy import RagasContextRelevancyEntry, RagasContextRelevancyEvaluator
from langevals_ragas.faithfulness import RagasFaithfulnessEntry, RagasFaithfulnessEvaluator
from langevals_ragas.answer_relevancy import RagasAnswerRelevancyEntry, RagasAnswerRelevancyEvaluator

entry1 = RagasAnswerRelevancyEntry(input="What is the capital of France?", output="Paris is the capital of France")

entry2 = RagasContextRelevancyEntry(output="Paris is the capital of France", contexts=[
    "Water can evaporate or turn into ice",
    "Dogs and Cats can be friends",
    "The sun is shining today"
])

entry3 = RagasFaithfulnessEntry(output="Paris is the capital of France", contexts=[
    "France is a country in Europe",
    "Lyon, Paris and Bordeaux are cities in France",
    "Paris is the capital of France"
])

result1 = RagasAnswerRelevancyEvaluator().evaluate(entry=entry1)
result2 = RagasContextRelevancyEvaluator().evaluate(entry=entry2)
result3 = RagasFaithfulnessEvaluator().evaluate(entry=entry3)

In the example above we import 3 evaluators and create 3 corresponding entries that are evaluated at the end. Each entry requires different set of input parameters, depending on what we want to evaluate. Pay attention to the entry2, how do you think, what will be the result of its evaluation on context relevancy?

Open in Notebook

You can access and run the code yourself in Jupyter Notebook

Extensive Unit TestingEvaluate on CI/CD Pipeline