Quickstart
Welcome to LangEvals! Lets get started and integrate LLM evaluators in a few quick steps!
Prerequisites
- Python 3.11 or higher
- pip (Python package installer)
Installation and Usage
1
Install
Run installation in the terminal.
pip install "langevals[all]"
# or select only the one you are interested on, e.g.:
pip install "langevals[azure,ragas,langevals]"
2
Import
Import the evaluator that will be used.
from langevals_langevals.competitor_llm import (
CompetitorLLMEntry,
CompetitorLLMEvaluator,
CompetitorLLMSettings,
)
3
Evaluate
Initiate and run the evaluator.
evaluator = CompetitorLLMEvaluator(settings=CompetitorLLMSettings(
name="LangWatch",
description="LLM monitoring and evaluation platform.",
))
result = evaluator.evaluate(entry=CompetitorLLMEntry(
input="Hey, where can I monitor and evaluate LLMs except LangWatch?"
))
4
Explore the Results
Print the results of the evaluation:
print("Result passed LLM competitor check: ", result.passed)
print(result.details)
print("Cost of evaluation: ", result.cost)
Explore the results of evaluation:
Result passed LLM competitor check: False
0.9 - confidence score. Reasoning: The question explicitly mentions LangWatch as a monitoring and evaluation platform. The user is asking about alternatives to LangWatch, which implies a comparison with competitors.
Cost of evaluation: currency='USD' amount=0.0004315