Welcome to LangEvals! Lets get started and integrate LLM evaluators in a few quick steps!

​
Prerequisites

  1. Python 3.11 or higher
  2. pip (Python package installer)

​
Installation and Usage

1

Install

Run installation in the terminal.

pip install "langevals[all]"
# or select only the one you are interested on, e.g.:
pip install "langevals[azure,ragas,langevals]"
2

Import

Import the evaluator that will be used.

from langevals_langevals.competitor_llm import (
    CompetitorLLMEntry,
    CompetitorLLMEvaluator,
    CompetitorLLMSettings,
)
3

Evaluate

Initiate and run the evaluator.

evaluator = CompetitorLLMEvaluator(settings=CompetitorLLMSettings(
    name="LangWatch",
    description="LLM monitoring and evaluation platform.",
))   
result = evaluator.evaluate(entry=CompetitorLLMEntry(
    input="Hey, where can I monitor and evaluate LLMs except LangWatch?"
))
4

Explore the Results

Print the results of the evaluation:

print("Result passed LLM competitor check: ", result.passed)
print(result.details)
print("Cost of evaluation: ", result.cost)

Explore the results of evaluation:

Result passed LLM competitor check:  False
0.9 - confidence score. Reasoning: The question explicitly mentions LangWatch as a monitoring and evaluation platform. The user is asking about alternatives to LangWatch, which implies a comparison with competitors.
Cost of evaluation:  currency='USD' amount=0.0004315