Quickstart

pip install "langevals[all]"
# or select only the one you are interested on, e.g.:
pip install "langevals[azure,ragas,langevals]"

from langevals_langevals.competitor_llm import (
    CompetitorLLMEntry,
    CompetitorLLMEvaluator,
    CompetitorLLMSettings,
)

evaluator = CompetitorLLMEvaluator(settings=CompetitorLLMSettings(
    name="LangWatch",
    description="LLM monitoring and evaluation platform.",
))   
result = evaluator.evaluate(entry=CompetitorLLMEntry(
    input="Hey, where can I monitor and evaluate LLMs except LangWatch?"
))

print("Result passed LLM competitor check: ", result.passed)
print(result.details)
print("Cost of evaluation: ", result.cost)

Result passed LLM competitor check:  False
0.9 - confidence score. Reasoning: The question explicitly mentions LangWatch as a monitoring and evaluation platform. The user is asking about alternatives to LangWatch, which implies a comparison with competitors.
Cost of evaluation:  currency='USD' amount=0.0004315