Skip to main content
POST
/
langevals
/
llm_category
/
evaluate
import langwatch

df = langwatch.dataset.get_dataset("dataset-id").to_pandas()

evaluation = langwatch.evaluation.init("my-incredible-evaluation")

for index, row in evaluation.loop(df.iterrows()):
# your execution code here
evaluation.run(
"langevals/llm_category",
index=index,
data={
"input": row["input"],
"output": output,
"contexts": row["contexts"],
},
settings={}
)
[
  {
    "status": "processed",
    "score": 123,
    "passed": true,
    "label": "<string>",
    "details": "<string>",
    "cost": {
      "currency": "<string>",
      "amount": 123
    },
    "raw_response": {},
    "error_type": "<string>",
    "traceback": [
      "<string>"
    ]
  }
]

Authorizations

X-Auth-Token
string
header
required

Body

application/json
data
object
required
settings
object

Evaluator settings

Response

Successful evaluation

status
enum<string>
required
Available options:
processed,
skipped,
error
score
number

Evaluation score

passed
boolean

Whether the evaluation passed

label
string

Evaluation label

details
string

Additional details about the evaluation

cost
object
raw_response
object

Raw response from the evaluator

error_type
string

Type of error if status is 'error'

traceback
string[]

Error traceback if status is 'error'

⌘I