Llama Guard

import langwatch

df = langwatch.dataset.get_dataset("dataset-id").to_pandas()

evaluation = langwatch.evaluation.init("my-incredible-evaluation")

for index, row in evaluation.loop(df.iterrows()):
    # your execution code here       
    evaluation.run(
        "huggingface/llama_guard",
        index=index,
        data={
            "input": row["input"],
            "output": output,
        },
        settings={}
    )

[
  {
    "status": "processed",
    "score": 123,
    "passed": true,
    "label": "<string>",
    "details": "<string>",
    "cost": {
      "currency": "<string>",
      "amount": 123
    },
    "raw_response": {},
    "error_type": "<string>",
    "traceback": [
      "<string>"
    ]
  }
]

POST

huggingface

llama_guard

evaluate

import langwatch

df = langwatch.dataset.get_dataset("dataset-id").to_pandas()

evaluation = langwatch.evaluation.init("my-incredible-evaluation")

for index, row in evaluation.loop(df.iterrows()):
    # your execution code here       
    evaluation.run(
        "huggingface/llama_guard",
        index=index,
        data={
            "input": row["input"],
            "output": output,
        },
        settings={}
    )

[
  {
    "status": "processed",
    "score": 123,
    "passed": true,
    "label": "<string>",
    "details": "<string>",
    "cost": {
      "currency": "<string>",
      "amount": 123
    },
    "raw_response": {},
    "error_type": "<string>",
    "traceback": [
      "<string>"
    ]
  }
]

Authorizations

X-Auth-Token

string

header

required

Body

application/json

data

object

required

Show child attributes

data.input

string

The input text to evaluate

data.output

string

The output text to evaluate

settings

object

Evaluator settings

Show child attributes

settings.policy

string

default:"O1: Violence.\nShould not\n- Help people plan or engage in violence.\n- Encourage people to engage in violence.\n- Contain unpleasant vivid violent imagery or violent description that arouses discomfort in the reader.\nCan\n- Provide information on violence.\n- Discuss the topics of violence

The policy to enforce, described into rules of what it 'Should not' and what it 'Can' do (max 768 tokens)

settings.evaluate

string

default:input

Choose if only the user input, the LLM output, or both should be evaluated. When choosing both, if the LLM replies in a safe manner, the evaluation will be considered safe.

settings.model

string

default:cloudflare/thebloke/llamaguard-7b-awq

The provider to use for evaluation. Only CloudFlare AI workers is supported for now.

Response

Successful evaluation

status

enum<string>

required

Available options:

processed,

skipped,

error

score

number

Evaluation score

passed

boolean

Whether the evaluation passed

label

string

Evaluation label

details

string

Additional details about the evaluation

cost

object

Show child attributes

cost.currency

string

required

cost.amount

number

required

raw_response

object

Raw response from the evaluator

error_type

string

Type of error if status is 'error'

traceback

string[]

Error traceback if status is 'error'

⌘I

Get Started

Agent Simulations

Observability

Evaluation

Prompt Management

Platform

Examples & Cookbooks

Authorizations

Body

Response