API Endpoints
OpenAI Moderation
This evaluator uses OpenAI’s moderation API to detect potentially harmful content in text, including harassment, hate speech, self-harm, sexual content, and violence.
Env vars: OPENAI_API_KEY
Docs: https://platform.openai.com/docs/guides/moderation/overview
POST
/
openai
/
moderation
/
evaluate
Body
application/json
data
object[]
requiredList of entries to be evaluated, check the field type for the necessary keys
settings
object | null
Evaluator settings, check the field type for what settings this evaluator supports
env
object | null
Optional environment variables to override the server ones
Response
200 - application/json
status
enum<string>
default: processedAvailable options:
processed
score
number
requiredThe model's confidence on primary category where the input violates the OpenAI's policy. The value is between 0 and 1, where higher values denote higher confidence.
passed
boolean | null
details
string | null
Short human-readable description of the result
cost
object | null