LangWatch home page

langwatch/langwatch
Support
Dashboard
langwatch/langwatch

API Endpoints

OpenAI Moderation

Documentation

Introduction
Evaluators
Unit Tests
API Example
How to Choose Your Evaluator Guide
Modular Architecture

Tutorials

Extensive Unit Testing
RAG Evaluation
Evaluate on CI/CD Pipeline

API Endpoints

POST
Language Detection
POST
OpenAI Moderation
POST
PII Detection
POST
Content Safety
POST
Jailbreak Detection
POST
Prompt Injection Detection
POST
Competitor Detection with LLM
POST
Prompt Injection Detection
POST
Off-Topic Detection
POST
Answer Relevancy
POST
Context Precision
POST
Context Recall
POST
Context Relevancy
POST
Context Utilization
POST
Faithfulness
POST
Haystack Faithfulness
POST
LLM Boolean Evaluator
POST
LLM Score Evaluator
POST
LLM Basic Evaluator
POST
LLM Similarity Evaluator
POST
Llama Guard

API Endpoints

OpenAI Moderation

This evaluator uses OpenAI’s moderation API to detect potentially harmful content in text, including harassment, hate speech, self-harm, sexual content, and violence.

Env vars: OPENAI_API_KEY

Docs: https://platform.openai.com/docs/guides/moderation/overview

POST

openai

moderation

evaluate

object

data

array

data

Required

array

List of entries to be evaluated, check the field type for the necessary keys

settings

object

settings

object

Evaluator settings, check the field type for what settings this evaluator supports

model

enum<string>

model

enum<string>

The model version to use, text-moderation-latest will be automatically upgraded over time, while text-moderation-stable will only be updated with advanced notice by OpenAI.

Body

application/json

data

object[]

required

List of entries to be evaluated, check the field type for the necessary keys

data.input

string | null

data.output

string | null

settings

object | null

Evaluator settings, check the field type for what settings this evaluator supports

settings.model

enum<string>

default: text-moderation-stable

The model version to use, text-moderation-latest will be automatically upgraded over time, while text-moderation-stable will only be updated with advanced notice by OpenAI.

Available options:

text-moderation-stable,

text-moderation-latest

settings.categories

object

The categories of content to check for moderation.

settings.categories.harassment

boolean

default: true

settings.categories.harassment_threatening

boolean

default: true

settings.categories.hate

boolean

default: true

settings.categories.hate_threatening

boolean

default: true

settings.categories.self_harm

boolean

default: true

settings.categories.self_harm_instructions

boolean

default: true

settings.categories.self_harm_intent

boolean

default: true

settings.categories.sexual

boolean

default: true

settings.categories.sexual_minors

boolean

default: true

settings.categories.violence

boolean

default: true

settings.categories.violence_graphic

boolean

default: true

env

object | null

Optional environment variables to override the server ones

env.{key}

string

Response

200 - application/json

status

enum<string>

default: processed

Available options:

processed

score

number

required

The model's confidence on primary category where the input violates the OpenAI's policy. The value is between 0 and 1, where higher values denote higher confidence.

passed

boolean | null

details

string | null

Short human-readable description of the result

cost

object | null

cost.currency

string

required

cost.amount

number

required

Language Detection PII Detection

github linkedin