LangWatch is open-source and can be run locally or self-hosted using docker.

Quickstart

You can have LangWatch fully running locally with 3 easy steps.

1

Clone the repository

First, clone LangWatch repository:

git clone https://github.com/langwatch/langwatch.git
2

Create the .env file

Copy langwatch/.env.example to langwatch/.env, you can customize it later on.

3

Run the application

docker compose up --build

Open LangWatch at http://localhost:3000

Pointing to your instance

Now that your have your own instance of LangWatch running, in order to send the messages to it, you need to set your LangWatch endpoint on your LLM application:

import langwatch

langwatch.endpoint = "http://localhost:3000"

Check out the integration guides for more information on integration.

Architecture

To understand how all the different pieces of the LangWatch platform connect together, here is an outline of the architecture:

LangWatch Architecture

In short, the main LangWatch application is the one talking to the databases and the two other python services: LangEvals and LangWatch NLP. Those services can make calls to LLMs and other APIs.

When you start up docker compose for running LangWatch, all those six components are started. When hosting in your own infrastructure, be sure to have each of them running and properly connected before setting the correct env vars and starting the main application.

Here is a brief explaination of each one:

LangWatch

LangWatch is the main application, it’s a NextJS application that servers both the frontend and backend of the platform.

When building for production, you can use the Dockerfile.prod file at the root of the repository to build the docker image.

Postgres

Postgres is the primary database, used for example for storing user, projects, saved datasets and settings. Since this is not where most of the heavy work is, any Postgres or Postgres-compatible database like AWS RDS should work.

ElasticSearch

ElasticSearch powers LangWatch capability to scale to hundreds of millions of messages for observability. It stores the messages, metrics, spans, events, evaluation results, doing search, filtering and analytics.

We recommend using Elastic Cloud for a managed ElasticSearch service.

Redis

Redis is used for LangWatch queues, for processing and retrying messages and evaluations in the background. Any Redis or Redis-compatible service like AWS ElastiCache for Redis should work.

LangEvals

LangEvals is the platform that powers LangWatch evaluators and guardrails, a Python service that connects with other APIs and LLMs for running evaluations.

A docker image is readily available from dockerhub for running LangEvals service.

LangWatch NLP

LangWatch NLP is the python backend service that proxies the playground LLM calls, processes sentiment analysis and runs the topic clustering algorithm.

The dockerfile to build this service is inside the langwatch_nlp/ folder on the main LangWatch repository.

Environment Variables

The default env vars already gets you up and running with the platform, but to make sure all features of LangWatch work properly, set up the environment variables as shown on the .env.example file.

Basic Configuration

Env VarDescription
NODE_ENVDefine the environment you are running in, you can change this to production for a production deployment
BASE_HOSTDefine the base host for the application, this is the hostname and port of the application
DEBUGThis is a comma separated list of the namespaces of the messages you want to print following npm debug library standard

Authentication

Env VarDescription
NEXTAUTH_PROVIDERLangWatch uses next-auth for authentication, either email or auth0
NEXTAUTH_SECRETThis is the key used to encrypt cookie keys, be sure to change it before deploying to production
AUTH0_CLIENT_IDClient ID in case you choose to configure auth0 as the authentication provider
AUTH0_CLIENT_SECRETClient secret in case you choose to configure auth0 as the authentication provider
AUTH0_ISSUERIssuer in case you choose to configure auth0 as the authentication provider

Services and Infrastructure

Env VarDescription
DATABASE_URLPostgres connection string for primary data storage
ELASTICSEARCH_NODE_URLElasticSearch connection string for observability data
REDIS_URLRedis connection string for queue processing
LANGWATCH_NLP_SERVICELangWatch NLP service endpoint
LANGEVALS_ENDPOINTLangEvals service endpoint
GOOGLE_APPLICATION_CREDENTIALSGoogle credentials with access to DLP services, used for automatically redacting PII in traces and spans
SENDGRID_API_KEYSendgrid API key for sending emails

Model Providers

Env VarDescription
OPENAI_API_KEYOpenAI API key for embeddings and GPT-class LLMs
AZURE_OPENAI_ENDPOINTAzure OpenAI endpoint for embeddings and GPT-class LLMs
AZURE_OPENAI_API_KEYAzure OpenAI API key for embeddings and GPT-class LLMs
GOOGLE_APPLICATION_CREDENTIALSGoogle Cloud credentials for Gemini-class LLMs
VERTEXAI_PROJECTGoogle VertexAI project for Gemini-class LLMs
VERTEXAI_LOCATIONGoogle VertexAI location for Gemini-class LLMs
ANTHROPIC_API_KEYAnthropic API key for Claude-class LLMs
GROQ_API_KEYGroq API key for Llama 3, Mixtral and other open-source LLMs with top inference speed

Support

If you need more support on the next steps for the production self-hosting installation on your cloud provider, reach out to us.

Email Support

If you are prefer direct assistance from our team with the self-hosting installation, you can reach out to our support team via email at [email protected].

Discord Channel

You can also join our Discord channel and ask questions directly for the community and the core team.