LangWatch TypeScript Repo
LangWatch TypeScript SDK version
LangWatch library is the easiest way to integrate your TypeScript application with LangWatch, the messages are synced on the background so it doesn’t intercept or block your LLM calls.
Protip: wanna to get started even faster? Copy our llms.txt and ask an AI to do this integration

​
Prerequisites

​
Installation

npm install langwatch

​
Configuration

Ensure LANGWATCH_API_KEY is set:
.env
LANGWATCH_API_KEY='your_api_key_here'

​
Basic Concepts

  • Each message triggering your LLM pipeline as a whole is captured with a Trace.
  • A Trace contains multiple Spans, which are the steps inside your pipeline.
    • A span can be an LLM call, a database query for a RAG retrieval, or a simple function transformation.
    • Different types of Spans capture different parameters.
    • Spans can be nested to capture the pipeline structure.
  • Traces can be grouped together on LangWatch Dashboard by having the same thread_id in their metadata, making the individual messages become part of a conversation.
    • It is also recommended to provide the user_id metadata to track user analytics.

​
Integration

The Vercel AI SDK supports tracing via Next.js OpenTelemetry integration. By using the LangWatchExporter, you can automatically collect those traces to LangWatch.First, you need to install the necessary dependencies:
npm install @vercel/otel langwatch @opentelemetry/api-logs @opentelemetry/instrumentation @opentelemetry/sdk-logs
Then, set up the OpenTelemetry for your application, follow one of the tabs below depending whether you are using AI SDK with Next.js or on Node.js:
You need to enable the instrumentationHook in your next.config.js file if you haven’t already:
/** @type {import('next').NextConfig} */
const nextConfig = {
  experimental: {
    instrumentationHook: true,
  },
};

module.exports = nextConfig;
Next, you need to create a file named instrumentation.ts (or .js) in the root directory of the project (or inside src folder if using one), with LangWatchExporter as the traceExporter:
import { registerOTel } from '@vercel/otel'
import { LangWatchExporter } from 'langwatch'

export function register() {
  registerOTel({
    serviceName: 'next-app',
    traceExporter: new LangWatchExporter({
      apiKey: process.env.LANGWATCH_API_KEY
    })
  })
}
(Read more about Next.js OpenTelemetry configuration on the official guide)Finally, enable experimental_telemetry tracking on the AI SDK calls you want to trace:
const result = await generateText({
  model: openai('gpt-4o-mini'),
  prompt: 'Explain why a chicken would make a terrible astronaut, be creative and humorous about it.',
  experimental_telemetry: {
    isEnabled: true,
    // optional metadata
    metadata: {
      userId: "myuser-123",
      threadId: "mythread-123",
    },
  },
});
That’s it! Your messages will now be visible on LangWatch:Vercel AI SDK

​
Example Project

You can find a full example project with a more complex pipeline and Vercel AI SDK and LangWatch integration on our GitHub.

​
Manual Integration

The docs from here below are for manual integration, in case you are not using the Vercel AI SDK OpenTelemetry integration, you can manually start a trace to capture your messages:
import { LangWatch } from 'langwatch';

const langwatch = new LangWatch();

const trace = langwatch.getTrace({
  metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
Then, you can start an LLM span inside the trace with the input about to be sent to the LLM.
import { convertFromVercelAIMessages } from 'langwatch'

const span = trace.startLLMSpan({
  name: "llm",
  model: model,
  input: {
    type: "chat_messages",
    value: convertFromVercelAIMessages(messages)
  },
});
This will capture the LLM input and register the time the call started. Once the LLM call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation, e.g.:
span.end({
  output: {
    type: "chat_messages",
    value: convertFromVercelAIMessages(output), // assuming output is Message[]
  },
  metrics: {
    promptTokens: chatCompletion.usage?.prompt_tokens,
    completionTokens: chatCompletion.usage?.completion_tokens,
  },
});
On short-live environments like Lambdas or Serverless Functions, be sure to call
await trace.sendSpans(); to wait for all pending requests to be sent before the runtime is destroyed.

​
Capture a RAG Span

Appart from LLM spans, another very used type of span is the RAG span. This is used to capture the retrieved contexts from a RAG that will be used by the LLM, and enables a whole new set of RAG-based features evaluations for RAG quality on LangWatch. To capture a RAG, you can simply start a RAG span inside the trace, giving it the input query being used: 
const ragSpan = trace.startRAGSpan({
  name: "my-vectordb-retrieval", // optional
  input: { type: "text", value: "search query" },
});

// proceed to do the retrieval normally
Then, after doing the retrieval, you can end the RAG span with the contexts that were retrieved and will be used by the LLM: 
ragSpan.end({
  contexts: [
    {
      documentId: "doc1",
      content: "document chunk 1",
    },
    {
      documentId: "doc2",
      content: "document chunk 2",
    },
  ],
});
On LangChain.js, RAG spans are captured automatically by the LangWatch callback when using LangChain Retrievers, with source as the documentId.

​
Capture an arbritary Span

You can also use generic spans to capture any type of operation, its inputs and outputs, for example for a function call: 
// before the function starts
const span = trace.startSpan({
  name: "weather_function",
  input: {
    type: "json",
    value: {
      city: "Tokyo",
    },
  },
});

// ...after the function ends
span.end({
  output: {
    type: "json",
    value: {
      weather: "sunny",
    },
  },
});
You can also nest spans one inside the other, capturing your pipeline structure, for example: 
const span = trace.startSpan({
  name: "pipeline",
});

const nestedSpan = span.startSpan({
  name: "nested_pipeline",
});

nestedSpan.end()

span.end()
Both LLM and RAG spans can also be nested like any arbritary span.

​
Capturing Exceptions

To capture also when your code throws an exception, you can simply wrap your code around a try/catch, and update or end the span with the exception: 
try {
  throw new Error("unexpected error");
} catch (error) {
  span.end({
    error: error,
  });
}

​
Capturing custom evaluation results

LangWatch Evaluators can run automatically on your traces, but if you have an in-house custom evaluator, you can also capture the evaluation results of your custom evaluator on the current trace or span by using the .addEvaluation method: 
import { type LangWatchTrace } from "langwatch";

async function llmStep({ message, trace }: { message: string, trace: LangWatchTrace }): Promise<string> {
    const span = trace.startLLMSpan({ name: "llmStep" });

    // ... your existing code

    span.addEvaluation({
        name: "custom evaluation",
        passed: true,
        score: 0.5,
        label: "category_detected",
        details: "explanation of the evaluation results",
    });
}
The evaluation name is required and must be a string. The other fields are optional, but at least one of passed, score or label must be provided.