TypeScript Integration Guide
LangWatch library is the easiest way to integrate your TypeScript application with LangWatch, the messages are synced on the background so it doesn’t intercept or block your LLM calls.
Prerequisites
- Obtain your
LANGWATCH_API_KEY
from the LangWatch dashboard.
Installation
npm install langwatch
Configuration
Ensure LANGWATCH_API_KEY
is set:
LANGWATCH_API_KEY='your_api_key_here'
Initialize LangWatch client:
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
Capturing Messages
- Each message triggering your LLM pipeline as a whole is captured with a Trace.
- A Trace contains multiple Spans, which are the steps inside your pipeline.
- Traces can be grouped together on LangWatch Dashboard by having the same
thread_id
in their metadata, making the individual messages become part of a conversation.- It is also recommended to provide the
user_id
metadata to track user analytics.
- It is also recommended to provide the
Create a Trace
// Trace creation
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
// Example of updating the metadata to add labels to the trace if needed
trace.update({
metadata: { labels: ["customer-support"] },
});
Capture an LLM Span
To capture your LLM calls, you can start an LLM span inside the trace with the input about to be sent to the LLM.
First, define the model and the messages you are going to use for your LLM call separately, so you can capture them:
import { OpenAI } from "openai";
// Model to be used and messages that will be sent to the LLM
const model = "gpt-4o"
const messages : OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: "You are a helpful assistant." },
{
role: "user",
content: "Write a tweet-size vegetarian lasagna recipe for 4 people.",
},
]
Then, start the LLM span from the trace, giving it the model and input messages:
const span = trace.startLLMSpan({
name: "llm",
model: model,
input: {
type: "chat_messages",
value: messages
},
});
This will capture the LLM input and register the time the call started. Now, continue with the LLM call normally, using the same parameters:
const openai = new OpenAI();
const chatCompletion = await openai.chat.completions.create({
messages: messages,
model: model,
});
Finally, after the OpenAI call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation:
span.end({
output: {
type: "chat_messages",
value: [chatCompletion.choices[0]!.message],
},
metrics: {
promptTokens: chatCompletion.usage?.prompt_tokens,
completionTokens: chatCompletion.usage?.completion_tokens,
},
});
On short-live environments like Lambdas or Serverless Functions, be sure to call
await trace.sendSpans();
to wait for all pending requests to be sent before the runtime is destroyed.
Capture a RAG Span
Appart from LLM spans, another very used type of span is the RAG span. This is used to capture the retrieved contexts from a RAG that will be used by the LLM, and enables a whole new set of RAG-based features evaluations for RAG quality on LangWatch.
To capture a RAG, you can simply start a RAG span inside the trace, giving it the input query being used:
const ragSpan = trace.startRAGSpan({
name: "my-vectordb-retrieval", // optional
input: { type: "text", value: "search query" },
});
// proceed to do the retrieval normally
Then, after doing the retrieval, you can end the RAG span with the contexts that were retrieved and will be used by the LLM:
ragSpan.end({
contexts: [
{
documentId: "doc1",
content: "document chunk 1",
},
{
documentId: "doc2",
content: "document chunk 2",
},
],
});
On LangChain.js, RAG spans are captured automatically by the LangWatch callback when using LangChain Retrievers, with source
as the documentId.
Capture an arbritary Span
You can also use generic spans to capture any type of operation, its inputs and outputs, for example for a function call:
// before the function starts
const span = trace.startSpan({
name: "weather_function",
input: {
type: "json",
value: {
city: "Tokyo",
},
},
});
// ...after the function ends
span.end({
output: {
type: "json",
value: {
weather: "sunny",
},
},
});
You can also nest spans one inside the other, capturing your pipeline structure, for example:
const span = trace.startSpan({
name: "pipeline",
});
const nestedSpan = span.startSpan({
name: "nested_pipeline",
});
nestedSpan.end()
span.end()
Both LLM and RAG spans can also be nested like any arbritary span.
Capturing Exceptions
To capture also when your code throws an exception, you can simply wrap your code around a try/catch, and update or end the span with the exception:
try {
throw new Error("unexpected error");
} catch (error) {
span.end({
error: error,
});
}
Capturing custom evaluation results
LangWatch Evaluators can run automatically on your traces, but if you have an in-house custom evaluator, you can also capture the evaluation
results of your custom evaluator on the current trace or span by using the .addEvaluation
method:
import { type LangWatchTrace } from "langwatch";
async function llmStep({ message, trace }: { message: string, trace: LangWatchTrace }): Promise<string> {
const span = trace.startLLMSpan({ name: "llmStep" });
// ... your existing code
span.addEvaluation({
name: "custom evaluation",
passed: true,
score: 0.5,
label: "category_detected",
details: "explanation of the evaluation results",
});
}
The evaluation name
is required and must be a string. The other fields are optional, but at least one of passed
, score
or label
must be provided.