# LangWatch
# FILE: ./introduction.mdx
---
title: Introduction
description: Welcome to LangWatch, the all-in-one [open-source](https://github.com/langwatch/langwatch) LLMops platform.
---
LangWatch allows you to track, monitor, guardrail and evaluate your LLMs apps for measuring quality and alert on issues.
For domain experts, it allows you to easily sift through conversations, see topics being discussed and annotate and score messages
for improvement in a collaborative manner with the development team.
For developers, it allows you to debug, build datasets, prompt engineer on the playground and
run batch evaluations or [DSPy experiments](./dspy-visualization/quickstart) to continuously improve the product.
Finally, for the business, it allows you to track conversation metrics and give full user and quality analytics, cost tracking, build
custom dashboards and even integrate it back on your own platform for reporting to your customers.
You can [sign up](https://app.langwatch.ai/) and already start the integration on our free tier by following the guides bellow:
You can also [open the demo project](https://app.langwatch.ai/demo) check out a [video](https://www.loom.com/share/17f827b1f5a648298779b36e2dc959e6) on our platform.
## Get in touch
Feel free to reach out to us directly at [support@langwatch.ai](mailto:support@langwatch.ai). You can also open a [GitHub issue](https://github.com/langwatch/langwatch/issues)
to report bugs and request features, or join our [Discord](https://discord.gg/kT4PhDS2gH) channel and ask questions directly for the community and the core team.
---
# FILE: ./concepts.mdx
---
title: Concepts
description: LLM tracing and observability conceptual guide
keywords: LangWatch, concepts, tracing, observability, LLM, AI, travel, blog, user, customer, labels, threads, traces, spans
---
Understanding the core concepts of LangWatch is essential for effective observability in your LLM applications. This guide explains key terms and their relationships using practical examples, like building an AI travel assistant or a text generation service.
### Threads: The Whole Conversation
Field: `thread_id`
Think of a **Thread** as the entire journey a user takes in a single session. It's the complete chat with your AI travel buddy, from "Where should I go?" to booking the flight. For the blog post generator, a `thread_id` bundles up the whole session – from brainstorming headlines to polishing the final SEO-optimized draft. It groups *all* the back-and-forth interactions (Traces) for a specific task or conversation.
### Traces: One Task, End-to-End
Field: `trace_id`
While previously LangWatch allowed you to pass in a custom `trace_id`, we now generate it for you automatically, and provide no way to pass in your own.
Zooming in from Threads, a **Trace** represents a single, complete task performed by your AI. It's one round trip.
* **Travel Bot:** A user asking, "What are the cheapest flights to Bali in July?" is one Trace. Asking, "Does the hotel allow llamas?" is another Trace.
* **Blog Tool:** Generating headline options? That's a Trace. Drafting the intro paragraph? Another Trace. Optimizing for keywords? You guessed it – a Trace.
Each `trace_id` captures an entire end-to-end generation, no matter how many internal steps (Spans) it takes.
### Spans: The Building Blocks
Field: `span_id`
While previously LangWatch allowed you to pass in a custom `span_id`, we now generate it for you automatically, and provide no way to pass in your own.
Now, let's get granular! **Spans** are the individual steps or operations *within* a single Trace. Think of them as the building blocks of getting the job done.
* **Travel Bot Trace:** Might have a Span for the LLM call figuring out destinations, another Span querying an airline API for prices, and a final Span formatting the response.
* **Blog Tool Trace:** Could involve a Span for the initial text generation, a second Span where the LLM critiques its own work (clever!), and a third Span refining the text based on that critique.
Each `span_id` pinpoints a specific action taken by your system or an LLM call.
### User ID: Who's Using the App?
Field: `user_id`
Simple but crucial: The **User ID** identifies the actual end-user interacting with your product. Whether they're planning trips or writing posts, this `user_id` (usually their account ID in your system) links the activity back to a real person, helping you see how different users experience your AI features.
### Customer ID: For Platform Builders
Field: `customer_id`
Are you building a platform *for other companies* to create *their own* LLM apps? That's where the **Customer ID** shines. If you're providing the tools for others (your customers) to build AI assistants for *their* users, the `customer_id` lets you (and them!) track usage and performance per customer account. It's perfect for offering custom analytics dashboards, showing your customers how *their* AI implementations are doing.
### Labels: Your Organizational Superpowers
Field: `labels`
Think of **Labels** as flexible tags you can slap onto Traces to organize, filter, and compare anything you want! They're your secret weapon for slicing and dicing your data.
* **Categorize Actions:** Use labels like `blogpost_title` or `blogpost_keywords`.
* **Track Versions:** Label traces with `version:v1.0.0`, then deploy an improved prompt and label new traces `version:v1.0.1`.
* **Run Experiments:** Tag traces with `experiment:prompt_a` vs. `experiment:prompt_b`.
Labels make it easy to zoom in on specific features or A/B test different approaches right within the LangWatch dashboard.
---
# FILE: ./integration/overview.mdx
---
title: Overview
description: Easily integrate LangWatch with your Python, TypeScript, or REST API projects.
---
Integrating LangWatch into your projects is designed to be a straightforward process. Regardless of the language or LLM model you are using, you can set up LangWatch with minimal configuration and start gathering valuable insights into your LLM's performance and user interactions.
---
# FILE: ./integration/python/reference.mdx
---
title: Python SDK API Reference
sidebarTitle: Python
description: LangWatch Python SDK API reference
---
## Setup
### `langwatch.setup()`
Initializes the LangWatch client, enabling data collection and tracing for your LLM application. This is typically the first function you'll call when integrating LangWatch.
```python Basic Setup
import langwatch
import os
client = langwatch.setup(
api_key=os.getenv("LANGWATCH_API_KEY"),
endpoint_url=os.getenv("LANGWATCH_ENDPOINT_URL") # Optional, defaults to LangWatch Cloud
)
```
```python Advanced Setup
import langwatch
import os
from opentelemetry.sdk.trace import TracerProvider
from langwatch.domain import SpanProcessingExcludeRule
from langwatch.types import BaseAttributes
# Example: Configure with a custom TracerProvider and exclude rules
my_provider = TracerProvider()
exclude_rules = [
SpanProcessingExcludeRule(
field_name="span_name",
match_value="InternalHealthCheck",
match_operation="exact_match"
)
]
custom_attributes = BaseAttributes(
tags=["my-app", "production"],
props={"service.version": "1.2.3"}
)
client = langwatch.setup(
api_key=os.getenv("LANGWATCH_API_KEY"),
tracer_provider=my_provider,
span_exclude_rules=exclude_rules,
base_attributes=custom_attributes,
debug=True
)
```
Your LangWatch API key. It's recommended to set this via an environment variable (e.g., `LANGWATCH_API_KEY`) and retrieve it using `os.getenv("LANGWATCH_API_KEY")`.
The URL of the LangWatch backend where traces will be sent. Defaults to the LangWatch Cloud service. For self-hosted instances, you'll need to provide this.
A `BaseAttributes` object allowing you to set default tags and properties that will be attached to all traces and spans. See `BaseAttributes` type for more details.
An OpenTelemetry `TracerProvider` instance. If you have an existing OpenTelemetry setup, you can pass your `TracerProvider` here. LangWatch will add its exporter to this provider. If not provided, LangWatch will configure its own.
A sequence of OpenTelemetry instrumentor instances. LangWatch can automatically apply these instrumentors. (Note: Specific instrumentor types might need to be defined or linked here).
A list of `SpanProcessingExcludeRule` objects. These rules allow you to filter out specific spans from being exported to LangWatch, based on span name or attributes. See `SpanProcessingExcludeRule` for details.
If `True`, enables debug logging for the LangWatch client, providing more verbose output about its operations.
**Returns**
An instance of the LangWatch `Client`.
## Tracing
### `@langwatch.trace()` / `langwatch.trace()`
This is the primary way to define the boundaries of a request or a significant operation in your application. It can be used as a decorator around a function or as a context manager.
When used, it creates a new trace and a root span for that trace. Any `@langwatch.span()` or other instrumented calls made within the decorated function or context manager will be nested under this root span.
```python Decorator Usage
import langwatch
langwatch.setup()
@langwatch.trace(name="MyRequestHandler", metadata={"user_id": "123"})
def handle_request(query: str):
# ... your logic ...
# langwatch.get_current_span().update(output=response)
return f"Processed: {query}"
handle_request("Hello LangWatch!")
```
```python Context Manager Usage
import langwatch
langwatch.setup()
def process_data(data: str):
with langwatch.trace(name="DataProcessingFlow", input={"data_input": data}) as current_trace:
# ... your logic ...
# langwatch.get_current_span().update(output={"status": "success"})
result = f"Processed data: {data}"
if langwatch.get_current_span(): # Check if span exists
langwatch.get_current_span().update(output=result)
return result
process_data("Sample Data")
```
The name for the root span of this trace. If used as a decorator and not provided, it defaults to the name of the decorated function. For context manager usage, a name like "LangWatch Trace" might be used if not specified.
(Deprecated) A specific ID to assign to this trace. If not provided, a new UUID will be generated. It's generally recommended to let LangWatch auto-generate trace IDs. This will be mapped to `deprecated.trace_id` in metadata.
A dictionary of metadata to attach to the entire trace. This can include things like user IDs, session IDs, or any other contextual information relevant to the whole operation. `TraceMetadata` is typically `Dict[str, Any]`.
If you have a known expected output for this trace (e.g., for testing or evaluation), you can provide it here.
Overrides the global API key for this specific trace. Useful if you need to direct traces to different projects or accounts dynamically.
If `True`, this trace (and its spans) will be processed but not sent to the LangWatch backend. This can be useful for local debugging or conditional tracing.
The maximum length for string values captured in inputs, outputs, and metadata. Longer strings will be truncated.
If `True`, a root span will not be automatically created for this trace. This is an advanced option, typically used if you intend to manage the root span's lifecycle manually or if the trace is purely a logical grouping without its own primary operation.
The following parameters are used to configure the root span that is automatically created when the trace starts. Refer to `@langwatch.span()` documentation for more details on these, as they behave similarly.
A specific ID for the root span.
Whether to capture the input of the decorated function/context block for the root span.
Whether to capture the output/result for the root span.
The type of the root span (e.g., 'llm', 'rag', 'agent', 'tool', 'span').
Explicitly sets the input for the root span. If not provided and `capture_input` is `True`, it may be inferred from function arguments.
Explicitly sets the output for the root span. If not provided and `capture_output` is `True`, it may be inferred from the function's return value.
Records an error for the root span.
Custom start and end timestamps for the root span.
RAG chunks or string contexts relevant to the root span.
The model used in the operation represented by the root span (e.g., an LLM model name).
Parameters associated with the operation of the root span (e.g., LLM call parameters).
Metrics for the root span (e.g., token counts, cost).
A list of `Evaluation` objects to associate directly with the root span.
**Context Manager Return**
When used as a context manager, `langwatch.trace()` returns a `LangWatchTrace` object.
The `LangWatchTrace` instance. You can use this object to call methods like `current_trace.add_evaluation()`.
#### `LangWatchTrace` Object Methods
When `langwatch.trace()` is used as a context manager, it yields a `LangWatchTrace` object. This object has several methods to interact with the current trace:
Updates attributes of the trace or its root span.
This method can take many of the same parameters as the `langwatch.trace()` decorator/context manager itself, such as `metadata`, `expected_output`, or any of the root span parameters like `name`, `input`, `output`, `metrics`, etc.
```python
with langwatch.trace(name="Initial Trace") as current_trace:
# ... some operations ...
current_trace.update(metadata={"step": "one_complete"})
# ... more operations ...
root_span_output = "Final output of root operation"
current_trace.update(output=root_span_output, metrics={"total_items": 10})
```
Adds an `Evaluation` object directly to the trace (or a specified span within it).
```python
from langwatch.domain import Evaluation, EvaluationTimestamps
with langwatch.trace(name="MyEvaluatedProcess") as current_trace:
# ... operation ...
eval_result = Evaluation(
name="Accuracy Check",
status="processed",
passed=True,
score=0.95,
details="All checks passed.",
timestamps=EvaluationTimestamps(started_at=..., finished_at=...)
)
current_trace.add_evaluation(**eval_result) # Pass fields as keyword arguments
```
(Refer to `Evaluation` type in Core Data Types and `langwatch.evaluations` module for more details on parameters.)
Triggers a remote evaluation for this trace using a pre-configured evaluator slug on the LangWatch platform.
```python
with langwatch.trace(name="ProcessWithRemoteEval") as current_trace:
output_to_evaluate = "Some generated text."
current_trace.evaluate(
slug="sentiment-analyzer",
output=output_to_evaluate,
as_guardrail=False
)
```
Parameters include `slug`, `name`, `input`, `output`, `expected_output`, `contexts`, `conversation`, `settings`, `as_guardrail`, `data`.
Returns: Result of the evaluation call.
An asynchronous version of `evaluate`.
Instruments an OpenAI client instance (e.g., `openai.OpenAI()`) to automatically create spans for any OpenAI API calls made using that client within the current trace.
```python
import openai
# client = openai.OpenAI() # or AzureOpenAI, etc.
with langwatch.trace(name="OpenAICallsDemo") as current_trace:
# current_trace.autotrack_openai_calls(client)
# response = client.chat.completions.create(...)
# The above call will now be automatically traced as a span.
pass
```
Takes the OpenAI client instance as an argument.
Enables automatic tracing for DSPy operations within the current trace. Requires DSPy to be installed and properly configured.
```python
with langwatch.trace(name="DSPyPipeline") as current_trace:
# current_trace.autotrack_dspy()
# ... your DSPy calls ...
pass
```
Returns a LangChain callback handler (`LangChainTracer`) associated with the current trace. This handler can be passed to LangChain runs to automatically trace LangChain operations.
```python
# from langchain_core.llms import FakeLLM (or any LangChain LLM)
# from langchain.chains import LLMChain
with langwatch.trace(name="LangChainFlow") as current_trace:
# handler = current_trace.get_langchain_callback()
# llm = FakeLLM()
# chain = LLMChain(llm=llm, prompt=...)
# chain.run("some input", callbacks=[handler])
pass
```
Returns: `LangChainTracer` instance.
(Potentially) Generates a shareable link or identifier for this trace. The exact behavior might depend on backend support and configuration.
Returns: A string, possibly a URL or an ID.
(Potentially) Revokes sharing for this trace if it was previously shared.
### `@langwatch.span()` / `langwatch.span()`
Use this to instrument specific operations or blocks of code within a trace. Spans can be nested to create a hierarchical view of your application's execution flow.
It can be used as a decorator around a function or as a context manager.
```python Decorator Usage
import langwatch
langwatch.setup()
@langwatch.trace(name="MainProcess")
def main_process():
do_first_step("data for step 1")
do_second_step("data for step 2")
@langwatch.span(name="StepOne", type="tool")
def do_first_step(data: str):
# langwatch.get_current_span().set_attributes({"custom_info": "info1"})
return f"Step 1 processed: {data}"
@langwatch.span() # Name will be 'do_second_step', type will be 'span'
def do_second_step(data: str):
# langwatch.get_current_span().update(metrics={"items_processed": 10})
return f"Step 2 processed: {data}"
main_process()
```
```python Context Manager Usage
import langwatch
langwatch.setup()
@langwatch.trace(name="ComplexOperation")
def complex_op():
# ... some initial work ...
with langwatch.span(name="DataRetrieval", type="rag") as rag_span:
# rag_span.update(contexts=[{"document_id": "doc1", "content": "..."}])
retrieved_data = "some data from RAG"
rag_span.update(output=retrieved_data)
# ... more work ...
with langwatch.span(name="LLMCall", type="llm", input={"prompt": "Summarize this"}, model="gpt-4o-mini") as llm_span:
summary = "summary from LLM"
llm_span.update(output=summary, metrics={"input_tokens": 50, "output_tokens": 15})
return summary
complex_op()
```
The name for the span. If used as a decorator and not provided, it defaults to the name of the decorated function. For context manager usage, a default name like "LangWatch Span" might be used if not specified.
The semantic type of the span, which helps categorize the operation. Common types include `'llm'`, `'rag'`, `'agent'`, `'tool'`, `'embedding'`, or a generic `'span'`. `SpanType` is typically a string literal from `langwatch.domain.SpanTypes`.
(Deprecated) A specific ID to assign to this span. It's generally recommended to let LangWatch auto-generate span IDs. This will be mapped to `deprecated.span_id` in the span's attributes.
Explicitly sets the parent for this span. If not provided, the span will be nested under the currently active `LangWatchSpan` or OpenTelemetry span.
If `True` (and used as a decorator), automatically captures the arguments of the decorated function as the span's input.
If `True` (and used as a decorator), automatically captures the return value of the decorated function as the span's output.
Explicitly sets the input for this span. `SpanInputType` can be a dictionary, a string, or a list of `ChatMessage` objects. This overrides automatic input capture.
Explicitly sets the output for this span. `SpanInputType` has the same flexibility as for `input`. This overrides automatic output capture.
Records an error for this span. If an exception occurs within a decorated function or context manager, it's often automatically recorded.
A `SpanTimestamps` object to explicitly set the `start_time` and `end_time` for the span. Useful for instrumenting operations where the duration is known or externally managed.
Relevant contextual information for this span, especially for RAG operations. `ContextsType` can be a list of `RAGChunk` objects or a list of strings.
The name or identifier of the model used in this operation (e.g., `'gpt-4o-mini'`, `'text-embedding-ada-002'`).
A dictionary or `SpanParams` object containing parameters relevant to the operation (e.g., temperature for an LLM call, k for a vector search).
A `SpanMetrics` object or dictionary to record quantitative measurements for this span, such as token counts (`input_tokens`, `output_tokens`), cost, or other custom metrics.
A list of `Evaluation` objects to directly associate with this span.
If `True`, suppresses the warning that is normally emitted if a span is created without an active parent trace.
**OpenTelemetry Parameters:**
These parameters are passed directly to the underlying OpenTelemetry span creation. Refer to OpenTelemetry documentation for more details.
The OpenTelemetry `SpanKind` (e.g., `INTERNAL`, `CLIENT`, `SERVER`, `PRODUCER`, `CONSUMER`).
An OpenTelemetry `Context` object to use for creating the span.
Additional custom attributes (key-value pairs) to attach to the span.
A list of OpenTelemetry `Link` objects to associate with this span.
An explicit start time for the span (in nanoseconds since epoch).
Whether OpenTelemetry should automatically record exceptions for this span.
Whether OpenTelemetry should automatically set the span status to error when an exception is recorded.
**Context Manager Return**
When used as a context manager, `langwatch.span()` returns a `LangWatchSpan` object.
The `LangWatchSpan` instance. You can use this object to call methods like `current_span.update()` or `current_span.add_evaluation()`.
#### `LangWatchSpan` Object Methods
When `langwatch.span()` is used as a context manager, it yields a `LangWatchSpan` object. This object provides methods to interact with the current span:
Updates attributes of the span. This is the primary method for adding or changing information on an active span.
It accepts most of the same parameters as the `@langwatch.span()` decorator itself, such as `name`, `type`, `input`, `output`, `error`, `timestamps`, `contexts`, `model`, `params`, `metrics`, and arbitrary key-value pairs for custom attributes.
```python
with langwatch.span(name="MySpan", type="tool") as current_span:
# ... some operation ...
current_span.update(output={"result": "success"}, metrics={"items_processed": 5})
# Add a custom attribute
current_span.update(custom_tool_version="1.2.3")
```
Adds an `Evaluation` object directly to this span.
```python
from langwatch.domain import Evaluation
with langwatch.span(name="SubOperation") as current_span:
# ... operation ...
eval_data = {
"name": "Correctness Check",
"status": "processed",
"passed": False,
"score": 0.2,
"label": "Incorrect"
} # Example, more fields available
current_span.add_evaluation(**eval_data)
```
(Refer to `Evaluation` type in Core Data Types and `langwatch.evaluations` module for more details on parameters.)
Triggers a remote evaluation for this span using a pre-configured evaluator slug on the LangWatch platform.
```python
with langwatch.span(name="LLM Generation", type="llm") as llm_span:
llm_output = "Some text generated by an LLM."
llm_span.update(output=llm_output)
llm_span.evaluate(slug="toxicity-check", output=llm_output, as_guardrail=True)
```
Parameters include `slug`, `name`, `input`, `output`, `expected_output`, `contexts`, `conversation`, `settings`, `as_guardrail`, `data`.
Returns: Result of the evaluation call.
An asynchronous version of `evaluate` for spans.
Explicitly ends the span. If you provide arguments (like `output`, `metrics`, etc.), it will call `update()` with those arguments before ending.
Usually not needed when using the span as a context manager, as `__exit__` handles this.
```python
# Manual span management example
# current_span = langwatch.span(name="ManualSpan").__enter__() # Start span manually
# try:
# # ... operations ...
# result = "some result"
# current_span.end(output=result) # Update and end
# except Exception as e:
# current_span.end(error=e) # Record error and end
# raise
```
**OpenTelemetry Span Methods:**
The `LangWatchSpan` object also directly exposes standard OpenTelemetry `trace.Span` API methods for more advanced use cases or direct OTel interop. These include:
- `record_error(exception)`: Records an exception against the span.
- `add_event(name, attributes)`: Adds a timed event to the span.
- `set_status(status_code, description)`: Sets the OTel status of the span (e.g., `StatusCode.ERROR`).
- `set_attributes(attributes_dict)`: Sets multiple OTel attributes at once.
- `update_name(new_name)`: Changes the name of the span.
- `is_recording()`: Returns `True` if the span is currently recording information.
- `get_span_context()`: Returns the `SpanContext` of the underlying OTel span.
Refer to the OpenTelemetry Python documentation for details on these methods.
```python
from opentelemetry.trace import Status, StatusCode
with langwatch.span(name="MyDetailedSpan") as current_span:
current_span.add_event("Checkpoint 1", {"detail": "Reached stage A"})
# ... some work ...
try:
# risky_operation()
pass
except Exception as e:
current_span.record_error(e)
current_span.set_status(Status(StatusCode.ERROR, description="Risky op failed"))
current_span.set_attributes({"otel_attribute": "value"})
```
## Context Accessors
These functions allow you to access the currently active LangWatch trace or span from anywhere in your code, provided that a trace/span has been started (e.g., via `@langwatch.trace` or `@langwatch.span`).
### `langwatch.get_current_trace()`
Retrieves the currently active `LangWatchTrace` object.
This is useful if you need to interact with the trace object directly, for example, to add trace-level metadata or evaluations from a helper function called within an active trace.
```python
import langwatch
langwatch.setup()
def some_utility_function(detail: str):
current_trace = langwatch.get_current_trace()
if current_trace:
current_trace.update(metadata={"utility_info": detail})
@langwatch.trace(name="MainOperation")
def main_operation():
# ... some work ...
some_utility_function("Processed step A")
# ... more work ...
main_operation()
```
If `True`, suppresses the warning that is normally emitted if this function is called when no LangWatch trace is currently in context.
The current `LangWatchTrace` object. If no trace is active and `suppress_warning` is `False`, a warning is issued and a new (detached) `LangWatchTrace` instance might be returned.
### `langwatch.get_current_span()`
Retrieves the currently active `LangWatchSpan` object.
This allows you to get a reference to the current span to update its attributes, add events, or record information specific to that span from nested function calls.
If no LangWatch-specific span is in context, it will attempt to wrap the current OpenTelemetry span.
```python
import langwatch
langwatch.setup()
def enrich_span_data(key: str, value: any):
current_span = langwatch.get_current_span()
if current_span:
current_span.update(**{key: value})
# Or using the older set_attributes for OpenTelemetry compatible attributes
# current_span.set_attributes({key: value})
@langwatch.trace(name="UserFlow")
def user_flow():
with langwatch.span(name="Step1") as span1:
# ... step 1 logic ...
enrich_span_data("step1_result", "success")
with langwatch.span(name="Step2") as span2:
# ... step 2 logic ...
enrich_span_data("step2_details", {"info": "more data"})
user_flow()
```
The current `LangWatchSpan` object. This could be a span initiated by `@langwatch.span`, the root span of a `@langwatch.trace`, or a `LangWatchSpan` wrapping an existing OpenTelemetry span if no LangWatch span is directly in context.
## Core Data Types
This section describes common data structures used throughout the LangWatch SDK, particularly as parameters to functions like `langwatch.setup()`, `@langwatch.trace()`, and `@langwatch.span()`, or as part of the data captured.
### `SpanProcessingExcludeRule`
Defines a rule to filter out spans before they are exported to LangWatch. Used in the `span_exclude_rules` parameter of `langwatch.setup()`.
The field of the span to match against. Currently, only `"span_name"` is supported.
The value to match for the specified `field_name`.
The operation to use for matching (e.g., `"exact_match"`, `"starts_with"`).
```python Example
from langwatch.domain import SpanProcessingExcludeRule
exclude_health_checks = SpanProcessingExcludeRule(
field_name="span_name",
match_value="/health_check",
match_operation="exact_match"
)
exclude_internal_utils = SpanProcessingExcludeRule(
field_name="span_name",
match_value="utils.",
match_operation="starts_with"
)
```
### `ChatMessage`
Represents a single message in a chat conversation, typically used for `input` or `output` of LLM spans.
The role of the message sender. `ChatRole` is a Literal: `"system"`, `"user"`, `"assistant"`, `"function"`, `"tool"`, `"guardrail"`, `"evaluation"`, `"unknown"`.
The textual content of the message.
For assistant messages that involve a function call (legacy OpenAI format).
Name of the function to call.JSON string of arguments for the function.
For assistant messages that involve tool calls (current OpenAI format).
ID of the tool call.Type of the tool (usually `"function"`).Details of the function to be called (see `FunctionCall` above).
For messages that are responses from a tool, this is the ID of the tool call that this message is a response to.
The name of the function whose result is in the `content` (if role is `"function"`), or the name of the tool/participant (if role is `"tool"`).
```python Example
from langwatch.domain import ChatMessage
user_message = ChatMessage(role="user", content="Hello, world!")
assistant_response = ChatMessage(role="assistant", content="Hi there!")
```
### `RAGChunk`
Represents a chunk of retrieved context, typically used with RAG (Retrieval Augmented Generation) operations in the `contexts` field of a span.
An identifier for the source document of this chunk.
An identifier for this specific chunk within the document.
The actual content of the RAG chunk. Can be a simple string or a more complex structured object.
```python Example
from langwatch.domain import RAGChunk
retrieved_chunk = RAGChunk(
document_id="doc_123",
chunk_id="chunk_005",
content="LangWatch helps monitor LLM applications."
)
```
### `SpanInputOutput`
This is a flexible type used for the `input` and `output` fields of spans. It's a Union that can take several forms to represent different kinds of data. LangWatch will store it as a typed value.
Common forms include:
- `TypedValueText`: For simple string inputs/outputs. `{"type": "text", "value": "your string"}`
- `TypedValueChatMessages`: For conversational inputs/outputs. `{"type": "chat_messages", "value": [ChatMessage, ...]}`
- `TypedValueJson`: For arbitrary JSON-serializable data. `{"type": "json", "value": {"key": "value"}}`
- `TypedValueRaw`: For data that should be stored as a raw string, escaping any special interpretation. `{"type": "raw", "value": "data"}`
- `TypedValueList`: For a list of `SpanInputOutput` objects. `{"type": "list", "value": [SpanInputOutput, ...]}`
When providing `input` or `output` to `@langwatch.span()` or `span.update()`, you can often provide the raw Python object (e.g., a string, a list of `ChatMessage` dicts, a dictionary for JSON), and the SDK will attempt to serialize it correctly. For more control, you can construct the `TypedValue` dictionaries yourself.
### `SpanTimestamps`
A dictionary defining custom start and end times for a span, in nanoseconds since epoch.
Timestamp when the span started (nanoseconds since epoch).
For LLM operations, timestamp when the first token was received (nanoseconds since epoch).
Timestamp when the span finished (nanoseconds since epoch).
### `SpanTypes` (Semantic Span Types)
A string literal defining the semantic type of a span. This helps categorize spans in the LangWatch UI and for analytics. Possible values include:
- `"span"` (generic span)
- `"llm"` (Language Model operation)
- `"chain"` (a sequence of operations, e.g., LangChain chain)
- `"tool"` (execution of a tool or function call)
- `"agent"` (an autonomous agent's operation)
- `"guardrail"` (a guardrail or safety check)
- `"evaluation"` (an evaluation step)
- `"rag"` (Retrieval Augmented Generation operation)
- `"workflow"` (a broader workflow or business process)
- `"component"` (a sub-component within a larger system)
- `"module"` (a logical module of code)
- `"server"` (server-side operation)
- `"client"` (client-side operation)
- `"producer"` (message producer)
- `"consumer"` (message consumer)
- `"task"` (a background task or job)
- `"unknown"`
### `SpanMetrics`
A dictionary for quantitative measurements associated with a span.
Number of tokens in the input/prompt to an LLM.
Number of tokens in the output/completion from an LLM.
Estimated or actual monetary cost of the operation (e.g., LLM API call cost).
### `SpanParams`
A dictionary for parameters related to an operation, especially LLM calls.
Examples include:
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
Modify the likelihood of specified tokens appearing in the completion. Accepts a dictionary mapping token IDs (or tokens, depending on the model) to a bias value from -100 to 100.
Whether to return log probabilities of the output tokens.
An integer between 0 and 5 specifying the number of most likely tokens to return at each token position, each with log probability. `logprobs` must be `True` if this is used.
The maximum number of tokens to generate in the completion.
How many completions to generate for each prompt.
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
If specified, the system will make a best effort to sample deterministically, such that repeated requests with the same `seed` and parameters should return the same result.
Up to 4 sequences where the API will stop generating further tokens.
If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available.
What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with `top_p` probability mass.
A list of tools the model may call. Currently, only functions are supported as a tool.
Controls which (if any) tool is called by the model. `none` means the model will not call any tool. `auto` means the model can pick between generating a message or calling a tool.
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse.
### `TraceMetadata`
A dictionary (`
---
# FILE: ./integration/python/guide.mdx
---
title: Python Integration Guide
sidebarTitle: Python
description: LangWatch Python SDK integration guide
keywords: LangWatch, Python, SDK, integration, guide, setup, tracing, spans, traces, OpenTelemetry, OpenAI, Celery, HTTP clients, databases, ORMs
---
Integrate LangWatch into your Python application to start observing your LLM interactions. This guide covers the setup and basic usage of the LangWatch Python SDK.
## Get your LangWatch API Key
First, you need a LangWatch API key. Sign up at [app.langwatch.ai](https://app.langwatch.ai) and find your API key in your project settings. The SDK will automatically use the `LANGWATCH_API_KEY` environment variable if it is set.
## Start Instrumenting
First, ensure you have the SDK installed:
```bash
pip install langwatch
```
Initialize LangWatch early in your application, typically where you configure services:
```python
import langwatch
import os
langwatch.setup()
# Your application code...
```
If you have an existing OpenTelemetry setup in your application, please see the [Already using OpenTelemetry?](#already-using-opentelemetry) section below.
## Capturing Messages
- Each message triggering your LLM pipeline as a whole is captured with a [Trace](/concepts#traces-one-task-end-to-end).
- A [Trace](/concepts#traces-one-task-end-to-end) contains multiple [Spans](/concepts#spans-the-building-blocks), which are the steps inside your pipeline.
- A span can be an LLM call, a database query for a RAG retrieval, or a simple function transformation.
- Different types of [Spans](/concepts#spans-the-building-blocks) capture different parameters.
- [Spans](/concepts#spans-the-building-blocks) can be nested to capture the pipeline structure.
- [Traces](/concepts#traces-one-task-end-to-end) can be grouped together on LangWatch Dashboard by having the same [`thread_id`](/concepts#threads-the-whole-conversation) in their metadata, making the individual messages become part of a conversation.
- It is also recommended to provide the [`user_id`](/concepts#user-id) metadata to track user analytics.
### Creating a Trace
To capture an end-to-end operation, like processing a user message, you can wrap the main function or entry point with the `@langwatch.trace()` decorator. This automatically creates a root span for the entire operation.
```python
import langwatch
from openai import OpenAI
client = OpenAI()
@langwatch.trace()
async def handle_message():
# This whole function execution is now a single trace
langwatch.get_current_trace().autotrack_openai_calls(client) # Automatically capture OpenAI calls
# ... rest of your message handling logic ...
pass
```
You can customize the trace name and add initial metadata if needed:
```python
@langwatch.trace(name="My Custom Trace Name", metadata={"foo": "bar"})
async def handle_message():
# ...
pass
```
Within a traced function, you can access the current trace context using `langwatch.get_current_trace()`.
### Capturing a Span
To instrument specific parts of your pipeline within a trace (like an llm operation, rag retrieval, or external api call), use the `@langwatch.span()` decorator.
```python
import langwatch
from langwatch.types import RAGChunk
@langwatch.span(type="rag", name="RAG Document Retrieval") # Add type and custom name
def rag_retrieval(query: str):
# ... logic to retrieve documents ...
search_results = [
{"id": "doc-1", "content": "..." },
{"id": "doc-2", "content": "..." }
]
# Add specific context data to the span
langwatch.get_current_span().update(
contexts=[
RAGChunk(document_id=doc["id"], content=doc["content"])
for doc in search_results
],
retrieval_strategy="vector_search",
)
return search_results
@langwatch.trace()
async def handle_message(message: cl.Message):
# ...
retrieved_docs = rag_retrieval(message.content) # This call creates a nested span
# ...
```
The `@langwatch.span()` decorator automatically captures the decorated function's arguments as the span's `input` and its return value as the `output`. This behavior can be controlled via the `capture_input` and `capture_output` arguments (both default to `True`).
Spans created within a function **decorated** with `@langwatch.trace()` will automatically be nested under the main trace span. You can add additional `type`, `name`, `metadata`, and `events`, or override the automatic input/output using decorator arguments or the `update()` method on the span object obtained via `langwatch.get_current_span()`.
For detailed guidance on manually creating traces and spans using context managers or direct start/end calls, see the [Manual Instrumentation Tutorial](/integration/python/tutorials/manual-instrumentation).
## Full Setup
```python
import os
import langwatch
from langwatch.attributes import AttributeKey
from langwatch.domain import SpanProcessingExcludeRule
from community.instrumentors import OpenAIInstrumentor # Example instrumentor
from opentelemetry.sdk.trace import TracerProvider
# Example: Providing an existing TracerProvider
# existing_provider = TracerProvider()
# Example: Defining exclude rules
exclude_rules = [
SpanProcessingExcludeRule(
field_name=["span_name"],
match_value="GET /health_check",
match_operation="exact_match"
),
]
langwatch.setup(
api_key=os.getenv("LANGWATCH_API_KEY"),
endpoint_url="https://your-langwatch-instance.com", # Optional: Defaults to env var or cloud
base_attributes={
AttributeKey.ServiceName: "my-awesome-service",
AttributeKey.ServiceVersion: "1.2.3",
# Add other custom attributes here
},
instrumentors=[OpenAIInstrumentor()], # Optional: List of instrumentors that conform to the `Instrumentor` protocol
# tracer_provider=existing_provider, # Optional: Provide your own TracerProvider
debug=True, # Optional: Enable debug logging
disable_sending=False, # Optional: Disable sending traces
flush_on_exit=True, # Optional: Flush traces on exit (default: True)
span_exclude_rules=exclude_rules, # Optional: Rules to exclude spans
ignore_global_tracer_provider_override_warning=False # Optional: Silence warning if global provider exists
)
# Your application code...
```
### Options
Your LangWatch API key. If not provided, it uses the `LANGWATCH_API_KEY` environment variable.
The LangWatch endpoint URL. Defaults to the `LANGWATCH_ENDPOINT` environment variable or `https://app.langwatch.ai`.
A dictionary of attributes to add to all spans (e.g., service name, version). Automatically includes SDK name, version, and language.
A list of automatic instrumentors (e.g., `OpenAIInstrumentor`, `LangChainInstrumentor`) to capture data from supported libraries.
An existing OpenTelemetry `TracerProvider`. If provided, LangWatch will use it (adding its exporter) instead of creating a new one. If not provided, LangWatch checks the global provider or creates a new one.
Enable debug logging for LangWatch. Defaults to `False` or checks if the `LANGWATCH_DEBUG` environment variable is set to `"true"`.
If `True`, disables sending traces to the LangWatch server. Useful for testing or development.
If `True` (the default), the tracer provider will attempt to flush all pending spans when the program exits via `atexit`.
If provided, the SDK will exclude spans from being exported to LangWatch based on the rules defined in the list (e.g., matching span names).
If `True`, suppresses the warning message logged when an existing global `TracerProvider` is detected and LangWatch attaches its exporter to it instead of overriding it.
## Integrations
LangWatch offers seamless integrations with a variety of popular Python libraries and frameworks. These integrations provide automatic instrumentation, capturing relevant data from your LLM applications with minimal setup.
Below is a list of currently supported integrations. Click on each to learn more about specific setup instructions and available features:
- [Agno](/integration/python/integrations/agno)
- [AWS Bedrock](/integration/python/integrations/aws-bedrock)
- [Azure AI](/integration/python/integrations/azure-ai)
- [Crew AI](/integration/python/integrations/crew-ai)
- [DSPy](/integration/python/integrations/dspy)
- [Haystack](/integration/python/integrations/haystack)
- [Langchain](/integration/python/integrations/langchain)
- [LangGraph](/integration/python/integrations/langgraph)
- [LiteLLM](/integration/python/integrations/lite-llm)
- [OpenAI](/integration/python/integrations/open-ai)
- [OpenAI Agents](/integration/python/integrations/open-ai-agents)
- [OpenAI Azure](/integration/python/integrations/open-ai-azure)
- [Pydantic AI](/integration/python/integrations/pydantic-ai)
- [Other Frameworks](/integration/python/integrations/other)
If you are using a library that is not listed here, you can still instrument your application manually. See the [Manual Instrumentation Tutorial](/integration/python/tutorials/manual-instrumentation) for more details. Since LangWatch is built on OpenTelemetry, it also supports any library or framework that integrates with OpenTelemetry. We are also continuously working on adding support for more integrations.
---
# FILE: ./integration/python/integrations/langchain.mdx
---
title: LangChain Instrumentation
sidebarTitle: Python
description: Learn how to instrument Langchain applications with the LangWatch Python SDK.
keywords: langchain, instrumentation, callback, opentelemetry, langwatch, python, tracing, openinference, openllmetry
---
Langchain is a powerful framework for building LLM applications. LangWatch integrates with Langchain to provide detailed observability into your chains, agents, LLM calls, and tool usage.
This guide covers the primary approaches to instrumenting Langchain with LangWatch:
1. **Using LangWatch's Langchain Callback Handler (Recommended)**: The most direct method, using a specific callback provided by LangWatch to capture rich Langchain-specific trace data.
2. **Using Community OpenTelemetry Instrumentors**: Leveraging dedicated Langchain instrumentors like those from OpenInference or OpenLLMetry.
3. **Utilizing Langchain's Native OpenTelemetry Export (Advanced)**: Configuring Langchain to send its own OpenTelemetry traces to an endpoint where LangWatch can collect them.
## 1. Using LangWatch's Langchain Callback Handler (Recommended)
This is the preferred and most comprehensive method for instrumenting Langchain with LangWatch. The LangWatch SDK provides a `LangchainCallbackHandler` that deeply integrates with Langchain's event system.
```python
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from langchain.schema.runnable import Runnable
from langchain.schema.runnable.config import RunnableConfig
import os
import asyncio
# Initialize LangWatch
langwatch.setup()
# Standard Langchain setup
model = ChatOpenAI(streaming=True)
prompt = ChatPromptTemplate.from_messages(
[("system", "You are a concise assistant."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - QA with Callback")
async def handle_message_with_callback(user_question: str):
current_trace = langwatch.get_current_trace()
current_trace.update(metadata={"user_id": "callback-user"})
langwatch_callback = current_trace.get_langchain_callback()
final_response = ""
async for chunk in runnable.astream(
{"question": user_question},
config=RunnableConfig(callbacks=[langwatch_callback])
):
final_response += chunk
return final_response
async def main_callback():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping Langchain callback example.")
return
response = await handle_message_with_callback("What is Langchain? Explain briefly.")
print(f"AI (Callback): {response}")
if __name__ == "__main__":
asyncio.run(main_callback())
```
**How it Works:**
- `@langwatch.trace()`: Creates a parent LangWatch trace.
- `current_trace.get_langchain_callback()`: Retrieves a LangWatch-specific callback handler linked to the current trace.
- `RunnableConfig(callbacks=[langwatch_callback])`: Injects the handler into Langchain's execution. Langchain emits events (on_llm_start, on_chain_end, etc.), which the handler converts into detailed LangWatch spans, correctly parented under the main trace.
**Key points:**
- Provides the most detailed Langchain-specific structural information (chains, agents, tools, LLMs as distinct steps).
- Works for all Langchain execution methods (`astream`, `stream`, `invoke`, `ainvoke`).
## 2. Using Community OpenTelemetry Instrumentors
Dedicated Langchain instrumentors from libraries like OpenInference and OpenLLMetry can also be used to capture Langchain operations as OpenTelemetry traces, which LangWatch can then ingest.
### Instrumenting Langchain with Dedicated Instrumentors
#### i. Via `langwatch.setup()`
```python OpenInference
# OpenInference Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from openinference.instrumentation.langchain import LangchainInstrumentor
import os
import asyncio
langwatch.setup(
instrumentors=[LangchainInstrumentor()] # Add OpenInference LangchainInstrumentor
)
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenInference via Setup")
async def handle_message_oinference_setup(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_oinference_setup():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenInference Langchain (setup) example.")
return
response = await handle_message_oinference_setup("Explain Langchain instrumentation with OpenInference.")
print(f"AI (OInference Setup): {response}")
if __name__ == "__main__":
asyncio.run(main_community_oinference_setup())
```
```python OpenLLMetry
# OpenLLMetry Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from opentelemetry.instrumentation.langchain import LangchainInstrumentor # Corrected import
import os
import asyncio
langwatch.setup(
instrumentors=[LangchainInstrumentor()] # Add OpenLLMetry LangchainInstrumentor
)
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenLLMetry via Setup")
async def handle_message_openllmetry_setup(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_openllmetry_setup():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenLLMetry Langchain (setup) example.")
return
response = await handle_message_openllmetry_setup("Explain Langchain instrumentation with OpenLLMetry.")
print(f"AI (OpenLLMetry Setup): {response}")
if __name__ == "__main__":
asyncio.run(main_community_openllmetry_setup())
```
#### ii. Direct Instrumentation
```python OpenInference
# OpenInference Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from openinference.instrumentation.langchain import LangchainInstrumentor
import os
import asyncio
langwatch.setup()
LangchainInstrumentor().instrument() # Instrument Langchain directly
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are very brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenInference Direct")
async def handle_message_oinference_direct(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_oinference_direct():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenInference Langchain (direct) example.")
return
response = await handle_message_oinference_direct("How does direct Langchain instrumentation work?")
print(f"AI (OInference Direct): {response}")
if __name__ == "__main__":
asyncio.run(main_community_oinference_direct())
```
```python OpenLLMetry
# OpenLLMetry Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from opentelemetry.instrumentation.langchain import LangchainInstrumentor # Corrected import
import os
import asyncio
langwatch.setup()
LangchainInstrumentor().instrument() # Instrument Langchain directly
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are very brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenLLMetry Direct")
async def handle_message_openllmetry_direct(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_openllmetry_direct():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenLLMetry Langchain (direct) example.")
return
response = await handle_message_openllmetry_direct("How does direct Langchain instrumentation work with OpenLLMetry?")
print(f"AI (OpenLLMetry Direct): {response}")
if __name__ == "__main__":
asyncio.run(main_community_openllmetry_direct())
```
**Key points for dedicated Langchain instrumentors:**
- Directly instrument Langchain operations, providing traces from Langchain's perspective.
- Requires installing the respective instrumentation package (e.g., `openinference-instrumentation-langchain` or `opentelemetry-instrumentation-langchain` for OpenLLMetry).
## 3. Utilizing Langchain's Native OpenTelemetry Export (Advanced)
Langchain itself can be configured to export OpenTelemetry traces. If you set this up and configure Langchain to send traces to an OpenTelemetry collector endpoint that LangWatch is also configured to receive from (or if LangWatch *is* your OTLP endpoint), then LangWatch can ingest these natively generated Langchain traces.
**Setup (Conceptual):**
1. Configure Langchain for OpenTelemetry export. This usually involves setting environment variables:
```bash
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_ENDPOINT= # Your OTLP gRPC endpoint (e.g., http://localhost:4317)
# Potentially other OTEL_EXPORTER_OTLP_* variables for more granular control
```
2. Initialize LangWatch: `langwatch.setup()`.
```python
# This example assumes Langchain is configured via environment variables
# to send OTel traces to an endpoint LangWatch can access.
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
import os
import asyncio
langwatch.setup() # LangWatch is ready to receive OTel traces
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - Native OTel Export") # Optional: group Langchain traces
async def handle_message_native_otel(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_native_otel():
if not os.getenv("OPENAI_API_KEY") or not os.getenv("LANGCHAIN_TRACING_V2") == "true":
print("Required env vars (OPENAI_API_KEY, LANGCHAIN_TRACING_V2='true') not set. Skipping native OTel.")
return
response = await handle_message_native_otel("Tell me about Langchain OTel export itself.")
print(f"AI (Native OTel): {response}")
if __name__ == "__main__":
asyncio.run(main_native_otel())
```
**Key points for Langchain's native OTel export:**
- LangWatch acts as a backend/collector for OpenTelemetry traces generated directly by Langchain.
- Requires careful configuration of Langchain's environment variables.
- The level of detail depends on Langchain's native OpenTelemetry instrumentation quality.
### Which Approach to Choose?
- **LangWatch's Langchain Callback Handler (Recommended)**: Provides the richest, most Langchain-aware traces directly integrated with LangWatch's tracing context. Ideal for most users.
- **Dedicated Langchain Instrumentors (OpenInference, OpenLLMetry)**: Good alternatives if you prefer an explicit instrumentor pattern for Langchain itself or are standardizing on these specific OpenTelemetry ecosystems.
- **Langchain's Native OTel Export (Advanced)**: Suitable if you have an existing OpenTelemetry collection infrastructure and want Langchain to be another OTel-compliant source.
For the best Langchain-specific observability within LangWatch, the **Langchain Callback Handler** is the generally recommended approach, with dedicated **Langchain Instrumentors** as strong alternatives for instrumentor-based setups.
---
# FILE: ./integration/python/integrations/agno.mdx
---
title: Agno Instrumentation
sidebarTitle: Agno
description: Learn how to instrument Agno agents and send traces to LangWatch using the Python SDK.
keywords: agno, openinference, openlit, opentelemetry, LangWatch, Python, tracing, observability
---
LangWatch supports seamless observability for [Agno](https://github.com/agno-agi/agno) agents. You can instrument your Agno-based applications and send traces directly to LangWatch for monitoring and analysis.
This guide shows how to set up tracing for Agno agents using the LangWatch Python SDK. Unlike the default OpenTelemetry/OTLP setup, you only need to call `langwatch.setup()`—no manual exporter or environment variable configuration is required.
## Prerequisites
- Install the required packages:
```bash
pip install agno openai langwatch openinference-instrumentation-agno
```
- Get your LangWatch API key from your [project settings](https://app.langwatch.ai/) and set it as an environment variable:
```bash
export LANGWATCH_API_KEY=your-langwatch-api-key
```
## Instrumenting Agno with LangWatch
You can use the [OpenInference Agno instrumentor](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-agno) to automatically capture traces from your Agno agents. Just pass the instrumentor to `langwatch.setup()`.
```python
import langwatch
import os
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.yfinance import YFinanceTools
from openinference.instrumentation.agno import AgnoInstrumentor
# Initialize LangWatch and instrument Agno
langwatch.setup(
instrumentors=[AgnoInstrumentor()]
)
# Create and configure your Agno agent
agent = Agent(
name="Stock Price Agent",
model=OpenAIChat(id="gpt-4o-mini"),
tools=[YFinanceTools()],
instructions="You are a stock price agent. Answer questions in the style of a stock analyst.",
debug_mode=True,
)
# Use the agent as usual—traces will be sent to LangWatch
agent.print_response("What is the current price of Tesla?")
```
**That's it!** All Agno agent activity will now be traced and sent to your LangWatch dashboard.
## Notes
- You do **not** need to set any OpenTelemetry environment variables or configure exporters manually—`langwatch.setup()` handles everything.
- You can combine Agno instrumentation with other instrumentors (e.g., OpenAI, LangChain) by adding them to the `instrumentors` list.
- For advanced configuration (custom attributes, endpoint, etc.), see the [Python integration guide](/integration/python/guide).
## Troubleshooting
- Make sure your `LANGWATCH_API_KEY` is set in the environment.
- If you see no traces in LangWatch, check that the instrumentor is included in `langwatch.setup()` and that your agent code is being executed.
For more details, see the [LangWatch Python SDK reference](/integration/python/reference) and the [Agno documentation](https://docs.agno.com/observability/langfuse).
---
# FILE: ./integration/python/integrations/haystack.mdx
---
title: Haystack Instrumentation
sidebarTitle: Haystack
description: Learn how to instrument Haystack pipelines with LangWatch using community OpenTelemetry instrumentors.
keywords: haystack, deepset, instrumentation, openinference, openllmetry, LangWatch, Python
---
LangWatch does not have a built-in auto-tracking integration for Haystack from deepset.ai. However, you can leverage community-provided OpenTelemetry instrumentors to integrate your Haystack pipelines with LangWatch.
## Integrating Community Instrumentors with LangWatch
Community-provided OpenTelemetry instrumentors for Haystack allow you to automatically capture detailed trace data from your Haystack pipeline components (Nodes, Pipelines, etc.). LangWatch can seamlessly integrate with these instrumentors.
There are two main ways to integrate these:
### 1. Via `langwatch.setup()`
You can pass an instance of the Haystack instrumentor to the `instrumentors` list in the `langwatch.setup()` call. LangWatch will then manage the lifecycle of this instrumentor.
```python openinference_setup.py
import langwatch
import os
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import PromptNode, PromptTemplate
from haystack.pipelines import Pipeline
from openinference.instrumentation.haystack import HaystackInstrumentor # Assuming this is the correct import
langwatch.setup(
instrumentors=[HaystackInstrumentor()]
)
# Initialize Haystack components
document_store = InMemoryDocumentStore(use_bm25=True)
# You can add documents to the store here if needed
prompt_template = PromptTemplate(
prompt="Answer the question based on the context: {query}",
output_parser=None # Or specify an output parser
)
prompt_node = PromptNode(
model_name_or_path="gpt-4o-mini", # Replace with your desired model
api_key=os.environ.get("OPENAI_API_KEY"),
default_prompt_template=prompt_template
)
pipeline = Pipeline()
pipeline.add_node(component=prompt_node, name="PromptNode", inputs=["Query"])
@langwatch.trace(name="Haystack Pipeline with OpenInference (Setup)")
def run_haystack_pipeline_oi_setup(query: str):
# The OpenInference instrumentor, when configured via langwatch.setup(),
# should automatically capture Haystack operations.
result = pipeline.run(query=query)
return result
if __name__ == "__main__":
if not os.environ.get("OPENAI_API_KEY"):
print("Please set the OPENAI_API_KEY environment variable.")
else:
user_query = "What is the capital of France?"
print(f"Running Haystack pipeline with OpenInference (setup) for query: {user_query}")
output = run_haystack_pipeline_oi_setup(user_query)
print("\n\nHaystack Pipeline Output:")
print(output)
```
```python openllmetry_setup.py
import langwatch
import os
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import PromptNode, PromptTemplate
from haystack.pipelines import Pipeline
from opentelemetry_instrumentation_haystack import HaystackInstrumentor # Assuming this is the correct import
langwatch.setup(
instrumentors=[HaystackInstrumentor()]
)
# Initialize Haystack components
document_store = InMemoryDocumentStore(use_bm25=True)
prompt_template = PromptTemplate(prompt="Answer the question: {query}")
prompt_node = PromptNode(
model_name_or_path="gpt-4o-mini",
api_key=os.environ.get("OPENAI_API_KEY"),
default_prompt_template=prompt_template
)
pipeline = Pipeline()
pipeline.add_node(component=prompt_node, name="MyPromptNode", inputs=["Query"])
@langwatch.trace(name="Haystack Pipeline with OpenLLMetry (Setup)")
def run_haystack_pipeline_ollm_setup(query: str):
# The OpenLLMetry instrumentor should automatically capture Haystack operations.
result = pipeline.run(query=query)
return result
if __name__ == "__main__":
if not os.environ.get("OPENAI_API_KEY"):
print("Please set the OPENAI_API_KEY environment variable.")
else:
user_query = "What is deep learning?"
print(f"Running Haystack pipeline with OpenLLMetry (setup) for query: {user_query}")
output = run_haystack_pipeline_ollm_setup(user_query)
print("\n\nHaystack Pipeline Output:")
print(output)
```
Ensure you have the respective community instrumentation library installed:
- For OpenInference: `pip install openinference-instrumentation-haystack`
- For OpenLLMetry: `pip install opentelemetry-instrumentation-haystack`
You'll also need Haystack: `pip install farm-haystack[openai]` (if using OpenAI models).
Consult the specific library's documentation for the exact package name and instrumentor class if the above assumptions are incorrect.
### 2. Direct Instrumentation
If you have an existing OpenTelemetry `TracerProvider` configured in your application (or if LangWatch is configured to use the global provider), you can use the community instrumentor's `instrument()` method directly. LangWatch will automatically pick up the spans generated by these instrumentors as long as its exporter is part of the active `TracerProvider`.
```python openinference_direct.py
import langwatch
import os
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import PromptNode, PromptTemplate
from haystack.pipelines import Pipeline
from openinference.instrumentation.haystack import HaystackInstrumentor # Assuming this is the correct import
langwatch.setup()
# Instrument Haystack directly using OpenInference
HaystackInstrumentor().instrument()
document_store = InMemoryDocumentStore(use_bm25=True)
prompt_template = PromptTemplate(prompt="Summarize this for a 5-year old: {query}")
prompt_node = PromptNode(
model_name_or_path="gpt-4o-mini",
api_key=os.environ.get("OPENAI_API_KEY"),
default_prompt_template=prompt_template
)
pipeline = Pipeline()
pipeline.add_node(component=prompt_node, name="SummarizerNode", inputs=["Query"])
@langwatch.trace(name="Haystack Pipeline with OpenInference (Direct)")
def run_haystack_pipeline_oi_direct(query: str):
# Spans from Haystack operations should be captured by the directly configured instrumentor.
result = pipeline.run(query=query)
return result
if __name__ == "__main__":
if not os.environ.get("OPENAI_API_KEY"):
print("Please set the OPENAI_API_KEY environment variable.")
else:
user_query = "The quick brown fox jumps over the lazy dog."
print(f"Running Haystack pipeline with OpenInference (direct) for query: {user_query}")
output = run_haystack_pipeline_oi_direct(user_query)
print("\n\nHaystack Pipeline Output:")
print(output)
```
```python openllmetry_direct.py
import langwatch
import os
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import PromptNode, PromptTemplate
from haystack.pipelines import Pipeline
from opentelemetry_instrumentation_haystack import HaystackInstrumentor # Assuming this is the correct import
langwatch.setup()
# Instrument Haystack directly using OpenLLMetry
HaystackInstrumentor().instrument()
document_store = InMemoryDocumentStore(use_bm25=True)
prompt_template = PromptTemplate(prompt="Translate to French: {query}")
prompt_node = PromptNode(
model_name_or_path="gpt-4o-mini",
api_key=os.environ.get("OPENAI_API_KEY"),
default_prompt_template=prompt_template
)
pipeline = Pipeline()
pipeline.add_node(component=prompt_node, name="TranslatorNode", inputs=["Query"])
@langwatch.trace(name="Haystack Pipeline with OpenLLMetry (Direct)")
def run_haystack_pipeline_ollm_direct(query: str):
result = pipeline.run(query=query)
return result
if __name__ == "__main__":
if not os.environ.get("OPENAI_API_KEY"):
print("Please set the OPENAI_API_KEY environment variable.")
else:
user_query = "Hello, how are you?"
print(f"Running Haystack pipeline with OpenLLMetry (direct) for query: {user_query}")
output = run_haystack_pipeline_ollm_direct(user_query)
print("\n\nHaystack Pipeline Output:")
print(output)
```
### Key points for community instrumentors:
- These instrumentors typically patch Haystack at a global level or integrate deeply with its execution flow (e.g., via `BaseComponent` or `Pipeline` hooks), meaning most Haystack operations should be captured once instrumented.
- If using `langwatch.setup(instrumentors=[...])`, LangWatch handles the setup and lifecycle of the instrumentor.
- If instrumenting directly (e.g., `HaystackInstrumentor().instrument()`), ensure that the `TracerProvider` used by the instrumentor is the same one LangWatch is exporting from. This usually means LangWatch is configured to use an existing global provider or one you explicitly pass to `langwatch.setup()`.
- Always refer to the specific documentation of the community instrumentor (OpenInference or OpenLLMetry) for the most accurate and up-to-date installation and usage instructions, including the correct class names for instrumentors and any specific setup requirements for different Haystack versions or components.
- The examples use a `PromptNode` with an OpenAI model for simplicity. Ensure you have the necessary API keys (e.g., `OPENAI_API_KEY`) set in your environment if you run these examples.
---
# FILE: ./integration/python/integrations/open-ai-azure.mdx
---
title: Azure OpenAI Instrumentation
sidebarTitle: Azure OpenAI
description: Learn how to instrument Azure OpenAI API calls with the LangWatch Python SDK
keywords: azure openai, openai, instrumentation, autotrack, openinference, openllmetry, LangWatch, Python
---
LangWatch offers robust integration with Azure OpenAI, allowing you to capture detailed information about your LLM calls automatically. There are two primary approaches to instrumenting your Azure OpenAI interactions:
1. **Using `autotrack_openai_calls()`**: This method, part of the LangWatch SDK, dynamically patches your `AzureOpenAI` client instance to capture calls made through it within a specific trace.
2. **Using Community OpenTelemetry Instrumentors**: Leverage existing OpenTelemetry instrumentation libraries like those from OpenInference or OpenLLMetry. These can be integrated with LangWatch by either passing them to the `langwatch.setup()` function or by using their native `instrument()` methods if you're managing your OpenTelemetry setup more directly.
This guide will walk you through both methods.
## Using `autotrack_openai_calls()`
The `autotrack_openai_calls()` function provides a straightforward way to capture all Azure OpenAI calls made with a specific client instance for the duration of the current trace.
You typically call this method on the trace object obtained via `langwatch.get_current_trace()` inside a function decorated with `@langwatch.trace()`.
```python
import langwatch
from openai import AzureOpenAI
import os
# Ensure LANGWATCH_API_KEY is set in your environment, or set it in `setup`
langwatch.setup()
# Initialize your AzureOpenAI client
# Ensure your Azure environment variables are set:
# AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT_NAME
client = AzureOpenAI(
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
api_version="2023-05-15" # Or your preferred API version
)
@langwatch.trace(name="Azure OpenAI Chat Completion")
async def get_azure_openai_chat_response(user_prompt: str):
# Get the current trace and enable autotracking for the 'client' instance
langwatch.get_current_trace().autotrack_openai_calls(client)
# All calls made with 'client' will now be automatically captured as spans
response = client.chat.completions.create(
model=os.getenv("AZURE_OPENAI_DEPLOYMENT_NAME"), # Use your Azure deployment name
messages=[{"role": "user", "content": user_prompt}],
)
completion = response.choices[0].message.content
return completion
async def main():
user_query = "Tell me a fact about the Azure cloud."
response = await get_azure_openai_chat_response(user_query)
print(f"User: {user_query}")
print(f"AI: {response}")
if __name__ == "__main__":
import asyncio
asyncio.run(main())
```
Key points for `autotrack_openai_calls()` with Azure OpenAI:
- It must be called on an active trace object (e.g., obtained via `langwatch.get_current_trace()`).
- It instruments a *specific instance* of the `AzureOpenAI` client. If you have multiple clients, you'll need to call it for each one you want to track.
- Ensure your `AzureOpenAI` client is correctly configured with `azure_endpoint`, `api_key`, `api_version`, and you use the deployment name for the `model` parameter.
## Using Community OpenTelemetry Instrumentors
If you prefer to use broader OpenTelemetry-based instrumentation, or are already using libraries like `OpenInference` or `OpenLLMetry`, LangWatch can seamlessly integrate with them. These libraries provide instrumentors that automatically capture data from the `openai` library, which `AzureOpenAI` is part of.
There are two main ways to integrate these:
### 1. Via `langwatch.setup()`
You can pass an instance of the instrumentor (e.g., `OpenAIInstrumentor` from OpenInference) to the `instrumentors` list in the `langwatch.setup()` call. LangWatch will then manage the lifecycle of this instrumentor.
```python
import langwatch
from openai import AzureOpenAI
import os
from openinference.instrumentation.openai import OpenAIInstrumentor
# Initialize LangWatch with the OpenAIInstrumentor
langwatch.setup(
instrumentors=[OpenAIInstrumentor()]
)
# Initialize your AzureOpenAI client
client = AzureOpenAI(
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
api_version="2023-05-15"
)
@langwatch.trace(name="Azure OpenAI Call with Community Instrumentor")
def generate_text_with_community_instrumentor(prompt: str):
# No need to call autotrack explicitly, the community instrumentor handles OpenAI calls globally.
response = client.chat.completions.create(
model=os.getenv("AZURE_OPENAI_DEPLOYMENT_NAME", "your-deployment-name"), # Use your Azure deployment name
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
if __name__ == "__main__":
user_query = "Explain Azure Machine Learning in simple terms."
response = generate_text_with_community_instrumentor(user_query)
print(f"User: {user_query}")
print(f"AI: {response}")
```
Ensure you have the respective community instrumentation library installed (e.g., `pip install openllmetry-instrumentation-openai` or `pip install openinference-instrumentation-openai`). The instrumentor works with `AzureOpenAI` as it's part of the same `openai` Python package.
### 2. Direct Instrumentation
If you have an existing OpenTelemetry `TracerProvider` configured in your application (or if LangWatch is configured to use the global provider), you can use the community instrumentor's `instrument()` method directly. LangWatch will automatically pick up the spans generated by these instrumentors as long as its exporter is part of the active `TracerProvider`.
```python
import langwatch
from openai import AzureOpenAI
import os
from openinference.instrumentation.openai import OpenAIInstrumentor
langwatch.setup() # LangWatch sets up or uses the global OpenTelemetry provider
# Initialize your AzureOpenAI client
client = AzureOpenAI(
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
api_version="2023-05-15"
)
# Instrument OpenAI directly using the community library
# This will patch the openai library, affecting AzureOpenAI instances too.
OpenAIInstrumentor().instrument()
@langwatch.trace(name="Azure OpenAI Call with Direct Community Instrumentation")
def get_creative_idea(topic: str):
response = client.chat.completions.create(
model=os.getenv("AZURE_OPENAI_DEPLOYMENT_NAME", "your-deployment-name"), # Use your Azure deployment name
messages=[
{"role": "system", "content": "You are an idea generation bot."},
{"role": "user", "content": f"Generate a creative idea about {topic}."}
]
)
return response.choices[0].message.content
if __name__ == "__main__":
subject = "sustainable energy"
idea = get_creative_idea(subject)
print(f"Topic: {subject}")
print(f"AI's Idea: {idea}")
```
### Key points for community instrumentors with Azure OpenAI:
- These instrumentors often patch the `openai` library at a global level, meaning all calls from any `OpenAI` or `AzureOpenAI` client instance will be captured once instrumented.
- If using `langwatch.setup(instrumentors=[...])`, LangWatch handles the instrumentor's setup.
- If instrumenting directly (e.g., `OpenAIInstrumentor().instrument()`), ensure that the `TracerProvider` used by the instrumentor is the same one LangWatch is exporting from. This typically happens automatically if LangWatch initializes the global provider or if you configure them to use the same explicit provider.
### Which Approach to Choose?
- **`autotrack_openai_calls()`** is ideal for targeted instrumentation within specific traces or when you want fine-grained control over which `AzureOpenAI` client instances are tracked. It's simpler if you're not deeply invested in a separate OpenTelemetry setup.
- **Community Instrumentors** are powerful if you're already using OpenTelemetry, want to capture Azure OpenAI calls globally across your application, or need to instrument other libraries alongside Azure OpenAI with a consistent OpenTelemetry approach. They provide a more holistic observability solution if you have multiple OpenTelemetry-instrumented components.
Choose the method that best fits your existing setup and instrumentation needs. Both approaches effectively send Azure OpenAI call data to LangWatch for monitoring and analysis.
---
# FILE: ./integration/python/integrations/dspy.mdx
---
title: DSPy Instrumentation
sidebarTitle: DSPy
description: Learn how to instrument DSPy programs with the LangWatch Python SDK
keywords: dspy, instrumentation, autotrack, LangWatch, Python
---
LangWatch provides seamless integration with DSPy, allowing you to automatically capture detailed information about your DSPy program executions, including module calls and language model interactions.
The primary way to instrument your DSPy programs is by using `autotrack_dspy()` on the current LangWatch trace.
## Using `autotrack_dspy()`
The `autotrack_dspy()` function, when called on an active trace object, dynamically patches the DSPy framework to capture calls made during the execution of your DSPy programs within that trace.
You typically call this method on the trace object obtained via `langwatch.get_current_trace()` inside a function decorated with `@langwatch.trace()`. This ensures that all DSPy operations within that traced function are monitored.
```python
import langwatch
import dspy
import os
# Ensure LANGWATCH_API_KEY is set, or set it in langwatch.setup()
langwatch.setup() # If not done elsewhere or API key not in env
# Initialize your DSPy LM (Language Model)
# This example uses OpenAI, ensure OPENAI_API_KEY is set
lm = dspy.LM("openai/gpt-4o-mini", api_key=os.environ.get("OPENAI_API_KEY"))
dspy.settings.configure(lm=lm)
class BasicRAG(dspy.Signature):
"""Answer questions with short factoid answers."""
question = dspy.InputField()
answer = dspy.OutputField(desc="often between 1 and 5 words")
class SimpleModule(dspy.Module):
def __init__(self):
super().__init__()
self.generate_answer = dspy.Predict(BasicRAG)
def forward(self, question):
prediction = self.generate_answer(question=question)
return dspy.Prediction(answer=prediction.answer)
@langwatch.trace(name="DSPy RAG Execution")
def run_dspy_program(user_query: str):
# Get the current trace and enable autotracking for DSPy
current_trace = langwatch.get_current_trace().autotrack_dspy()
program = SimpleModule()
prediction = program(question=user_query)
return prediction.answer
def main():
user_question = "What is the capital of France?"
response = run_dspy_program(user_question)
print(f"Question: {user_question}")
print(f"Answer: {response}")
if __name__ == "__main__":
main()
```
### Key points for `autotrack_dspy()`:
- It must be called on an active trace object (e.g., obtained via `langwatch.get_current_trace()`).
- It instruments DSPy operations specifically for the duration and scope of the current trace.
- LangWatch will capture interactions with DSPy modules (like `dspy.Predict`, `dspy.ChainOfThought`, `dspy.Retrieve`) and the underlying LM calls.
## Using Community OpenTelemetry Instrumentors
If you prefer to use broader OpenTelemetry-based instrumentation, or are already using libraries like OpenInference, LangWatch can seamlessly integrate with them. These libraries provide instrumentors that automatically capture data from various LLM frameworks, including DSPy.
The [OpenInference community provides an instrumentor for DSPy](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-dspy) which can be used with LangWatch.
There are two main ways to integrate these:
### 1. Via `langwatch.setup()`
You can pass an instance of the instrumentor (e.g., `OpenInferenceDSPyInstrumentor` from OpenInference) to the `instrumentors` list in the `langwatch.setup()` call. LangWatch will then manage the lifecycle of this instrumentor.
```python
import langwatch
import dspy
import os
from openinference.instrumentation.dspy import DSPyInstrumentor
# Initialize LangWatch with the DSPyInstrumentor
langwatch.setup(
instrumentors=[DSPyInstrumentor()]
)
# Configure DSPy LM
# This example uses OpenAI, ensure OPENAI_API_KEY is set
lm = dspy.LM("openai/gpt-4o-mini", api_key=os.environ.get("OPENAI_API_KEY"))
dspy.settings.configure(lm=lm)
class BasicRAG(dspy.Signature):
question = dspy.InputField()
answer = dspy.OutputField()
class SimpleModule(dspy.Module):
def __init__(self):
super().__init__()
self.generate_answer = dspy.Predict(BasicRAG)
def forward(self, question):
prediction = self.generate_answer(question=question)
return dspy.Prediction(answer=prediction.answer)
@langwatch.trace(name="DSPy Call with Community Instrumentor")
def generate_with_community_instrumentor(prompt: str):
# No need to call autotrack_dspy explicitly,
# the community instrumentor handles DSPy calls globally.
program = SimpleModule()
prediction = program(question=prompt)
return prediction.answer
if __name__ == "__main__":
# from dotenv import load_dotenv # Make sure to load .env if you have one
# load_dotenv()
user_query = "What is DSPy?"
response = generate_with_community_instrumentor(user_query)
print(f"User: {user_query}")
print(f"AI: {response}")
```
Ensure you have the respective community instrumentation library installed (e.g., `pip install openinference-instrumentation-dspy`).
### 2. Direct Instrumentation
If you have an existing OpenTelemetry `TracerProvider` configured in your application (or if LangWatch is configured to use the global provider), you can use the community instrumentor's `instrument()` method directly. LangWatch will automatically pick up the spans generated by these instrumentors as long as its exporter is part of the active `TracerProvider`.
```python
import langwatch
import dspy
import os
from openinference.instrumentation.dspy import DSPyInstrumentor
langwatch.setup() # Initialize LangWatch
# Configure DSPy LM
lm = dspy.LM("openai/gpt-4o-mini", api_key=os.environ.get("OPENAI_API_KEY"))
dspy.settings.configure(lm=lm)
# Instrument DSPy directly using the community library
DSPyInstrumentor().instrument()
class BasicRAG(dspy.Signature):
question = dspy.InputField()
answer = dspy.OutputField()
class SimpleModule(dspy.Module):
def __init__(self):
super().__init__()
self.generate_answer = dspy.Predict(BasicRAG)
def forward(self, question):
prediction = self.generate_answer(question=question)
return dspy.Prediction(answer=prediction.answer)
@langwatch.trace(name="DSPy Call with Direct Community Instrumentation")
def ask_dspy_directly_instrumented(original_question: str):
program = SimpleModule()
prediction = program(question=original_question)
return prediction.answer
if __name__ == "__main__":
query = "Explain the concept of few-shot learning in LLMs."
response = ask_dspy_directly_instrumented(query)
print(f"Query: {query}")
print(f"Response: {response}")
```
### Key points for community instrumentors:
- These instrumentors often patch DSPy at a global level, meaning all DSPy calls will be captured once instrumented.
- If using `langwatch.setup(instrumentors=[...])`, LangWatch handles the setup.
- If instrumenting directly (e.g., `DSPyInstrumentor().instrument()`), ensure that the `TracerProvider` used by the instrumentor is the same one LangWatch is exporting from. This usually means LangWatch is configured to use an existing global provider or one you explicitly pass to `langwatch.setup()`.
### Which Approach to Choose?
- **`autotrack_dspy()`** is ideal for targeted instrumentation within specific traces or when you want fine-grained control over which DSPy program executions are tracked. It's simpler if you're not deeply invested in a separate OpenTelemetry setup.
- **Community Instrumentors** (like OpenInference's `DSPyInstrumentor`) are powerful if you're already using OpenTelemetry, want to capture DSPy calls globally across your application, or need to instrument other libraries alongside DSPy with a consistent OpenTelemetry approach. They provide a more holistic observability solution if you have multiple OpenTelemetry-instrumented components.
Choose the method that best fits your existing setup and instrumentation needs. Both approaches effectively send DSPy call data to LangWatch for monitoring and analysis.
## Example: Chainlit Bot with DSPy and LangWatch
The following is a more complete example demonstrating `autotrack_dspy()` in a Chainlit application, similar to the `dspy_bot.py` found in the SDK examples.
```python
import os
import chainlit as cl
import langwatch
import dspy
langwatch.setup()
# Configure DSPy LM and RM (Retriever Model)
# Ensure OPENAI_API_KEY is in your environment for the LM
lm = dspy.LM("openai/gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"])
# This example uses a public ColBERTv2 instance for retrieval
colbertv2_wiki17_abstracts = dspy.ColBERTv2(
url="http://20.102.90.50:2017/wiki17_abstracts"
)
dspy.settings.configure(lm=lm, rm=colbertv2_wiki17_abstracts)
class GenerateAnswer(dspy.Signature):
"""Answer questions with short factoid answers."""
context = dspy.InputField(desc="may contain relevant facts")
question = dspy.InputField()
answer = dspy.OutputField(desc="often between 1 and 5 words")
class RAG(dspy.Module):
def __init__(self, num_passages=3):
super().__init__()
self.retrieve = dspy.Retrieve(k=num_passages)
self.generate_answer = dspy.ChainOfThought(GenerateAnswer)
def forward(self, question):
context = self.retrieve(question).passages # type: ignore
prediction = self.generate_answer(question=question, context=context)
return dspy.Prediction(answer=prediction.answer)
@cl.on_message # Decorator for Chainlit message handling
@langwatch.trace() # Decorator to trace this function with LangWatch
async def main_chat_handler(message: cl.Message):
# Get the current LangWatch trace and enable DSPy autotracking
current_trace = langwatch.get_current_trace()
if current_trace:
current_trace.autotrack_dspy()
msg_ui = cl.Message(content="") # Chainlit UI message
# Initialize and run the DSPy RAG program
program = RAG()
prediction = program(question=message.content)
# Stream the answer to the Chainlit UI
# Note: For simplicity, this example doesn't stream token by token
# from the DSPy prediction if it were a streaming response.
# DSPy's prediction.answer is typically the full string.
final_answer = prediction.answer
if isinstance(final_answer, str):
await msg_ui.stream_token(final_answer)
else:
# Handle cases where answer might not be a direct string (e.g. structured output)
await msg_ui.stream_token(str(final_answer))
await msg_ui.update()
# To run this example:
# 1. Ensure you have .env file with OPENAI_API_KEY and LANGWATCH_API_KEY.
# 2. Install dependencies: pip install langwatch dspy-ai openai chainlit python-dotenv
# 3. Run with Chainlit: chainlit run your_script_name.py -w
```
By calling `autotrack_dspy()` within your LangWatch-traced functions, you gain valuable insights into your DSPy program's behavior, including latencies, token counts, and the flow of data through your defined modules and signatures. This is essential for debugging, optimizing, and monitoring your DSPy-powered AI applications.
---
# FILE: ./integration/python/integrations/aws-bedrock.mdx
---
title: AWS Bedrock Instrumentation
sidebarTitle: AWS Bedrock
description: Learn how to instrument AWS Bedrock calls with the LangWatch Python SDK using OpenInference.
keywords: aws, bedrock, boto3, instrumentation, opentelemetry, openinference, langwatch, python, tracing
---
AWS Bedrock, accessed via the `boto3` library, allows you to leverage powerful foundation models. By using the OpenInference Bedrock instrumentor, you can automatically capture OpenTelemetry traces for your Bedrock API calls. LangWatch, being an OpenTelemetry-compatible observability platform, can seamlessly ingest these traces, providing insights into your LLM interactions.
This guide explains how to configure your Python application to send Bedrock traces to LangWatch.
## Prerequisites
1. **Install LangWatch SDK**:
```bash
pip install langwatch
```
2. **Install Bedrock Instrumentation and Dependencies**:
You'll need `boto3` to interact with AWS Bedrock, and the OpenInference instrumentation library for Bedrock.
```bash
pip install boto3 openinference-instrumentation-bedrock
```
Note: `openinference-instrumentation-bedrock` will install necessary OpenTelemetry packages. Ensure your `boto3` and `botocore` versions are compatible with the Bedrock features you intend to use (e.g., `botocore >= 1.34.116` for the `converse` API).
## Instrumenting AWS Bedrock with LangWatch
The integration involves initializing LangWatch to set up the OpenTelemetry environment and then applying the Bedrock instrumentor.
### Steps:
1. **Initialize LangWatch**: Call `langwatch.setup()` at the beginning of your application. This configures the global OpenTelemetry SDK to export traces to LangWatch.
2. **Instrument Bedrock**: Import `BedrockInstrumentor` and call its `instrument()` method. This will patch `boto3` to automatically create spans for Bedrock client calls.
```python
import langwatch
import boto3
import json
import os
import asyncio
# 1. Initialize LangWatch for OpenTelemetry trace export
langwatch.setup()
# 2. Instrument Boto3 for Bedrock
from openinference.instrumentation.bedrock import BedrockInstrumentor
BedrockInstrumentor().instrument()
# Global Bedrock client (initialize after instrumentation)
bedrock_runtime = None
try:
aws_session = boto3.session.Session(
region_name=os.environ.get("AWS_REGION_NAME") # Ensure region is set
)
bedrock_runtime = aws_session.client("bedrock-runtime")
except Exception as e:
print(f"Error creating Bedrock client: {e}. Ensure AWS credentials and region are configured.")
@langwatch.span(name="Bedrock - Invoke Claude")
async def invoke_claude(prompt_text: str):
if not bedrock_runtime:
print("Bedrock client not initialized. Skipping API call.")
return None
current_span = langwatch.get_current_span()
current_span.update(model_id="anthropic.claude-v2", action="invoke_model")
try:
body = json.dumps({
"prompt": f"Human: {prompt_text} Assistant:",
"max_tokens_to_sample": 200
})
response = bedrock_runtime.invoke_model(modelId="anthropic.claude-v2", body=body)
response_body = json.loads(response.get("body").read())
completion = response_body.get("completion")
current_span.update(outputs={"completion_preview": completion[:50] + "..." if completion else "N/A"})
return completion
except Exception as e:
print(f"Error invoking model: {e}")
if current_span:
current_span.record_exception(e)
current_span.set_status("error", str(e))
raise
@langwatch.trace(name="Bedrock - Example Usage")
async def main():
try:
prompt = "Explain the concept of OpenTelemetry in one sentence."
print(f"Invoking model with prompt: '{prompt}'")
response = await invoke_claude(prompt)
if response:
print(f"Response from Claude: {response}")
except Exception as e:
print(f"An error occurred in main: {e}")
if __name__ == "__main__":
asyncio.run(main())
```
**Key points for this approach:**
- `langwatch.setup()`: Initializes the global OpenTelemetry environment configured for LangWatch. This must be called before any instrumented code is run.
- `BedrockInstrumentor().instrument()`: This call patches the `boto3` library. Any subsequent Bedrock calls made using a `boto3.client("bedrock-runtime")` will automatically generate OpenTelemetry spans.
- `@langwatch.trace()`: Creates a parent trace in LangWatch. The automated Bedrock spans generated by OpenInference will be nested under this parent trace if the Bedrock calls are made within the decorated function. This provides a clear hierarchy for your operations.
- **API Versions**: The example shows both `invoke_model` and `converse` APIs. The `converse` API requires `botocore` version `1.34.116` or newer.
By following these steps, your application's interactions with AWS Bedrock will be traced, and the data will be sent to LangWatch for monitoring and analysis. This allows you to observe latencies, errors, and other metadata associated with your foundation model calls.
For more details on the specific attributes captured by the OpenInference Bedrock instrumentor, please refer to the [OpenInference Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/ai/openinference/). (Note: Link to general OTel AI/OpenInference conventions, specific Bedrock attributes might be detailed in OpenInference's own docs).
Remember to replace placeholder values for AWS credentials and adapt the model IDs and prompts to your specific use case.
---
# FILE: ./integration/python/integrations/crew-ai.mdx
---
title: CrewAI
description: Learn how to instrument the CrewAI Python SDK with LangWatch.
keywords: crewai, python, sdk, instrumentation, opentelemetry, langwatch, tracing
---
LangWatch does not have a built-in auto-tracking integration for CrewAI. However, you can use community-provided instrumentors to integrate CrewAI with LangWatch.
## Community Instrumentors
There are two main community instrumentors available for CrewAI:
OpenLLMetry provides an OpenTelemetry-based instrumentation package for CrewAI.
You can find more details and installation instructions on their GitHub repository:
[traceloop/openllmetry/packages/opentelemetry-instrumentation-crewai](https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-crewai)
OpenInference, by Arize AI, also offers an instrumentation solution for CrewAI, compatible with OpenTelemetry.
For more information and setup guides, please visit their GitHub repository:
[Arize-ai/openinference/python/instrumentation/openinference-instrumentation-crewai](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-crewai)
To use these instrumentors with LangWatch, you would typically configure them to export telemetry data via OpenTelemetry, which LangWatch can then ingest.
## Integrating Community Instrumentors with LangWatch
Community-provided OpenTelemetry instrumentors for CrewAI, like those from OpenLLMetry or OpenInference, allow you to automatically capture detailed trace data from your CrewAI agents and tasks. LangWatch can seamlessly integrate with these instrumentors.
There are two main ways to integrate these:
### 1. Via `langwatch.setup()`
You can pass an instance of the CrewAI instrumentor to the `instrumentors` list in the `langwatch.setup()` call. LangWatch will then manage the lifecycle of this instrumentor.
```python openinference_setup.py
import langwatch
from crewai import Agent, Task, Crew
import os
from openinference.instrumentation.crewai import CrewAIInstrumentor # Assuming this is the correct import
# Ensure LANGWATCH_API_KEY is set in your environment, or set it in `setup`
langwatch.setup(
instrumentors=[CrewAIInstrumentor()]
)
# Define your CrewAI agents and tasks
researcher = Agent(
role='Senior Researcher',
goal='Discover new insights on AI',
backstory='A seasoned researcher with a knack for uncovering hidden gems.'
)
writer = Agent(
role='Expert Writer',
goal='Craft compelling content on AI discoveries',
backstory='A wordsmith who can make complex AI topics accessible and engaging.'
)
task1 = Task(description='Investigate the latest advancements in LLM prompting techniques.', agent=researcher)
task2 = Task(description='Write a blog post summarizing the findings.', agent=writer)
# Create and run the crew
crew = Crew(
agents=[researcher, writer],
tasks=[task1, task2],
verbose=2
)
@langwatch.trace(name="CrewAI Execution with OpenInference")
def run_crewai_process_oi():
result = crew.kickoff()
return result
if __name__ == "__main__":
print("Running CrewAI process with OpenInference...")
output = run_crewai_process_oi()
print("\n\nCrewAI Process Output:")
print(output)
```
```python openllmetry_setup.py
import langwatch
from crewai import Agent, Task, Crew
import os
from opentelemetry_instrumentation_crewai import CrewAIInstrumentor # Assuming this is the correct import
# Ensure LANGWATCH_API_KEY is set in your environment, or set it in `setup`
langwatch.setup(
instrumentors=[CrewAIInstrumentor()]
)
# Define your CrewAI agents and tasks
researcher = Agent(
role='Senior Researcher',
goal='Discover new insights on AI',
backstory='A seasoned researcher with a knack for uncovering hidden gems.'
)
writer = Agent(
role='Expert Writer',
goal='Craft compelling content on AI discoveries',
backstory='A wordsmith who can make complex AI topics accessible and engaging.'
)
task1 = Task(description='Investigate the latest advancements in LLM prompting techniques.', agent=researcher)
task2 = Task(description='Write a blog post summarizing the findings.', agent=writer)
# Create and run the crew
crew = Crew(
agents=[researcher, writer],
tasks=[task1, task2],
verbose=2
)
@langwatch.trace(name="CrewAI Execution with OpenLLMetry")
def run_crewai_process_ollm():
result = crew.kickoff()
return result
if __name__ == "__main__":
print("Running CrewAI process with OpenLLMetry...")
output = run_crewai_process_ollm()
print("\n\nCrewAI Process Output:")
print(output)
```
Ensure you have the respective community instrumentation library installed:
- For OpenLLMetry: `pip install opentelemetry-instrumentation-crewai`
- For OpenInference: `pip install openinference-instrumentation-crewai`
Consult the specific library's documentation for the exact package name and instrumentor class if the above assumptions are incorrect.
### 2. Direct Instrumentation
If you have an existing OpenTelemetry `TracerProvider` configured in your application (or if LangWatch is configured to use the global provider), you can use the community instrumentor's `instrument()` method directly. LangWatch will automatically pick up the spans generated by these instrumentors as long as its exporter is part of the active `TracerProvider`.
```python openinference_direct.py
import langwatch
from crewai import Agent, Task, Crew
import os
from openinference.instrumentation.crewai import CrewAIInstrumentor # Assuming this is the correct import
# from opentelemetry.sdk.trace import TracerProvider # If managing your own provider
# from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter # If managing your own provider
langwatch.setup()
# Instrument CrewAI directly using OpenInference
CrewAIInstrumentor().instrument()
planner = Agent(
role='Event Planner',
goal='Plan an engaging tech conference',
backstory='An experienced planner with a passion for technology events.'
)
task_planner = Task(description='Outline the agenda for a 3-day AI conference.', agent=planner)
conference_crew = Crew(agents=[planner], tasks=[task_planner])
@langwatch.trace(name="CrewAI Direct Instrumentation with OpenInference")
def plan_conference_oi():
agenda = conference_crew.kickoff()
return agenda
if __name__ == "__main__":
print("Planning conference with OpenInference (direct)...")
conference_agenda = plan_conference_oi()
print("\n\nConference Agenda:")
print(conference_agenda)
```
```python openllmetry_direct.py
import langwatch
from crewai import Agent, Task, Crew
import os
from opentelemetry_instrumentation_crewai import CrewAIInstrumentor # Assuming this is the correct import
# from opentelemetry.sdk.trace import TracerProvider # If managing your own provider
# from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter # If managing your own provider
langwatch.setup()
# Instrument CrewAI directly using OpenLLMetry
CrewAIInstrumentor().instrument()
planner = Agent(
role='Event Planner',
goal='Plan an engaging tech conference',
backstory='An experienced planner with a passion for technology events.'
)
task_planner = Task(description='Outline the agenda for a 3-day AI conference.', agent=planner)
conference_crew = Crew(agents=[planner], tasks=[task_planner])
@langwatch.trace(name="CrewAI Direct Instrumentation with OpenLLMetry")
def plan_conference_ollm():
agenda = conference_crew.kickoff()
return agenda
if __name__ == "__main__":
print("Planning conference with OpenLLMetry (direct)...")
conference_agenda = plan_conference_ollm()
print("\n\nConference Agenda:")
print(conference_agenda)
```
### Key points for community instrumentors:
- These instrumentors typically patch CrewAI at a global level or integrate deeply with its execution flow, meaning all CrewAI operations (agents, tasks, tools) should be captured once instrumented.
- If using `langwatch.setup(instrumentors=[...])`, LangWatch handles the setup and lifecycle of the instrumentor.
- If instrumenting directly (e.g., `CrewAIInstrumentor().instrument()`), ensure that the `TracerProvider` used by the instrumentor is the same one LangWatch is exporting from. This usually means LangWatch is configured to use an existing global provider or one you explicitly pass to `langwatch.setup()`.
- Always refer to the specific documentation of the community instrumentor (OpenLLMetry or OpenInference) for the most accurate and up-to-date installation and usage instructions, including the correct class names for instrumentors and any specific setup requirements.
---
# FILE: ./integration/python/integrations/pydantic-ai.mdx
---
title: PydanticAI Instrumentation
sidebarTitle: PydanticAI
description: Learn how to instrument PydanticAI applications with the LangWatch Python SDK.
keywords: pydantic-ai, pydanticai, instrumentation, opentelemetry, langwatch, python, tracing
---
PydanticAI is a library for building AI applications with Pydantic models. It features built-in, optional support for OpenTelemetry, allowing detailed tracing of agent runs and model interactions. LangWatch, being an OpenTelemetry-compatible observability platform, can seamlessly ingest these traces.
This guide explains how to configure PydanticAI and LangWatch to capture this observability data. For more background on PydanticAI's observability features, refer to their debugging and monitoring documentation.
## Prerequisites
1. **Install LangWatch SDK**:
```bash
pip install langwatch
```
2. **Install PydanticAI**:
```bash
pip install pydantic-ai
```
## Instrumenting PydanticAI with LangWatch
The primary way to integrate PydanticAI with LangWatch is by leveraging PydanticAI's native OpenTelemetry emission in an environment configured by `langwatch.setup()`.
### Using PydanticAI's Built-in OpenTelemetry with LangWatch Global Setup
When `langwatch.setup()` is called, it initializes a global OpenTelemetry environment, including a trace exporter configured for LangWatch. If PydanticAI's instrumentation is enabled (via `Agent(instrument=True)` or `Agent.instrument_all()`), it will emit OpenTelemetry traces that are automatically captured by LangWatch.
```python
import langwatch
from pydantic_ai import Agent
from pydantic_ai.agent import InstrumentationSettings # Optional, for event_mode
import os
import asyncio
# 1. Initialize LangWatch
# This sets up the global OpenTelemetry environment for LangWatch.
langwatch.setup()
# 2. Enable PydanticAI Instrumentation
# Option A: Instrument all agents globally
Agent.instrument_all()
# Option B: Instrument a specific agent instance
# For this example, we'll instrument a specific agent.
# If targeting a generic OTel collector like LangWatch, event_mode='logs' is recommended
# as it aligns with OTel semantic conventions for capturing message events.
# The default mode might use a custom attribute format for events.
instrumentation_settings_for_langwatch = InstrumentationSettings(event_mode='logs')
agent = Agent(model_name='openai:gpt-4o-mini', instrument=instrumentation_settings_for_langwatch)
@langwatch.trace(name="PydanticAI - City Capital Query")
async def get_capital_city(country: str):
langwatch.get_current_trace().update(metadata={"country_queried": country})
try:
# PydanticAI agent calls will now generate OpenTelemetry spans
# that LangWatch captures under the "PydanticAI - City Capital Query" trace.
result = await agent.run(f'What is the capital of {country}?')
return result.output
except Exception as e:
if current_trace:
current_trace.record_exception(e)
current_trace.set_status("error", str(e))
raise
async def main():
try:
capital = await get_capital_city("France")
print(f"The capital of France is: {capital}")
except Exception as e:
print(f"Error getting capital of France: {e}")
try:
capital_error = await get_capital_city("NonExistentCountry") # Example to show error tracing
print(f"Query for NonExistentCountry returned: {capital_error}")
except Exception as e:
print(f"Correctly caught error for NonExistentCountry: {e}")
if __name__ == "__main__":
asyncio.run(main())
```
**Key points for this approach:**
- `langwatch.setup()`: Essential for initializing the OpenTelemetry environment that LangWatch uses.
- `Agent(instrument=True)` or `Agent.instrument_all()`: Activates PydanticAI's OpenTelemetry signal emission.
- `InstrumentationSettings(event_mode='logs')`: As per PydanticAI documentation, using `event_mode='logs'` aligns message capture with OpenTelemetry Semantic Conventions for Generative AI, which might be better for generic OTel collectors. The default mode (`json_array`) uses a custom attribute format for events.
- `@langwatch.trace()`: Creates a parent trace in LangWatch, under which PydanticAI's operation spans will be nested.
By following these steps, you can effectively monitor your PydanticAI applications using LangWatch, gaining insights into agent behavior, model performance, and overall application flow.
---
# FILE: ./integration/python/integrations/langgraph.mdx
---
title: LangGraph Instrumentation
sidebarTitle: LangGraph
description: Learn how to instrument LangGraph applications with the LangWatch Python SDK.
keywords: langgraph, instrumentation, callback, opentelemetry, langwatch, python, tracing, openinference, openllmetry
---
LangGraph is a powerful framework for building LLM applications. LangWatch integrates with LangGraph to provide detailed observability into your chains, agents, LLM calls, and tool usage.
The community instrumentors below are for LangChain, but LangGraph is compatible with them.
## 2. Using Community OpenTelemetry Instrumentors
Dedicated LangChain instrumentors from libraries like OpenInference and OpenLLMetry can also be used to capture LangGraph operations as OpenTelemetry traces, which LangWatch can then ingest.
### Instrumenting LangGraph with Dedicated Instrumentors
#### i. Via `langwatch.setup()`
```python OpenInference
# OpenInference Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from openinference.instrumentation.langchain import LangchainInstrumentor
import os
import asyncio
langwatch.setup(
instrumentors=[LangchainInstrumentor()] # Add OpenInference LangchainInstrumentor
)
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenInference via Setup")
async def handle_message_oinference_setup(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_oinference_setup():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenInference Langchain (setup) example.")
return
response = await handle_message_oinference_setup("Explain Langchain instrumentation with OpenInference.")
print(f"AI (OInference Setup): {response}")
if __name__ == "__main__":
asyncio.run(main_community_oinference_setup())
```
```python OpenLLMetry
# OpenLLMetry Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from opentelemetry.instrumentation.langchain import LangchainInstrumentor # Corrected import
import os
import asyncio
langwatch.setup(
instrumentors=[LangchainInstrumentor()] # Add OpenLLMetry LangchainInstrumentor
)
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenLLMetry via Setup")
async def handle_message_openllmetry_setup(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_openllmetry_setup():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenLLMetry Langchain (setup) example.")
return
response = await handle_message_openllmetry_setup("Explain Langchain instrumentation with OpenLLMetry.")
print(f"AI (OpenLLMetry Setup): {response}")
if __name__ == "__main__":
asyncio.run(main_community_openllmetry_setup())
```
#### ii. Direct Instrumentation
```python OpenInference
# OpenInference Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from openinference.instrumentation.langchain import LangchainInstrumentor
import os
import asyncio
langwatch.setup()
LangchainInstrumentor().instrument() # Instrument Langchain directly
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are very brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenInference Direct")
async def handle_message_oinference_direct(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_oinference_direct():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenInference Langchain (direct) example.")
return
response = await handle_message_oinference_direct("How does direct Langchain instrumentation work?")
print(f"AI (OInference Direct): {response}")
if __name__ == "__main__":
asyncio.run(main_community_oinference_direct())
```
```python OpenLLMetry
# OpenLLMetry Example
import langwatch
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from opentelemetry.instrumentation.langchain import LangchainInstrumentor # Corrected import
import os
import asyncio
langwatch.setup()
LangchainInstrumentor().instrument() # Instrument Langchain directly
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are very brief."), ("human", "{question}")]
)
runnable = prompt | model | StrOutputParser()
@langwatch.trace(name="Langchain - OpenLLMetry Direct")
async def handle_message_openllmetry_direct(user_question: str):
response = await runnable.ainvoke({"question": user_question})
return response
async def main_community_openllmetry_direct():
if not os.getenv("OPENAI_API_KEY"):
print("OPENAI_API_KEY not set. Skipping OpenLLMetry Langchain (direct) example.")
return
response = await handle_message_openllmetry_direct("How does direct Langchain instrumentation work with OpenLLMetry?")
print(f"AI (OpenLLMetry Direct): {response}")
if __name__ == "__main__":
asyncio.run(main_community_openllmetry_direct())
```
---
# FILE: ./integration/python/integrations/open-ai-agents.mdx
---
title: OpenAI Agents SDK Instrumentation
sidebarTitle: OpenAI Agents
description: Learn how to instrument OpenAI Agents with the LangWatch Python SDK
keywords: openai-agents, instrumentation, openinference, LangWatch, Python, tracing
---
LangWatch allows you to monitor your OpenAI Agents by integrating with their tracing capabilities. Since OpenAI Agents manage their own execution flow, including LLM calls and tool usage, the direct `autotrack_openai_calls()` method used for the standard OpenAI client is not applicable here.
Instead, you can integrate LangWatch in one of two ways:
1. **Using OpenInference Instrumentation (Recommended)**: Leverage the `openinference-instrumentation-openai-agents` library, which provides OpenTelemetry-based instrumentation for OpenAI Agents. This is generally the simplest and most straightforward method.
2. **Alternative: Using OpenAI Agents' Built-in Tracing with a Custom Processor**: If you choose not to use OpenInference or have highly specific requirements, you can adapt the built-in tracing mechanism of the `openai-agents` SDK to forward trace data to LangWatch by implementing your own custom `TracingProcessor`.
This guide will walk you through both methods.
## 1. Using OpenInference Instrumentation for OpenAI Agents (Recommended)
The most straightforward way to integrate LangWatch with OpenAI Agents is by using the OpenInference instrumentation library specifically designed for it: `openinference-instrumentation-openai-agents`. This library is currently in an Alpha stage, so while ready for experimentation, it may undergo breaking changes.
This approach uses OpenTelemetry-based instrumentation and is generally recommended for ease of setup.
### Installation
First, ensure you have the necessary packages installed:
```bash
pip install langwatch openai-agents openinference-instrumentation-openai-agents
```
### Integration via `langwatch.setup()`
You can pass an instance of the `OpenAIAgentsInstrumentor` from `openinference-instrumentation-openai-agents` to the `instrumentors` list in the `langwatch.setup()` call. LangWatch will then manage the lifecycle of this instrumentor.
```python
import langwatch
from openai_agents.agents import Agent, Runner # Using openai_agents SDK
from openinference.instrumentation.openai_agents import OpenAIAgentsInstrumentor
import os
import asyncio
# Ensure LANGWATCH_API_KEY is set in your environment, or pass it to setup
# e.g., langwatch.setup(api_key="your_api_key", instrumentors=[OpenAIAgentsInstrumentor()])
# If LANGWATCH_API_KEY is in env, this is sufficient:
langwatch.setup(
instrumentors=[OpenAIAgentsInstrumentor()]
)
# Initialize your agent
agent = Agent(name="ExampleAgent", instructions="You are a helpful assistant.")
@langwatch.trace(name="OpenAI Agent Run with OpenInference")
async def run_agent_with_openinference(prompt: str):
# The OpenAIAgentsInstrumentor will automatically capture agent activities.
result = await Runner.run(agent, prompt)
return result.final_output
async def main():
user_query = "Tell me a fun fact."
response = await run_agent_with_openinference(user_query)
print(f"User: {user_query}")
print(f"AI: {response}")
if __name__ == "__main__":
asyncio.run(main())
```
The `OpenAIAgentsInstrumentor` is part of the `openinference-instrumentation-openai-agents` package. Always refer to its [official documentation](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-openai-agents) for the latest updates, especially as it's in Alpha.
### Direct Instrumentation
Alternatively, if you manage your OpenTelemetry `TracerProvider` more directly (e.g., if LangWatch is configured to use an existing global provider), you can use the instrumentor's `instrument()` method. LangWatch will pick up the spans if its exporter is part of the active `TracerProvider`.
```python
import langwatch
from openai_agents.agents import Agent, Runner
from openinference.instrumentation.openai_agents import OpenAIAgentsInstrumentor
import os
import asyncio
# Initialize LangWatch (it will set up its OTel exporter)
langwatch.setup() # Ensure API key is available via env or parameters
# Instrument OpenAI Agents directly
OpenAIAgentsInstrumentor().instrument()
agent = Agent(name="ExampleAgentDirect", instructions="You are a helpful assistant.")
@langwatch.trace(name="OpenAI Agent Run with Direct OpenInference")
async def run_agent_direct_instrumentation(prompt: str):
result = await Runner.run(agent, prompt)
return result.final_output
async def main():
user_query = "What's the weather like?"
response = await run_agent_direct_instrumentation(user_query)
print(f"User: {user_query}")
print(f"AI: {response}")
if __name__ == "__main__":
asyncio.run(main())
```
Key points for OpenInference instrumentation:
- It patches `openai-agents` activities globally once instrumented.
- Ensure `langwatch.setup()` is called so LangWatch's OpenTelemetry exporter is active and configured.
- The `@langwatch.trace()` decorator on your calling function helps create a parent span under which the agent's detailed operations will be nested.
## 2. Alternative: Using OpenAI Agents' Built-in Tracing with a Custom Processor
If you prefer not to use the OpenInference instrumentor, or if you have highly specific tracing requirements not met by it, you can leverage the `openai-agents` SDK's own [built-in tracing system](https://openai.github.io/openai-agents-python/tracing/).
This involves creating a custom `TracingProcessor` that intercepts trace data from the `openai-agents` SDK and then uses the standard OpenTelemetry Python API to create OpenTelemetry spans. LangWatch will then ingest these OpenTelemetry spans, provided `langwatch.setup()` has been called.
**Conceptual Outline for Your Custom Processor:**
1. **Initialize LangWatch**: Ensure `langwatch.setup()` is called in your application. This sets up LangWatch to receive OpenTelemetry data.
2. **Implement Your Custom `TracingProcessor`**:
- Following the `openai-agents` SDK documentation, create a class that implements their `TracingProcessor` interface (see their docs on [Custom Tracing Processors](https://openai.github.io/openai-agents-python/tracing/#custom-tracing-processors) and the API reference for [`TracingProcessor`](https://openai.github.io/openai-agents-python/api_reference/tracing/processor_interface/)).
- In your processor's methods (e.g., `on_span_start`, `on_span_end`), you will receive `Trace` and `Span` objects from the `openai-agents` SDK.
- You will then use the `opentelemetry-api` and `opentelemetry-sdk` (e.g., `opentelemetry.trace.get_tracer(__name__).start_span()`) to translate this information into OpenTelemetry spans, including their names, attributes, timings, and status. Consult the `openai-agents` documentation on [Traces and spans](https://openai.github.io/openai-agents-python/tracing/#traces-and-spans) for details on their data structures.
3. **Register Your Custom Processor**: Use `openai_agents.tracing.add_trace_processor(your_custom_processor)` or `openai_agents.tracing.set_trace_processors([your_custom_processor])` as per the `openai-agents` SDK documentation.
**Implementation Guidance:**
LangWatch does not provide a pre-built custom `TracingProcessor` for this purpose. The implementation of such a processor is your responsibility and should be based on the official `openai-agents` SDK documentation. This ensures your processor correctly interprets the agent's trace data and remains compatible with `openai-agents` SDK updates.
- Key `openai-agents` documentation:
- [OpenAI Agents Tracing Documentation](https://openai.github.io/openai-agents-python/tracing/)
- [API Reference for Tracing](https://openai.github.io/openai-agents-python/api_reference/tracing/)
Implementing a custom `TracingProcessor` is an advanced task that requires:
- A thorough understanding of both the `openai-agents` tracing internals and OpenTelemetry concepts and semantic conventions.
- Careful mapping of `openai-agents` `SpanData` types to OpenTelemetry attributes.
- Robust handling of span parenting, context propagation, and error states.
- Diligent maintenance to keep your processor aligned with any changes in the `openai-agents` SDK.
This approach offers maximum flexibility but comes with significant development and maintenance overhead.
## Which Approach to Choose?
- **OpenInference Instrumentation (Recommended)**:
- **Pros**: Significantly simpler to set up and maintain. Relies on a community-supported library (`openinference-instrumentation-openai-agents`) designed for OpenTelemetry integration. Aligns well with standard OpenTelemetry practices.
- **Cons**: As the `openinference-instrumentation-openai-agents` library is in Alpha, it may have breaking changes. You have less direct control over the exact span data compared to a fully custom processor.
- **Custom `TracingProcessor` (Alternative for advanced needs)**:
- **Pros**: Offers complete control over the transformation of trace data from `openai-agents` to OpenTelemetry. Allows for highly customized span data and behaviors.
- **Cons**: Far more complex to implement correctly and maintain. Requires deep expertise in both `openai-agents` tracing and OpenTelemetry. You are responsible for adapting your processor to any changes in the `openai-agents` SDK.
For most users, the **OpenInference instrumentation is the recommended path** due to its simplicity and lower maintenance burden.
The **custom `TracingProcessor`** approach should generally be reserved for situations where the OpenInference instrumentor is unsuitable, or when you have highly specialized tracing requirements that demand direct manipulation of the agent's trace data before converting it to OpenTelemetry spans.
---
Always refer to the latest documentation for `langwatch`, `openai-agents`, and `openinference-instrumentation-openai-agents` for the most up-to-date instructions and API details.
---
# FILE: ./integration/python/integrations/lite-llm.mdx
---
title: LiteLLM Instrumentation
sidebarTitle: LiteLLM
description: Learn how to instrument LiteLLM calls with the LangWatch Python SDK.
keywords: litellm, instrumentation, autotrack, opentelemetry, langwatch, python, tracing, openinference, openllmetry
---
LiteLLM provides a unified interface to various Large Language Models. LangWatch integrates with LiteLLM by capturing OpenTelemetry traces, enabling detailed observability into your LLM calls made through LiteLLM.
This guide outlines three primary approaches for instrumenting LiteLLM with LangWatch:
1. **Using `autotrack_litellm_calls()`**: This method, part of the LangWatch SDK, dynamically patches your LiteLLM module instance for the current trace to capture its calls.
2. **Using LiteLLM's Native OpenTelemetry Tracing with Global Setup**: LiteLLM can automatically generate OpenTelemetry traces for its operations when a global OpenTelemetry environment (established by `langwatch.setup()`) is active.
3. **Using Community OpenTelemetry Instrumentors (for Underlying SDKs)**: If LiteLLM internally uses other instrumented SDKs (like the `openai` SDK for OpenAI models), you can leverage community instrumentors for those specific underlying SDKs.
## Using `autotrack_litellm_calls()`
The `autotrack_litellm_calls()` function, called on a trace object, provides a straightforward way to capture all LiteLLM calls for the duration of the current trace. This is often the most direct way to ensure LiteLLM operations are captured by LangWatch within a specific traced function.
You typically call this method on the trace object obtained via `langwatch.get_current_trace()` inside a function decorated with `@langwatch.trace()`.
```python
import langwatch
import litellm
import os
import asyncio
from typing import cast
from litellm import CustomStreamWrapper # For streaming example
from litellm.types.utils import StreamingChoices # For streaming example
langwatch.setup()
@langwatch.trace(name="LiteLLM Autotrack Example")
async def get_litellm_response_autotrack(user_prompt: str):
# Get the current trace and enable autotracking for the litellm module
langwatch.get_current_trace().autotrack_litellm_calls(litellm) # Pass the imported litellm module
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": user_prompt}
]
# This call will now be automatically captured as a span by LangWatch
response = await litellm.acompletion(
model="groq/llama3-70b-8192",
messages=messages,
)
return response.choices[0].message.content
async def main():
reply = await get_litellm_response_autotrack("Tell me about LiteLLM.")
print("AI Response:", reply)
if __name__ == "__main__":
asyncio.run(main())
```
**Key points for `autotrack_litellm_calls()`:**
- It must be called on an active trace object (e.g., obtained via `langwatch.get_current_trace()`).
- It instruments the passed `litellm` module instance specifically for the current trace.
## Using Community OpenTelemetry Instrumentors
If you are already using a dedicated community instrumentor for LiteLLM, such as the one provided by OpenInference, you can pass an instance of `LiteLLMInstrumentor` from `openinference.instrumentation.litellm` to the `instrumentors` list in `langwatch.setup()`.
### 1. Via `langwatch.setup()`
You can pass an instance of `LiteLLMInstrumentor` from `openinference.instrumentation.litellm` to the `instrumentors` list in `langwatch.setup()`.
```python
import langwatch
import litellm
import os
import asyncio
# Example using OpenInference's LiteLLMInstrumentor
from openinference.instrumentation.litellm import LiteLLMInstrumentor
# 1. Initialize LangWatch with the LiteLLMInstrumentor
langwatch.setup(
instrumentors=[LiteLLMInstrumentor()] # Instruments LiteLLM directly
)
@langwatch.trace(name="LiteLLM via OpenInference Instrumentor Setup")
async def get_response_via_litellm_instrumentor_setup(user_prompt: str):
messages = [
{"role": "user", "content": user_prompt}
]
# This LiteLLM call will be captured by the LiteLLMInstrumentor
response = await litellm.acompletion(
model="groq/llama3-70b-8192", # Example model
messages=messages
)
return response.choices[0].message.content
async def main_community_litellm_instrumentor_setup():
reply = await get_response_via_litellm_instrumentor_setup("Explain OpenInference for LiteLLM.")
print(f"AI Response (OpenInference via setup): {reply}")
if __name__ == "__main__":
asyncio.run(main_community_litellm_instrumentor_setup())
```
Ensure you have the `openinference-instrumentation-litellm` library installed.
### 2. Direct Instrumentation with `LiteLLMInstrumentor`
If you are managing your OpenTelemetry setup more directly, you can call `instrument()` on an instance of `LiteLLMInstrumentor`.
```python
import langwatch
import litellm
import os
import asyncio
from openinference.instrumentation.litellm import LiteLLMInstrumentor
# 1. Initialize LangWatch (sets up global OTel provider for LangWatch exporter)
langwatch.setup()
# 2. Instrument LiteLLM directly using its OpenInference instrumentor
# This should be done once, early in your application lifecycle.
LiteLLMInstrumentor().instrument()
@langwatch.trace(name="LiteLLM via Directly Instrumented OpenInference")
async def get_response_direct_litellm_instrumentation(user_prompt: str):
messages = [
{"role": "user", "content": user_prompt}
]
response = await litellm.acompletion(model="groq/llama3-70b-8192", messages=messages)
return response.choices[0].message.content
async def main_direct_litellm_instrumentation():
reply = await get_response_direct_litellm_instrumentation("How does direct OTel instrumentation work for LiteLLM?")
print(f"AI Response (direct OpenInference): {reply}")
if __name__ == "__main__":
asyncio.run(main_direct_litellm_instrumentation())
```
**Key points for using OpenInference `LiteLLMInstrumentor`:**
- This instrumentor specifically targets LiteLLM calls.
- It provides an alternative to `autotrack_litellm_calls` if you prefer an explicit instrumentor pattern or are using OpenInference across your stack.
### Which Approach to Choose?
- **`autotrack_litellm_calls()`**: Best for explicit, trace-specific instrumentation of LiteLLM. Offers clear control over when LiteLLM calls are tracked by LangWatch within a given trace.
- **OpenInference `LiteLLMInstrumentor`**: Use if you are standardizing on OpenInference instrumentors or prefer this explicit way of instrumenting LiteLLM itself (rather than its underlying SDKs). It provides traces directly from LiteLLM's perspective.
Choose the method that best fits your instrumentation strategy and the level of detail required.
---
# FILE: ./integration/python/integrations/other.mdx
---
title: Other OpenTelemetry Instrumentors
sidebarTitle: Other
description: Learn how to use any OpenTelemetry-compatible instrumentor with LangWatch.
keywords: opentelemetry, instrumentation, custom, other, generic, BaseInstrumentor, LangWatch, Python
---
LangWatch is designed to be compatible with the broader OpenTelemetry ecosystem. Beyond the specifically documented integrations, you can use LangWatch with any Python library that has an OpenTelemetry instrumentor, provided that the instrumentor adheres to the standard OpenTelemetry Python `BaseInstrumentor` interface.
## Using Custom/Third-Party OpenTelemetry Instrumentors
If you have a specific library you want to trace, and there's an OpenTelemetry instrumentor available for it (either a community-provided one not yet listed in our specific integrations, or one you've developed yourself), you can integrate it with LangWatch.
The key is that the instrumentor should be an instance of a class that inherits from `opentelemetry.instrumentation.instrumentor.BaseInstrumentor`. You can find the official documentation for this base class here:
- [OpenTelemetry BaseInstrumentor Documentation](https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/base/instrumentor.html#opentelemetry.instrumentation.instrumentor.BaseInstrumentor)
### Integration via `langwatch.setup()`
To use such an instrumentor, you simply pass an instance of it to the `instrumentors` list in the `langwatch.setup()` call. LangWatch will then manage its lifecycle (calling its `instrument()` and `uninstrument()` methods appropriately).
Here's a conceptual example using the OpenTelemetry `LoggingInstrumentor`:
```python
import langwatch
import os
import logging # Standard Python logging
# Import an off-the-shelf OpenTelemetry instrumentor
# Ensure you have this package installed: pip install opentelemetry-instrumentation-logging
from opentelemetry.instrumentation.logging import LoggingInstrumentor
# Ensure LANGWATCH_API_KEY is set in your environment, or set it in `setup`
langwatch.setup(
instrumentors=[
LoggingInstrumentor() # Pass an instance of the instrumentor
]
)
# Configure standard Python logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# You might want to add a handler if you also want to see logs in the console
# handler = logging.StreamHandler()
# formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
# handler.setFormatter(formatter)
# logger.addHandler(handler)
@langwatch.trace(name="Task with Instrumented Logging")
def perform_task_with_logging():
logger.info("Starting the task.")
# ... some work ...
logger.warning("Something to be aware of happened during the task.")
# ... more work ...
logger.info("Task completed.")
return "Task finished successfully"
if __name__ == "__main__":
print("Running example with LoggingInstrumentor...")
result = perform_task_with_logging()
print(f"Result: {result}")
# Spans for the log messages (e.g., logger.info, logger.warning)
# would be generated by LoggingInstrumentor and captured by LangWatch.
```
When this code runs, the `LoggingInstrumentor` (managed by `langwatch.setup()`) will automatically create OpenTelemetry spans for any log messages emitted by the standard Python `logging` module. LangWatch will then capture these spans.
## Discovering More Community Instrumentors
Many Python libraries, especially in the AI/ML space, are instrumented by community-driven OpenTelemetry projects. If you're looking for pre-built instrumentors, these are excellent places to start:
* **OpenInference (by Arize AI):** [https://github.com/Arize-ai/openinference](https://github.com/Arize-ai/openinference)
* This project provides instrumentors for a wide range of AI/ML libraries and frameworks. Examples include:
* OpenAI
* Anthropic
* LiteLLM
* Haystack
* LlamaIndex
* LangChain
* Groq
* Google Gemini
* And more (check their repository for the full list).
* **OpenLLMetry (by Traceloop):** [https://github.com/traceloop/openllmetry](https://github.com/traceloop/openllmetry)
* This project also offers a comprehensive suite of instrumentors for LLM applications and related tools. Examples include:
* OpenAI
* CrewAI
* Haystack
* LangChain
* LlamaIndex
* Pinecone
* ChromaDB
* And more (explore their repository for details).
You can browse these repositories to find instrumentors for other libraries you might be using. If an instrumentor from these projects (or any other source) adheres to the `BaseInstrumentor` interface, you can integrate it with LangWatch using the `langwatch.setup(instrumentors=[...])` method described above.
### Key Considerations:
1. **`BaseInstrumentor` Compliance:** Ensure the instrumentor correctly implements the `BaseInstrumentor` interface, particularly the `instrument()` and `uninstrument()` methods, and `instrumentation_dependencies()`.
2. **Installation:** You'll need to have the custom instrumentor package installed in your Python environment, along with the library it instruments.
3. **TracerProvider:** LangWatch configures an OpenTelemetry `TracerProvider`. The instrumentor, when activated by LangWatch, will use this provider to create spans. If you are managing your OpenTelemetry setup more directly (e.g., providing your own `TracerProvider` to `langwatch.setup()`), the instrumentor will use that instead.
4. **Data Quality:** The quality and detail of the telemetry data captured will depend on how well the custom instrumentor is written.
By leveraging the `BaseInstrumentor` interface, LangWatch remains flexible and extensible, allowing you to bring telemetry from a wide array of Python libraries into your observability dashboard.
---
# FILE: ./integration/python/integrations/azure-ai.mdx
---
title: Azure AI Inference SDK Instrumentation
sidebarTitle: Python
description: Learn how to instrument the Azure AI Inference Python SDK with LangWatch.
keywords: azure ai inference, python, sdk, instrumentation, opentelemetry, langwatch, tracing
---
The `azure-ai-inference` Python SDK provides a unified way to interact with various AI models deployed on Azure, including those on Azure OpenAI Service, GitHub Models, and Azure AI Foundry Serverless/Managed Compute endpoints. For more details on the SDK, refer to the [official Azure AI Inference client library documentation](https://learn.microsoft.com/en-us/python/api/overview/azure/ai-inference-readme?view=azure-python-preview).
LangWatch can capture traces generated by the `azure-ai-inference` SDK by leveraging its built-in OpenTelemetry support. This guide will show you how to set it up.
## Prerequisites
1. **Install LangWatch SDK**:
```bash
pip install langwatch
```
2. **Install Azure AI Inference SDK with OpenTelemetry support**:
The `azure-ai-inference` SDK can be installed with OpenTelemetry capabilities. You might also need the core Azure OpenTelemetry tracing package.
```bash
pip install azure-ai-inference[opentelemetry] azure-core-tracing-opentelemetry
```
Refer to the [Azure SDK documentation](https://learn.microsoft.com/en-us/python/api/overview/azure/ai-inference-readme?view=azure-python-preview#install-the-package) for the most up-to-date installation instructions.
## Instrumentation with `AIInferenceInstrumentor`
The `azure-ai-inference` SDK provides an `AIInferenceInstrumentor` that automatically captures traces for its operations when enabled. LangWatch, when set up, will include an OpenTelemetry exporter that can collect these traces.
Here's how to instrument your application:
```python
import langwatch
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.tracing import AIInferenceInstrumentor
from azure.core.credentials import AzureKeyCredential
import os
import asyncio
# 1. Initialize LangWatch
langwatch.setup(
instrumentors=[AIInferenceInstrumentor()]
)
# 2. Configure your Azure AI Inference client
azure_openai_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
azure_openai_api_key = os.getenv("AZURE_OPENAI_API_KEY")
azure_openai_api_version = "2024-06-01"
chat_client = ChatCompletionsClient(
endpoint=azure_openai_endpoint,
credential=AzureKeyCredential(azure_openai_api_key),
api_version=azure_openai_api_version
)
@langwatch.trace(name="Azure AI Inference Chat")
async def get_ai_response(prompt: str):
# This call will now be automatically traced by the AIInferenceInstrumentor and
# captured by LangWatch as a span within the "Azure AI Inference Chat" trace.
response = await chat_client.complete(
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
async def main():
user_prompt = "What is the Azure AI Inference SDK?"
try:
ai_reply = await get_ai_response(user_prompt)
print(f"User: {user_prompt}")
print(f"AI: {ai_reply}")
except Exception as e:
print(f"An error occurred: {e}")
if __name__ == "__main__":
asyncio.run(main())
```
The example uses the synchronous `ChatCompletionsClient` for simplicity in demonstrating instrumentation. The `azure-ai-inference` SDK also provides asynchronous clients under the `azure.ai.inference.aio` namespace (e.g., `azure.ai.inference.aio.ChatCompletionsClient`). If you are using `async/await` in your application, you should use these asynchronous clients. The `AIInferenceInstrumentor` will work with both synchronous and asynchronous clients.
## How it Works
1. `langwatch.setup()`: Initializes the LangWatch SDK, which includes setting up an OpenTelemetry trace exporter. This exporter is ready to receive spans from any OpenTelemetry-instrumented library in your application.
2. `AIInferenceInstrumentor().instrument()`: This command, provided by the `azure-ai-inference` SDK, patches the relevant Azure AI clients (like `ChatCompletionsClient` or `EmbeddingsClient`) to automatically create OpenTelemetry spans for their operations (e.g., a `complete` or `embed` call).
3. `@langwatch.trace()`: By decorating your own functions (like `get_ai_response` in the example), you create a parent trace in LangWatch. The spans automatically generated by the `AIInferenceInstrumentor` for calls made within this decorated function will then be nested under this parent trace. This provides a full end-to-end view of your operation.
With this setup, calls made using the `azure-ai-inference` clients will be automatically traced and sent to LangWatch, providing visibility into the performance and behavior of your AI model interactions.
---
# FILE: ./integration/python/integrations/open-ai.mdx
---
title: OpenAI Instrumentation
sidebarTitle: Python
description: Learn how to instrument OpenAI API calls with the LangWatch Python SDK
keywords: openai, instrumentation, autotrack, openinference, openllmetry, LangWatch, Python
---
LangWatch offers robust integration with OpenAI, allowing you to capture detailed information about your LLM calls automatically. There are two primary approaches to instrumenting your OpenAI interactions:
1. **Using `autotrack_openai_calls()`**: This method, part of the LangWatch SDK, dynamically patches your OpenAI client instance to capture calls made through it within a specific trace.
2. **Using Community OpenTelemetry Instrumentors**: Leverage existing OpenTelemetry instrumentation libraries like those from OpenInference or OpenLLMetry. These can be integrated with LangWatch by either passing them to the `langwatch.setup()` function or by using their native `instrument()` methods if you're managing your OpenTelemetry setup more directly.
This guide will walk you through both methods.
## Using `autotrack_openai_calls()`
The `autotrack_openai_calls()` function provides a straightforward way to capture all OpenAI calls made with a specific client instance for the duration of the current trace.
You typically call this method on the trace object obtained via `langwatch.get_current_trace()` inside a function decorated with `@langwatch.trace()`.
```python
import langwatch
from openai import OpenAI
# Ensure LANGWATCH_API_KEY is set in your environment, or set it in `setup`
langwatch.setup()
# Initialize your OpenAI client
client = OpenAI()
@langwatch.trace(name="OpenAI Chat Completion")
async def get_openai_chat_response(user_prompt: str):
# Get the current trace and enable autotracking for the 'client' instance
langwatch.get_current_trace().autotrack_openai_calls(client)
# All calls made with 'client' will now be automatically captured as spans
response = client.chat.completions.create(
model="gpt-4.1-nano",
messages=[{"role": "user", "content": user_prompt}],
)
completion = response.choices[0].message.content
return completion
async def main():
user_query = "Tell me a joke about Python programming."
response = await get_openai_chat_response(user_query)
print(f"User: {user_query}")
print(f"AI: {response}")
if __name__ == "__main__":
import asyncio
asyncio.run(main())
```
Key points for `autotrack_openai_calls()`:
- It must be called on an active trace object (e.g., obtained via `langwatch.get_current_trace()`).
- It instruments a *specific instance* of the OpenAI client. If you have multiple clients, you'll need to call it for each one you want to track.
## Using Community OpenTelemetry Instrumentors
If you prefer to use broader OpenTelemetry-based instrumentation, or are already using libraries like `OpenInference` or `OpenLLMetry`, LangWatch can seamlessly integrate with them. These libraries provide instrumentors that automatically capture data from various LLM providers, including OpenAI.
There are two main ways to integrate these:
### 1. Via `langwatch.setup()`
You can pass an instance of the instrumentor (e.g., `OpenAIInstrumentor` from OpenInference or OpenLLMetry) to the `instrumentors` list in the `langwatch.setup()` call. LangWatch will then manage the lifecycle of this instrumentor.
```python
import langwatch
from openai import OpenAI
import os
# Example using OpenInference's OpenAIInstrumentor
from openinference.instrumentation.openai import OpenAIInstrumentor
# Initialize LangWatch with the OpenAIInstrumentor
langwatch.setup(
instrumentors=[OpenAIInstrumentor()]
)
client = OpenAI()
@langwatch.trace(name="OpenAI Call with Community Instrumentor")
def generate_text_with_community_instrumentor(prompt: str):
# No need to call autotrack explicitly, the community instrumentor handles OpenAI calls globally.
response = client.chat.completions.create(
model="gpt-4.1-nano",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
if __name__ == "__main__":
user_query = "Tell me a joke about Python programming."
response = generate_text_with_community_instrumentor(user_query)
print(f"User: {user_query}")
print(f"AI: {response}")
```
Ensure you have the respective community instrumentation library installed (e.g., `pip install openllmetry-instrumentation-openai` or `pip install openinference-instrumentation-openai`).
### 2. Direct Instrumentation
If you have an existing OpenTelemetry `TracerProvider` configured in your application (or if LangWatch is configured to use the global provider), you can use the community instrumentor's `instrument()` method directly. LangWatch will automatically pick up the spans generated by these instrumentors as long as its exporter is part of the active `TracerProvider`.
```python
import langwatch
from openai import OpenAI
import os
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor
langwatch.setup()
client = OpenAI()
# Instrument OpenAI directly using the community library
OpenAIInstrumentor().instrument()
@langwatch.trace(name="OpenAI Call with Direct Community Instrumentation")
def get_story_ending(beginning: str):
response = client.chat.completions.create(
model="gpt-4.1-nano",
messages=[
{"role": "system", "content": "You are a creative writer. Complete the story."},
{"role": "user", "content": beginning}
]
)
return response.choices[0].message.content
if __name__ == "__main__":
story_start = "In a land of dragons and wizards, a young apprentice found a mysterious map..."
ending = get_story_ending(story_start)
print(f"Story Start: {story_start}")
print(f"AI's Ending: {ending}")
```
### Key points for community instrumentors:
- These instrumentors often patch OpenAI at a global level, meaning all OpenAI calls from any client instance will be captured once instrumented.
- If using `langwatch.setup(instrumentors=[...])`, LangWatch handles the setup.
- If instrumenting directly (e.g., `OpenAIInstrumentor().instrument()`), ensure that the `TracerProvider` used by the instrumentor is the same one LangWatch is exporting from. This usually means LangWatch is configured to use an existing global provider or one you explicitly pass to `langwatch.setup()`.
### Which Approach to Choose?
- **`autotrack_openai_calls()`** is ideal for targeted instrumentation within specific traces or when you want fine-grained control over which OpenAI client instances are tracked. It's simpler if you're not deeply invested in a separate OpenTelemetry setup.
- **Community Instrumentors** are powerful if you're already using OpenTelemetry, want to capture OpenAI calls globally across your application, or need to instrument other libraries alongside OpenAI with a consistent OpenTelemetry approach. They provide a more holistic observability solution if you have multiple OpenTelemetry-instrumented components.
Choose the method that best fits your existing setup and instrumentation needs. Both approaches effectively send OpenAI call data to LangWatch for monitoring and analysis.
---
# FILE: ./integration/python/tutorials/capturing-metadata.mdx
---
title: Capturing Metadata and Attributes
sidebarTitle: Metadata & Attributes
description: Learn how to enrich your traces and spans with custom metadata and attributes using the LangWatch Python SDK.
---
Metadata and attributes are key-value pairs that allow you to add custom contextual information to your traces and spans. This enrichment is invaluable for debugging, analysis, filtering, and gaining deeper insights into your LLM application's behavior.
LangWatch distinguishes between two main types of custom data:
* **Trace Metadata**: Information that applies to the entire lifecycle of a request or a complete operation.
* **Span Attributes**: Information specific to a particular unit of work or step within a trace.
This tutorial will guide you through capturing both types using the Python SDK.
## Trace Metadata
Trace metadata provides context for the entire trace. It's ideal for information that remains constant throughout the execution of a traced operation, such as:
* User identifiers (`user_id`)
* Session or conversation identifiers (`session_id`, `thread_id`)
* Application version (`app_version`)
* Environment (`env: "production"`)
* A/B testing flags or variant names
You can set trace metadata when a trace is initiated or update it at any point while the trace is active.
### Setting Trace Metadata at Initialization
The easiest way to add metadata to a trace is by passing a `metadata` dictionary to the `langwatch.trace()` decorator or context manager.
```python
import langwatch
import os
# Initialize LangWatch (ensure this is done once in your application)
langwatch.setup(api_key=os.getenv("LANGWATCH_API_KEY"))
@langwatch.trace(name="UserQueryHandler", metadata={"user_id": "user_123", "session_id": "session_abc"})
def handle_user_query(query: str):
# Your application logic here
# For example, process the query and interact with an LLM
processed_query = f"Query processed: {query}"
# You can also update trace metadata from within the trace
current_trace = langwatch.get_current_trace()
if current_trace:
current_trace.update(metadata={"query_language": "en"})
return processed_query
handle_user_query("Hello, LangWatch!")
```
In this example, `user_id` and `session_id` are attached to the "UserQueryHandler" trace from the start. Later, `query_language` is added.
Refer to the [`langwatch.trace()` API reference](/integration/python/reference#langwatchtrace) for more details on its parameters.
### Updating Trace Metadata Dynamically
If you need to add or modify trace metadata after the trace has started (e.g., based on some intermediate result), you can use the `update()` method on the `LangWatchTrace` object.
You can get the current trace object using `langwatch.get_current_trace()` or from the `langwatch.trace()` context manager.
```python
import langwatch
import os
import uuid
langwatch.setup(api_key=os.getenv("LANGWATCH_API_KEY"))
def process_customer_request(customer_id: str, request_details: dict):
trace_metadata = {
"customer_id": customer_id,
"initial_request_type": request_details.get("type")
}
with langwatch.trace(name="CustomerRequestFlow", metadata=trace_metadata) as current_trace:
# Simulate some processing
print(f"Processing request for customer {customer_id}")
# Update metadata based on some condition or result
is_priority_customer = customer_id.startswith("vip_")
current_trace.update(metadata={"priority_customer": is_priority_customer})
# ... further processing ...
if request_details.get("type") == "complaint":
current_trace.update(metadata={"escalation_needed": True})
print(f"Trace ID: {current_trace.id}") # Example of accessing trace properties
process_customer_request(f"vip_{uuid.uuid4()}", {"type": "complaint", "content": "Service issue"})
process_customer_request(f"user_{uuid.uuid4()}", {"type": "inquiry", "content": "Product question"})
```
## Span Attributes
Span attributes (often referred to simply as "attributes") provide context for a specific operation or unit of work *within* a trace. They are useful for details that are relevant only to that particular step. Examples include:
* For an LLM call span: `model_name`, `prompt_template_version`, `temperature`
* For a tool call span: `tool_name`, `api_endpoint`, specific input parameters
* For a RAG span: `retrieved_document_ids`, `chunk_count`
* Custom business logic flags or intermediate results specific to that span.
### Setting Span Attributes
You can set attributes on a span when it's created using the `attributes` parameter (less common for dynamic values) or, more typically, by calling the `update()` method on the `LangWatchSpan` object.
The `update()` method is flexible and allows you to pass attributes as keyword arguments.
```python Using update() method
import langwatch
import os
langwatch.setup(api_key=os.getenv("LANGWATCH_API_KEY"))
@langwatch.trace(name="ArticleGenerator")
def generate_article(topic: str):
with langwatch.span(name="FetchResearchData", type="tool") as research_span:
# Simulate fetching data
research_data = f"Data about {topic}"
research_span.update(
source="internal_db",
query_complexity="medium",
items_retrieved=10
)
# research_span.set_attributes({"source": "internal_db"}) # Also works
with langwatch.span(name="GenerateText", type="llm") as llm_span:
llm_span.update(model="gpt-4o-mini", prompt_length=len(topic))
# Simulate LLM call
article_text = f"Here's an article about {topic} based on {research_data}."
llm_span.update(output_length=len(article_text), tokens_used=150)
return article_text
generate_article("AI in Healthcare")
```
```python Using attributes parameter (for static attributes)
import langwatch
import os
langwatch.setup(api_key=os.getenv("LANGWATCH_API_KEY"))
@langwatch.trace(name="StaticAttributeDemo")
def process_with_static_info():
# 'version' is a static attribute for this span's definition
with langwatch.span(name="ComponentX", attributes={"version": "1.0.2", "region": "us-east-1"}):
# ... logic for ComponentX ...
print("ComponentX with static attributes executed.")
# Dynamic attributes can still be added via update()
langwatch.get_current_span().update(status_code=200)
process_with_static_info()
```
In the first example, `source`, `query_complexity`, and `items_retrieved` are added to the "FetchResearchData" span. Similarly, `model`, `prompt_length`, `output_length`, and `tokens_used` contextualize the "GenerateText" LLM span.
The `LangWatchSpan` object also has a `set_attributes()` method which takes a dictionary, similar to OpenTelemetry's underlying span API.
For more details on span parameters and methods, see the [`langwatch.span()` API reference](/integration/python/reference#langwatchspan) and the [`LangWatchSpan` object methods](/integration/python/reference#langwatchspan-object-methods).
## Key Differences: Trace Metadata vs. Span Attributes
| Feature | Trace Metadata | Span Attributes |
|-----------------|-------------------------------------------------|--------------------------------------------------------|
| **Scope** | Entire trace (e.g., a whole user request) | Specific span (e.g., one LLM call, one tool use) |
| **Granularity** | Coarse-grained, applies to the overall operation | Fine-grained, applies to a specific part of the operation |
| **Purpose** | General context for the entire operation | Specific details about a particular step or action |
| **Examples** | `user_id`, `session_id`, `app_version` | `model_name`, `tool_parameters`, `retrieved_chunk_id` |
| **SDK Access** | `langwatch.trace(metadata={...})` `trace.update(metadata={...})` | `langwatch.span(attributes={...})` `span.update(key=value, ...)` `span.set_attributes({...})` |
**When to use which:**
* Use **Trace Metadata** for information that you'd want to associate with every single span within that trace, or that defines the overarching context of the request (e.g., who initiated it, what version of the service is running).
* Use **Span Attributes** for details specific to the execution of that particular span. This helps in understanding the parameters, behavior, and outcome of individual components within your trace.
## Viewing in LangWatch
All captured trace metadata and span attributes will be visible in the LangWatch UI.
- **Trace Metadata** is typically displayed in the trace details view, providing an overview of the entire operation.
- **Span Attributes** are shown when you inspect individual spans within a trace.
This rich contextual data allows you to:
- **Filter and search** for traces and spans based on specific metadata or attribute values.
- **Analyze performance** by correlating metrics with different metadata/attributes (e.g., comparing latencies for different `user_id`s or `model_name`s).
- **Debug issues** by quickly understanding the context and parameters of a failed or slow operation.
## Conclusion
Effectively using trace metadata and span attributes is crucial for maximizing the observability of your LLM applications. By enriching your traces with relevant contextual information, you empower yourself to better understand, debug, and optimize your systems with LangWatch.
Remember to instrument your code thoughtfully, adding data that provides meaningful insights without being overly verbose.
---
# FILE: ./integration/python/tutorials/capturing-evaluations-guardrails.mdx
---
title: Capturing Evaluations & Guardrails
sidebarTitle: Evaluations & Guardrails
description: Learn how to log custom evaluations, trigger managed evaluations, and implement guardrails with LangWatch.
keywords: Custom Evaluations, Managed Evaluations, Guardrails, LangWatch Evaluations, add_evaluation, evaluate, async_evaluate, Evaluation Metric, Evaluation Score, Evaluation Pass/Fail, Evaluation Label, Evaluation Details, Evaluation Cost, Evaluation Status, Evaluation Error, Evaluation Timestamps, Evaluation Type, Guardrail
---
LangWatch provides a flexible system for capturing various types of evaluations and implementing guardrails within your LLM applications. This allows you to track performance, ensure quality, and control application flow based on defined criteria.
There are three main ways to work with evaluations and guardrails:
1. **Client-Side Custom Evaluations (`add_evaluation`)**: Log any custom evaluation metric, human feedback, or external system score directly from your Python code. These are primarily for observational purposes.
2. **Server-Side Managed Evaluations (`evaluate`, `async_evaluate`)**: Trigger predefined or custom evaluation logic that runs on the LangWatch backend. These can return scores, pass/fail results, and other details.
3. **Guardrails**: A special application of evaluations (either client-side or server-side) used to make decisions or enforce policies within your application flow.
## 1. Client-Side Custom Evaluations (`add_evaluation`)
You can log custom evaluation data directly from your application code using the `add_evaluation()` method on a `LangWatchSpan` or `LangWatchTrace` object. This is useful for recording metrics specific to your domain, results from external systems, or human feedback.
When you call `add_evaluation()`, LangWatch typically creates a new child span of type `evaluation` (or `guardrail` if `is_guardrail=True`) under the target span. This child span, named after your custom evaluation, stores its details, primarily in its `output` attribute.
Here's an example:
```python
import langwatch
# Assume langwatch.setup() has been called
@langwatch.span(name="Generate Response")
def process_request(user_query: str):
response_text = f"Response to: {user_query}"
langwatch.get_current_span().update(output=response_text)
# Example 1: A simple pass/fail custom evaluation
contains_keyword = "LangWatch" in response_text
langwatch.get_current_span().add_evaluation(
name="Keyword Check: LangWatch",
passed=contains_keyword,
details=f"Checked for 'LangWatch'. Found: {contains_keyword}"
)
# Example 2: A custom score for response quality
human_score = 4.5
langwatch.get_current_span().add_evaluation(
name="Human Review: Quality Score",
score=human_score,
label="Good",
details="Reviewed by Jane Doe. Response is clear and relevant."
)
# Example 3: A client-side guardrail check
is_safe = not ("unsafe_word" in response_text)
langwatch.get_current_span().add_evaluation(
name="Safety Check (Client-Side)",
passed=is_safe,
is_guardrail=True, # Mark this as a guardrail
details=f"Content safety check. Passed: {is_safe}"
)
if not is_safe:
# Potentially alter flow or log a critical warning
print("Warning: Client-side safety check failed!")
return response_text
@langwatch.trace(name="Process User Request")
def main():
user_question = "Tell me about LangWatch."
generated_response = process_request(user_question)
print(f"Query: {user_question}")
print(f"Response: {generated_response}")
if __name__ == "__main__":
main()
```
### `add_evaluation()` Parameters
The `add_evaluation()` method is available on both `LangWatchSpan` and `LangWatchTrace` objects (when using on a trace, you must specify the target `span`). For detailed parameter descriptions, please refer to the API reference:
- [`LangWatchSpan.add_evaluation()`](/integration/python/reference#add_evaluation-1)
- [`LangWatchTrace.add_evaluation()`](/integration/python/reference#add_evaluation)
## 2. Server-Side Managed Evaluations (`evaluate` & `async_evaluate`)
LangWatch allows you to trigger evaluations that are performed by the LangWatch backend. These can be [built-in evaluators](/llm-evaluation/list) (e.g., for faithfulness, relevance) or [custom evaluators you define](/evaluations/custom-evaluator-integration) in your LangWatch project settings.
You use the `evaluate()` (synchronous) or `async_evaluate()` (asynchronous) functions for this. These functions send the necessary data to the LangWatch API, which then processes the evaluation. These server-side evaluations are a core part of setting up [real-time monitoring and evaluations in production](/llm-evaluation/realtime/setup).
```python
import langwatch
from langwatch.evaluations import BasicEvaluateData
# from langwatch.types import RAGChunk # For RAG contexts
# Assume langwatch.setup() has been called
@langwatch.span()
def handle_rag_query(user_query: str):
retrieved_contexts_str = [
"LangWatch helps monitor LLM applications.",
"Evaluations can be run on the server."
]
# For richer context, use RAGChunk
# retrieved_contexts_rag = [
# RAGChunk(content="LangWatch helps monitor LLM applications.", document_id="doc1"),
# RAGChunk(content="Evaluations can be run on the server.", document_id="doc2")
# ]
# Add the RAG contexts to the current span
langwatch.get_current_span().update(contexts=retrieved_contexts_str)
# Simulate LLM call
llm_output = f"Based on the context, LangWatch is for monitoring and server-side evals."
# Prepare data for server-side evaluation
eval_data = BasicEvaluateData(
input=user_query,
output=llm_output,
contexts=retrieved_contexts_str
)
# Trigger a server-side "faithfulness" evaluation
# The 'faithfulness-evaluator' slug must be configured in your LangWatch project
try:
faithfulness_result = langwatch.evaluate(
slug="faithfulness-evaluator", # Slug of the evaluator in LangWatch
name="Faithfulness Check (Server)",
data=eval_data,
)
print(f"Faithfulness Evaluation Result: {faithfulness_result}")
# faithfulness_result is an EvaluationResultModel(status, passed, score, details, etc.)
# Example: Using it as a guardrail
if faithfulness_result.passed is False:
print("Warning: Faithfulness check failed!")
except Exception as e:
print(f"Error during server-side evaluation: {e}")
return llm_output
@langwatch.trace()
def main():
query = "What can LangWatch do with contexts?"
response = handle_rag_query(query)
print(f"Query: {query}")
print(f"Response: {response}")
if __name__ == "__main__":
main()
```
### `evaluate()` / `async_evaluate()` Key Parameters
The `evaluate()` and `async_evaluate()` methods are available on both `LangWatchSpan` and `LangWatchTrace` objects. They can also be imported from `langwatch.evaluations` and called as `langwatch.evaluate()` or `langwatch.async_evaluate()`, where you would then explicitly pass the `span` or `trace` argument. For detailed parameter descriptions, refer to the API reference:
- [`LangWatchSpan.evaluate()`](/integration/python/reference#evaluate-1) and [`LangWatchSpan.async_evaluate()`](/integration/python/reference#async_evaluate-1)
- [`LangWatchTrace.evaluate()`](/integration/python/reference#evaluate) and [`LangWatchTrace.async_evaluate()`](/integration/python/reference#async_evaluate)
**Understanding the `data` Parameter:**
The core parameters like `slug`, `data`, `settings`, `as_guardrail`, `span`, and `trace` are generally consistent.
For the `data` parameter specifically: while `BasicEvaluateData` is commonly used to provide a standardized structure for `input`, `output`, and `contexts` (which many built-in or common evaluators expect), it's important to know that `data` can be **any dictionary**. This flexibility allows you to pass arbitrary data structures tailored to custom server-side evaluators you might define. Using `BasicEvaluateData` with fields like `expected_output` is particularly useful when [evaluating if the LLM is generating the right answers](/llm-evaluation/offline/platform/answer-correctness) against a set of expected outputs. For scenarios where a golden answer isn't available, LangWatch also supports more open-ended evaluations, such as using an [LLM-as-a-judge](/llm-evaluation/offline/platform/llm-as-a-judge).
The `slug` parameter refers to the unique identifier of the evaluator configured in your LangWatch project settings. You can find a list of available evaluator types and learn how to configure them in our [LLM Evaluation documentation](/llm-evaluation/list).
The functions return an `EvaluationResultModel` containing `status`, `passed`, `score`, `details`, `label`, and `cost`.
## 3. Guardrails
Guardrails are evaluations used to make decisions or enforce policies within your application. They typically result in a boolean `passed` status that your code can act upon.
**Using Server-Side Evaluations as Guardrails:**
Set `as_guardrail=True` when calling `evaluate` or `async_evaluate`.
```python
# ... (inside a function with a current span)
eval_data = BasicEvaluateData(output=llm_response)
pii_check_result = langwatch.evaluate(
slug="pii-detection-guardrail",
data=eval_data,
as_guardrail=True,
span=langwatch.get_current_span()
)
if pii_check_result.passed is False:
# Take action: sanitize response, return a canned message, etc.
return "Response redacted due to PII."
```
A key behavior of `as_guardrail=True` for server-side evaluations is that if the *evaluation process itself* encounters an error (e.g., the evaluator service is down), the result will have `status="error"` but `passed` will default to `True`. This is a fail-safe to prevent your application from breaking due to an issue in the guardrail execution itself, assuming a "pass by default on error" stance is desired. For more on setting up safety-focused real-time evaluations like PII detection or prompt injection monitors, see our guide on [Setting up Real-Time Evaluations](/llm-evaluation/realtime/setup).
**Using Client-Side `add_evaluation` as Guardrails:**
Set `is_guardrail=True` when calling `add_evaluation`.
```python
# ... (inside a function with a current span)
is_too_long = len(llm_response) > 1000
response_span.add_evaluation(
name="Length Guardrail",
passed=(not is_too_long),
is_guardrail=True,
details=f"Length: {len(llm_response)}. Max: 1000"
)
if is_too_long:
# Take action: truncate response, ask for shorter output, etc.
return llm_response[:1000] + "..."
```
For client-side guardrails added with `add_evaluation`, your code is fully responsible for interpreting the `passed` status and handling any errors during the local check.
## How Evaluations and Guardrails Appear in LangWatch
Both client-side and server-side evaluations (including those marked as guardrails) are logged as spans in LangWatch.
- `add_evaluation`: Creates a child span of type `evaluation` (or `guardrail` if `is_guardrail=True`).
- `evaluate`/`async_evaluate`: Also create a child span of type `evaluation` (or `guardrail` if `as_guardrail=True`).
These spans will contain the evaluation's name, result (score, passed, label), details, cost, and any associated metadata, typically within their `output` attribute. This allows you to:
- See a history of all evaluation outcomes.
- Filter traces by evaluation results.
- Analyze the performance of different evaluators or guardrails.
- Correlate evaluation outcomes with other trace data (e.g., LLM inputs/outputs, latencies).
## Use Cases
- **Quality Assurance**:
- **Client-Side**: Log scores from a custom heuristic checking for politeness in responses.
- **Server-Side**: Trigger a managed ["Toxicity" evaluator](/llm-evaluation/list) on LLM outputs, or use more open-ended approaches like an [LLM-as-a-judge](/llm-evaluation/offline/platform/llm-as-a-judge) for tasks without predefined correct answers.
- **Compliance & Safety**:
- **Client-Side Guardrail**: Perform a regex check for forbidden words and log it with `is_guardrail=True`.
- **Server-Side Guardrail**: Use a managed ["PII Detection" evaluator](/llm-evaluation/list) with `as_guardrail=True` to decide if a response can be shown.
- **Performance Monitoring**:
- **Client-Side**: Log human feedback scores (`add_evaluation`) for helpfulness.
- **Server-Side**: Evaluate RAG system outputs for ["Context Relevancy" and "Faithfulness"](/llm-evaluation/list) using managed evaluators.
- **A/B Testing**: Log custom metrics or trigger standard evaluations for different model versions or prompts to compare their performance.
- **Feedback Integration**: `add_evaluation` can be used to pipe scores from an external human review platform directly into the relevant trace.
By combining these methods, you can build a robust evaluation and guardrailing strategy tailored to your application's needs, all observable within LangWatch.
---
# FILE: ./integration/python/tutorials/capturing-rag.mdx
---
title: Capturing RAG
sidebarTitle: Capturing RAG
description: Learn how to capture Retrieval Augmented Generation (RAG) data with LangWatch.
keywords: RAG, Retrieval Augmented Generation, LangChain, LangWatch, LangChain RAG, RAG Span, RAG Chunk, RAG Tool
---
Retrieval Augmented Generation (RAG) is a common pattern in LLM applications where you first retrieve relevant context from a knowledge base and then use that context to generate a response. LangWatch provides specific ways to capture RAG data, enabling better observability and evaluation of your RAG pipelines.
By capturing the `contexts` (retrieved documents) used by the LLM, you unlock several benefits in LangWatch:
- Specialized RAG evaluators (e.g., Faithfulness, Context Relevancy).
- Analytics on document usage (e.g., which documents are retrieved most often, which ones lead to better responses).
- Deeper insights into the retrieval step of your pipeline.
There are two main ways to capture RAG spans: manually creating a RAG span or using framework-specific integrations like the one for LangChain.
## Manual RAG Span Creation
You can manually create a RAG span by decorating a function with `@langwatch.span(type="rag")`. Inside this function, you should perform the retrieval and then update the span with the retrieved contexts.
The `contexts` should be a list of strings or `RAGChunk` objects. The `RAGChunk` object allows you to provide more metadata about each retrieved chunk, such as `document_id` and `source`.
Here's an example:
```python
import langwatch
import time # For simulating work
# Assume langwatch.setup() has been called elsewhere
@langwatch.span(type="llm")
def generate_answer_from_context(contexts: list[str], user_query: str):
# Simulate LLM call using the contexts
time.sleep(0.5)
response = f"Based on the context, the answer to '{user_query}' is..."
# You can update the LLM span with model details, token counts, etc.
langwatch.get_current_span().update(
model="gpt-4o-mini",
prompt=f"Contexts: {contexts}\nQuery: {user_query}",
completion=response
)
return response
@langwatch.span(type="rag", name="My Custom RAG Process")
def perform_rag(user_query: str):
# 1. Retrieve contexts
# Simulate retrieval from a vector store or other source
time.sleep(0.3)
retrieved_docs = [
"LangWatch helps monitor LLM applications.",
"RAG combines retrieval with generation for better answers.",
"Python is a popular language for AI development."
]
# Update the current RAG span with the retrieved contexts
# You can pass a list of strings directly
langwatch.get_current_span().update(contexts=retrieved_docs)
# Alternatively, for richer context information:
# from langwatch.types import RAGChunk
# rag_chunks = [
# RAGChunk(content="LangWatch helps monitor LLM applications.", document_id="doc1", source="internal_wiki/langwatch"),
# RAGChunk(content="RAG combines retrieval with generation for better answers.", document_id="doc2", source="blog/rag_explained")
# ]
# langwatch.get_current_span().update(contexts=rag_chunks)
# 2. Generate answer using the contexts
final_answer = generate_answer_from_context(contexts=retrieved_docs, user_query=user_query)
# The RAG span automatically captures its input (user_query) and output (final_answer)
# if capture_input and capture_output are not set to False.
return final_answer
@langwatch.trace(name="User Question Handler")
def handle_user_question(question: str):
langwatch.get_current_trace().update(
input=question,
metadata={"user_id": "example_user_123"}
)
answer = perform_rag(user_query=question)
langwatch.get_current_trace().update(output=answer)
return answer
if __name__ == "__main__":
user_question = "What is LangWatch used for?"
response = handle_user_question(user_question)
print(f"Question: {user_question}")
print(f"Answer: {response}")
```
In this example:
1. `perform_rag` is decorated with `@langwatch.span(type="rag")`.
2. Inside `perform_rag`, we simulate a retrieval step.
3. `langwatch.get_current_span().update(contexts=retrieved_docs)` is called to explicitly log the retrieved documents.
4. The generation step (`generate_answer_from_context`) is called, which itself can be another span (e.g., an LLM span).
## LangChain RAG Integration
If you are using LangChain, LangWatch provides utilities to simplify capturing RAG data from retrievers and tools.
### Capturing RAG from a Retriever
You can wrap your LangChain retriever with `langwatch.langchain.capture_rag_from_retriever`. This function takes your retriever and a lambda function to transform the retrieved `Document` objects into `RAGChunk` objects.
```python
import langwatch
from langwatch.types import RAGChunk
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores.faiss import FAISS
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain.tools.retriever import create_retriever_tool
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain.prompts import ChatPromptTemplate
from langchain.schema.runnable.config import RunnableConfig
# 1. Setup LangWatch (if not done globally)
# langwatch.setup()
# 2. Prepare your retriever
loader = WebBaseLoader("https://docs.langwatch.ai/introduction") # Example source
docs = loader.load()
documents = RecursiveCharacterTextSplitter(
chunk_size=1000, chunk_overlap=200
).split_documents(docs)
vector = FAISS.from_documents(documents, OpenAIEmbeddings())
retriever = vector.as_retriever()
# 3. Wrap the retriever for LangWatch RAG capture
# This lambda tells LangWatch how to extract data for RAGChunk from LangChain's Document
langwatch_retriever_tool = create_retriever_tool(
langwatch.langchain.capture_rag_from_retriever(
retriever,
lambda document: RAGChunk(
document_id=document.metadata.get("source", "unknown_source"), # Use a fallback for source
content=document.page_content,
# You can add other fields like 'score' if available in document.metadata
),
),
"langwatch_docs_search", # Tool name
"Search for information about LangWatch.", # Tool description
)
# 4. Use the wrapped retriever in your agent/chain
tools = [langwatch_retriever_tool]
model = ChatOpenAI(model="gpt-4o-mini", streaming=True)
prompt = ChatPromptTemplate.from_messages(
[
("system", "You are a helpful assistant. Answer questions based on the retrieved context.\n{agent_scratchpad}"),
("human", "{question}"),
]
)
agent = create_tool_calling_agent(model, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True) # type: ignore
@langwatch.trace(name="LangChain RAG Agent Execution")
def run_langchain_rag(user_input: str):
current_trace = langwatch.get_current_trace()
current_trace.update(metadata={"user_id": "lc_rag_user"})
# Ensure the LangChain callback is used to capture all LangChain steps
response = agent_executor.invoke(
{"question": user_input},
config=RunnableConfig(
callbacks=[current_trace.get_langchain_callback()]
),
)
output = response.get("output", "No output found.")=
return output
if __name__ == "__main__":
question = "What is LangWatch?"
answer = run_langchain_rag(question)
print(f"Question: {question}")
print(f"Answer: {answer}")
```
#### Key elements
- `langwatch.langchain.capture_rag_from_retriever(retriever, lambda document: ...)`: This wraps your existing retriever.
- The lambda function `lambda document: RAGChunk(...)` defines how to map fields from LangChain's `Document` to LangWatch's `RAGChunk`. This is crucial for providing detailed context information.
- The wrapped retriever is then used to create a tool, which is subsequently used in an agent or chain.
- Remember to include `langwatch.get_current_trace().get_langchain_callback()` in your `RunnableConfig` when invoking the chain/agent to capture all LangChain operations.
### Capturing RAG from a Tool
Alternatively, if your RAG mechanism is encapsulated within a generic LangChain `BaseTool`, you can use `langwatch.langchain.capture_rag_from_tool`.
```python
import langwatch
from langwatch.types import RAGChunk
@langwatch.trace()
def main():
my_custom_tool = ...
wrapped_tool = langwatch.langchain.capture_rag_from_tool(
my_custom_tool, lambda response: [
RAGChunk(
document_id=response["id"], # optional
chunk_id=response["chunk_id"], # optional
content=response["content"]
)
]
)
tools = [wrapped_tool] # use the new wrapped tool in your agent instead of the original one
model = ChatOpenAI(streaming=True)
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful assistant that only reply in short tweet-like responses, using lots of emojis and use tools only once.\n\n{agent_scratchpad}",
),
("human", "{question}"),
]
)
agent = create_tool_calling_agent(model, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
return executor.invoke(user_input, config=RunnableConfig(
callbacks=[langWatchCallback]
))
```
The `capture_rag_from_tool` approach is generally less direct for RAG from retrievers because you have to parse the tool's output (which is usually a string) to extract structured context information. `capture_rag_from_retriever` is preferred when dealing directly with LangChain retrievers.
By effectively capturing RAG spans, you gain much richer data in LangWatch, enabling more powerful analysis and evaluation of your RAG systems. Refer to the SDK examples for more detailed implementations.
---
# FILE: ./integration/python/tutorials/capturing-mapping-input-output.mdx
---
title: Capturing and Mapping Inputs & Outputs
sidebarTitle: Inputs & Outputs
description: Learn how to control the capture and structure of input and output data for traces and spans with the LangWatch Python SDK.
---
Effectively capturing the inputs and outputs of your LLM application's operations is crucial for observability. LangWatch provides flexible ways to manage this data, whether you prefer automatic capture or explicit control to map complex objects, format data, or redact sensitive information.
This tutorial covers how to:
* Understand automatic input/output capture.
* Explicitly set inputs and outputs for traces and spans.
* Dynamically update this data on active traces/spans.
* Handle different data formats, especially for chat messages.
## Automatic Input and Output Capture
By default, when you use `@langwatch.trace()` or `@langwatch.span()` as decorators on functions, the SDK attempts to automatically capture:
* **Inputs**: The arguments passed to the decorated function.
* **Outputs**: The value returned by the decorated function.
This behavior can be controlled using the `capture_input` and `capture_output` boolean parameters.
```python
import langwatch
import os
# Assume we have already setup LangWatch
# langwatch.setup()
@langwatch.trace(name="GreetUser", capture_input=True, capture_output=True)
def greet_user(name: str, greeting: str = "Hello"):
# 'name' and 'greeting' will be captured as input.
# The returned string will be captured as output.
return f"{greeting}, {name}!"
greet_user("Alice")
@langwatch.span(name="SensitiveOperation", capture_input=False, capture_output=False)
def process_sensitive_data(data: dict):
# Inputs and outputs for this span will not be automatically captured.
# You might explicitly set a sanitized version if needed.
print("Processing sensitive data...")
return {"status": "processed"}
@langwatch.trace(name="MainFlow")
def main_flow():
greet_user("Bob", greeting="Hi")
process_sensitive_data({"secret": "data"})
main_flow()
```
Refer to the API reference for [`@langwatch.trace()`](/integration/python/reference#%40langwatch-trace-%2F-langwatch-trace) and [`@langwatch.span()`](/integration/python/reference#%40langwatch-span-%2F-langwatch-span) for more details on `capture_input` and `capture_output` parameters.
## Explicitly Setting Inputs and Outputs
You often need more control over what data is recorded. You can explicitly set inputs and outputs using the `input` and `output` parameters when initiating a trace or span, or by using the `update()` method on the respective objects.
This is useful for:
* Capturing only specific parts of complex objects.
* Formatting data in a more readable or structured way (e.g., as a list of `ChatMessage` objects).
* Redacting sensitive information before it's sent to LangWatch.
* Providing inputs/outputs when not using decorators (e.g., with context managers for parts of a function).
### At Initialization
When using `@langwatch.trace()` or `@langwatch.span()` (either as decorators or context managers), you can pass `input` and `output` arguments.
```python Trace with explicit input/output
import langwatch
import os
# Assume we have already setup LangWatch
# langwatch.setup()
@langwatch.trace(
name="UserIntentProcessing",
input={"user_query": "Book a flight to London"},
# Output can be set later via update() if determined by function logic
)
def process_user_intent(raw_query_data: dict):
# raw_query_data might be large or contain sensitive info
# The 'input' parameter above provides a clean version.
intent = "book_flight"
entities = {"destination": "London"}
# Explicitly set the output for the root span of the trace
current_trace = langwatch.get_current_trace()
if current_trace:
current_trace.update(output={"intent": intent, "entities": entities})
return {"status": "success", "intent": intent} # Actual function return
process_user_intent({"query": "Book a flight to London", "user_id": "123"})
```
```python Span with explicit input/output
import langwatch
import os
from langwatch.domain import ChatMessage
# Assume we have already setup LangWatch
# langwatch.setup()
@langwatch.trace(name="ChatbotInteraction")
def handle_chat():
user_message = ChatMessage(role="user", content="What is LangWatch?")
with langwatch.span(
name="LLMCall",
type="llm",
input=[user_message],
model="gpt-4o-mini"
) as llm_span:
# Simulate LLM call
assistant_response_content = "LangWatch helps you monitor your LLM applications."
assistant_message = ChatMessage(role="assistant", content=assistant_response_content)
# Set output on the span object
llm_span.update(output=[assistant_message])
print("Chat finished.")
handle_chat()
```
If you provide `input` or `output` directly, it overrides what might have been automatically captured for that field.
### Dynamically Updating Inputs and Outputs
You can modify the input or output of an active trace or span using its `update()` method. This is particularly useful when the input/output data is determined or refined during the operation.
```python
import langwatch
import os
# Assume we have already setup LangWatch
# langwatch.setup()
@langwatch.trace(name="DataTransformationPipeline")
def run_pipeline(initial_data: dict):
# Initial input is automatically captured if capture_input=True (default)
with langwatch.span(name="Step1_CleanData") as step1_span:
# Suppose initial_data is complex, we want to record a summary as input
step1_span.update(input={"data_keys": list(initial_data.keys())})
cleaned_data = {k: v for k, v in initial_data.items() if v is not None}
step1_span.update(output={"cleaned_item_count": len(cleaned_data)})
# ... further steps ...
# Update the root span's output for the entire trace
final_result = {"status": "completed", "items_processed": len(cleaned_data)}
langwatch.get_current_trace().update(output=final_result)
return final_result
run_pipeline({"a": 1, "b": None, "c": 3})
```
The `update()` method on `LangWatchTrace` and `LangWatchSpan` objects is versatile. See the reference for [`LangWatchTrace` methods](/integration/python/reference#%40langwatch-trace-%2F-langwatch-trace) and [`LangWatchSpan` methods](/integration/python/reference#%40langwatch-span-%2F-langwatch-span).
## Handling Different Data Formats
LangWatch can store various types of input and output data:
* **Strings**: Simple text.
* **Dictionaries**: Automatically serialized as JSON. This is useful for structured data.
* **Lists of `ChatMessage` objects**: The standard way to represent conversations for LLM interactions. This ensures proper display and analysis in the LangWatch UI.
### Capturing Chat Messages
For LLM interactions, structure your inputs and outputs as a list of `ChatMessage` objects.
```python
import langwatch
import os
from langwatch.domain import ChatMessage, ToolCall, FunctionCall # For more complex messages
# Assume we have already setup LangWatch
# langwatch.setup()
@langwatch.trace(name="AdvancedChat")
def advanced_chat_example():
messages = [
ChatMessage(role="system", content="You are a helpful assistant."),
ChatMessage(role="user", content="What is the weather in London?")
]
with langwatch.span(name="GetWeatherToolCall", type="llm", input=messages, model="gpt-4o-mini") as llm_span:
# Simulate model deciding to call a tool
tool_call_id = "call_abc123"
assistant_response_with_tool = ChatMessage(
role="assistant",
tool_calls=[
ToolCall(
id=tool_call_id,
type="function",
function=FunctionCall(name="get_weather", arguments='''{"location": "London"}''')
)
]
)
llm_span.update(output=[assistant_response_with_tool])
# Simulate tool execution
with langwatch.span(name="RunGetWeatherTool", type="tool") as tool_span:
tool_input = {"tool_name": "get_weather", "arguments": {"location": "London"}}
tool_span.update(input=tool_input)
tool_result_content = '''{"temperature": "15C", "condition": "Cloudy"}'''
tool_span.update(output=tool_result_content)
# Prepare message for next LLM call
tool_response_message = ChatMessage(
role="tool",
tool_call_id=tool_call_id,
name="get_weather",
content=tool_result_content
)
messages.append(assistant_response_with_tool) # Assistant's decision to call tool
messages.append(tool_response_message) # Tool's response
with langwatch.span(name="FinalLLMResponse", type="llm", input=messages, model="gpt-4o-mini") as final_llm_span:
final_assistant_content = "The weather in London is 15°C and cloudy."
final_assistant_message = ChatMessage(role="assistant", content=final_assistant_content)
final_llm_span.update(output=[final_assistant_message])
advanced_chat_example()
```
For the detailed structure of `ChatMessage`, `ToolCall`, and other related types, please refer to the [Core Data Types section in the API Reference](/integration/python/reference#core-data-types).
## Use Cases and Best Practices
* **Redacting Sensitive Information**: If your function arguments or return values contain sensitive data (PII, API keys), disable automatic capture (`capture_input=False`, `capture_output=False`) and explicitly set sanitized versions using `input`/`output` parameters or `update()`.
* **Mapping Complex Objects**: If your inputs/outputs are complex Python objects, map them to a dictionary or a simplified string representation for clearer display in LangWatch.
* **Improving Readability**: For long text inputs/outputs (e.g., full documents), consider capturing a summary or metadata instead of the entire content to reduce noise, unless the full content is essential for debugging or evaluating.
* **Clearing Captured Data**: You can set `input=None` or `output=None` via the `update()` method to remove previously captured (or auto-captured) data if it's no longer relevant or was captured in error.
```python
import langwatch
import os
# Assume we have already setup LangWatch
# langwatch.setup()
@langwatch.trace(name="DataRedactionExample")
def handle_user_data(user_profile: dict):
# user_profile might contain PII
# Automatic capture is on by default.
# Let's update the input to a redacted version for the root span.
redacted_input = {
"user_id": user_profile.get("id"),
"has_email": "email" in user_profile
}
langwatch.get_current_trace().update(input=redacted_input)
# Process data...
result = {"status": "processed", "user_id": user_profile.get("id")}
langwatch.get_current_trace().update(output=result)
return result # Actual function return can still be the full data
handle_user_data({"id": "user_xyz", "email": "test@example.com", "name": "Sensitive Name"})
```
## Conclusion
Controlling how inputs and outputs are captured in LangWatch allows you to tailor the observability data to your specific needs. By using automatic capture flags, explicit parameters, dynamic updates, and appropriate data formatting (especially `ChatMessage` for conversations), you can ensure that your traces provide clear, relevant, and secure insights into your LLM application's behavior.
---
# FILE: ./integration/python/tutorials/manual-instrumentation.mdx
---
title: Manual Instrumentation
description: Learn how to manually instrument your code with the LangWatch Python SDK
keywords: manual instrumentation, context managers, span, trace, async, synchronous, LangWatch, Python
---
While decorators offer a concise way to instrument functions, you might prefer or need to manually manage trace and span lifecycles. This is useful in asynchronous contexts, for finer control, or when decorators are inconvenient. The LangWatch Python SDK provides two primary ways to do this manually:
### Using Context Managers (`with`/`async with`)
The `langwatch.trace()` and `langwatch.span()` functions can be used directly as asynchronous (`async with`) or synchronous (`with`) context managers. This is the recommended approach for manual instrumentation as it automatically handles ending the trace/span, even if errors occur.
Here's how you can achieve the same instrumentation as the decorator examples, but using context managers:
```python
import langwatch
from langwatch.types import RAGChunk
from langwatch.attributes import AttributeKey # For semantic attribute keys
import asyncio # Assuming async operation
langwatch.setup()
async def rag_retrieval_manual(query: str):
# Use async with for the span, instead of a decorator
async with langwatch.span(type="rag", name="RAG Document Retrieval") as span:
# ... your async retrieval logic ...
await asyncio.sleep(0.05) # Simulate async work
search_results = [
{"id": "doc-1", "content": "Content for doc 1."},
{"id": "doc-2", "content": "Content for doc 2."},
]
# Update the span with input, context, metadata, and output
span.update(
input=query,
contexts=[
RAGChunk(document_id=doc["id"], content=doc["content"])
for doc in search_results
],
output=search_results,
strategy="manual_vector_search"
)
return search_results
async def handle_user_query_manual(query: str):
# Use async with for the trace
async with langwatch.trace(name="Manual User Query Handling", metadata={"user_id": "manual-user", "query": query}) as trace:
# Call the manually instrumented RAG function
retrieved_docs = await rag_retrieval_manual(query)
# --- Simulate LLM Call Step (manual span) ---
llm_response = ""
async with langwatch.span(type="llm", name="Manual LLM Generation") as llm_span:
llm_input = {"role": "user", "content": f"Context: {retrieved_docs}\nQuery: {query}"}
llm_metadata = {"model_name": "gpt-4o-mini"}
# ... your async LLM call logic ...
await asyncio.sleep(0.1)
llm_response = "This is the manual LLM response."
llm_output = {"role": "assistant", "content": llm_response}
# Set input, metadata and output via update
llm_span.update(
input=llm_input,
output=llm_output
llm_metadata=llm_metadata,
)
# Set final trace output via update
trace.update(output=llm_response)
return llm_response
# Example execution (in an async context)
async def main():
result = await handle_user_query_manual("Tell me about manual tracing with context managers.")
print(result)
asyncio.run(main())
```
Key points for manual instrumentation with context managers:
- Use `with langwatch.trace(...)` or `async with langwatch.trace(...)` to start a trace.
- Use `with langwatch.span(...)` or `async with langwatch.span(...)` inside a trace block to create nested spans.
- The trace or span object is available in the `as trace:` or `as span:` part of the `with` statement.
- Use methods like `span.add_event()`, and primarily `span.update(...)` / `trace.update(...)` to add details. The `update()` method is flexible for adding structured data like `input`, `output`, `metadata`, and `contexts`.
- This approach gives explicit control over the start and end of each instrumented block, as the context manager handles ending the span automatically.
### Direct Span Creation (`span.end()`)
Alternatively, you can manage span and trace lifecycles completely manually. Call `langwatch.span()` or `langwatch.trace()` directly to start them, and then explicitly call the `end()` method on the returned object (`span.end()` or `trace.end()`) when the operation finishes. **This requires careful handling to ensure `end()` is always called, even if errors occur (e.g., using `try...finally`).** Context managers are generally preferred as they handle this automatically.
```python
import langwatch
import time
# Assume langwatch.setup() and a trace context exist
def process_data_manually(data):
span = langwatch.span(name="Manual Data Processing") # Start the span
try:
span.update(input=data)
# ... synchronous processing logic ...
time.sleep(0.02)
result = f"Processed: {data}"
span.update(output=result)
return result
except Exception as e:
span.record_exception(e) # Record exceptions
span.set_status("error", description=str(e))
raise # Re-raise the exception
finally:
span.end() # CRITICAL: Ensure the span is ended
# with langwatch.trace(): # Needs to be within a trace
# processed = process_data_manually("some data")
```
---
# FILE: ./integration/python/tutorials/open-telemetry.mdx
---
title: Using LangWatch with OpenTelemetry
description: Learn how to integrate the LangWatch Python SDK with your existing OpenTelemetry setup.
keywords: OpenTelemetry, OTel, auto-instrumentation, OpenAI, Celery, HTTP clients, databases, ORMs, LangWatch, Python
---
The LangWatch Python SDK is built entirely on top of the robust [OpenTelemetry (OTel)](https://opentelemetry.io/) standard. This means seamless integration with existing OTel setups and interoperability with the wider OTel ecosystem.
## LangWatch Spans are OpenTelemetry Spans
It's important to understand that LangWatch traces and spans **are** standard OpenTelemetry traces and spans. LangWatch adds specific semantic attributes (like `langwatch.span.type`, `langwatch.inputs`, `langwatch.outputs`, `langwatch.metadata`) to these standard spans to power its observability features.
This foundation provides several benefits:
- **Interoperability:** Traces generated with LangWatch can be sent to any OTel-compatible backend (Jaeger, Tempo, Datadog, etc.) alongside your other application traces.
- **Familiar API:** If you're already familiar with OpenTelemetry concepts and APIs, working with LangWatch's manual instrumentation will feel natural.
- **Leverage Existing Setup:** LangWatch integrates smoothly with your existing OTel `TracerProvider` and instrumentation.
Perhaps the most significant advantage is that **LangWatch seamlessly integrates with the vast ecosystem of standard OpenTelemetry auto-instrumentation libraries.** This means you can easily combine LangWatch's LLM-specific observability with insights from other parts of your application stack. For example, if you use `opentelemetry-instrumentation-celery`, traces initiated by LangWatch for an LLM task can automatically include spans generated within your Celery workers, giving you a complete end-to-end view of the request, including background processing, without any extra configuration.
## Leverage the OpenTelemetry Ecosystem: Auto-Instrumentation
One of the most powerful benefits of LangWatch's OpenTelemetry foundation is its **automatic compatibility with the extensive ecosystem of OpenTelemetry auto-instrumentation libraries.**
When you use standard OTel auto-instrumentation for libraries like web frameworks, databases, or task queues alongside LangWatch, you gain **complete end-to-end visibility** into your LLM application's requests. Because LangWatch and these auto-instrumentors use the same underlying OpenTelemetry tracing system and context propagation mechanisms, spans generated across different parts of your application are automatically linked together into a single, unified trace.
This means you don't need to manually stitch together observability data from your LLM interactions and the surrounding infrastructure. If LangWatch instruments an LLM call, and that call involves fetching data via an instrumented database client or triggering a background task via an instrumented queue, all those operations will appear as connected spans within the same trace view in LangWatch (and any other OTel backend you use).
### Examples of Auto-Instrumentation Integration
Here are common scenarios where combining LangWatch with OTel auto-instrumentation provides significant value:
* **Web Frameworks (FastAPI, Flask, Django):** Using libraries like `opentelemetry-instrumentation-fastapi`, an incoming HTTP request automatically starts a trace. When your request handler calls a function instrumented with `@langwatch.trace` or `@langwatch.span`, those LangWatch spans become children of the incoming request span. You see the full request lifecycle, from web server entry to LLM processing and response generation.
* **HTTP Clients (Requests, httpx, aiohttp):** If your LLM application makes outbound API calls (e.g., to fetch external data, call a vector database API, or use a non-instrumented LLM provider via REST) using libraries instrumented by `opentelemetry-instrumentation-requests` or similar, these HTTP request spans will automatically appear within your LangWatch trace, showing the latency and success/failure of these external dependencies.
* **Task Queues (Celery, RQ):** When a request handled by your web server (and traced by LangWatch) enqueues a background job using `opentelemetry-instrumentation-celery`, the trace context is automatically propagated. The spans generated by the Celery worker processing that job will be linked to the original LangWatch trace, giving you visibility into asynchronous operations triggered by your LLM pipeline.
* **Databases & ORMs (SQLAlchemy, Psycopg2, Django ORM):** Using libraries like `opentelemetry-instrumentation-sqlalchemy`, any database queries executed during your LLM processing (e.g., for RAG retrieval, user data lookup, logging results) will appear as spans within the relevant LangWatch trace, pinpointing database interaction time and specific queries.
To enable this, simply ensure you have installed and configured the relevant OpenTelemetry auto-instrumentation libraries according to their documentation, typically involving an installation (`pip install opentelemetry-instrumentation-`) and sometimes an initialization step (like `CeleryInstrumentor().instrument()`). As long as they use the same (or the global) `TracerProvider` that LangWatch is configured with, the integration is automatic.
#### Example: Combining LangWatch, RAG, OpenAI, and Celery
Let's illustrate this with a simplified example involving a web request that performs RAG, calls OpenAI, and triggers a background Celery task.
```txt requirements.txt
langwatch
openai
celery
opentelemetry-instrumentation-celery
```
```python example.py
import langwatch
import os
import asyncio
from celery import Celery
from openai import OpenAI
from langwatch.types import RAGChunk
# 1. Configure Celery App
celery_app = Celery('tasks', broker=os.getenv('CELERY_BROKER_URL', 'redis://localhost:6379/0'))
# 2. Setup LangWatch and OpenTelemetry Instrumentation
from opentelemetry_instrumentation.celery import CeleryInstrumentor
CeleryInstrumentor().instrument()
# Now setup LangWatch (it will likely pick up the global provider configured by Celery)
langwatch.setup(
# If you have other OTel exporters, configure your TracerProvider manually
# and pass it via tracer_provider=..., setting ignore_warning=True
ignore_global_tracer_provider_override_warning=True
)
client = OpenAI()
# 3. Define the Celery Task
@celery_app.task
def process_result_background(result_id: str, llm_output: str):
# This task execution will be automatically linked to the trace
# that enqueued it, thanks to CeleryInstrumentor.
# Spans created here (e.g., database writes) would be part of the same trace.
print(f"[Celery Worker] Processing result {result_id}...")
# Simulate work
import time
time.sleep(1)
print(f"[Celery Worker] Finished processing {result_id}")
return f"Processed: {llm_output[:10]}..."
# 4. Define RAG and Main Processing Logic
@langwatch.span(type="rag")
def retrieve_documents(query: str) -> list:
# Simulate RAG retrieval
print(f"Retrieving documents for: {query}")
chunks = [
RAGChunk(document_id="doc-abc", content="LangWatch uses OpenTelemetry."),
RAGChunk(document_id="doc-def", content="Celery integrates with OpenTelemetry."),
]
langwatch.get_current_span().update(contexts=chunks)
time.sleep(0.1)
return [c.content for c in chunks]
@langwatch.trace(name="Handle User Query with Celery")
def handle_request(user_query: str):
# This is the root span for the request
langwatch.get_current_trace().autotrack_openai_calls(client)
langwatch.get_current_trace().update(metadata={"user_query": user_query})
context_docs = retrieve_documents(user_query)
try:
completion = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": f"Use this context: {context_docs}"},
{"role": "user", "content": user_query}
],
temperature=0.5,
)
llm_result = completion.choices[0].message.content
except Exception as e:
langwatch.get_current_trace().record_exception(e)
llm_result = "Error calling OpenAI"
result_id = f"res_{int(time.time())}"
# The current trace context is automatically propagated
process_result_background.delay(result_id, llm_result)
print(f"Enqueued background processing task {result_id}")
return llm_result
# 5. Simulate Triggering the Request
if __name__ == "__main__":
print("Simulating web request...")
final_answer = handle_request("How does LangWatch work with Celery?")
print(f"\nFinal Answer returned to user: {final_answer}")
# Allow time for task to be processed if running worker locally
time.sleep(3) # Add a small delay to see Celery output
# To run this example:
# 1. Start a Celery worker: celery -A your_module_name worker --loglevel=info
# 2. Run this Python script.
# 3. Observe the logs and the trace in LangWatch/OTel backend.
```
In this example:
- The `handle_request` function is the main trace.
- `retrieve_documents` is a child span created by LangWatch.
- The OpenAI call creates child spans (due to `autotrack_openai_calls`).
- The call to `process_result_background.delay` creates a span indicating the task was enqueued.
- Critically, `CeleryInstrumentor` automatically propagates the trace context, so when the Celery worker picks up the `process_result_background` task, its execution is linked as a child span (or spans, if the task itself creates more) under the original `handle_request` trace.
This gives you a unified view of the entire operation, from the initial request through LLM processing, RAG, and background task execution.
## Integrating with `langwatch.setup()`
When you call `langwatch.setup()`, it intelligently interacts with your existing OpenTelemetry environment:
1. **Checks for Existing `TracerProvider`:**
- If you provide a `TracerProvider` instance via the `tracer_provider` argument in `langwatch.setup()`, LangWatch will use that specific provider.
- If you *don't* provide one, LangWatch checks if a global `TracerProvider` has already been set (e.g., by another library or your own OTel setup code).
- If neither is found, LangWatch creates a new `TracerProvider`.
2. **Adding the LangWatch Exporter:**
- If LangWatch uses an *existing* `TracerProvider` (either provided via the argument or detected globally), it will **add its own OTLP Span Exporter** to that provider's list of Span Processors. It does *not* remove existing processors or exporters.
- If LangWatch creates a *new* `TracerProvider`, it configures it with the LangWatch OTLP Span Exporter.
## Default Behavior: All Spans Go to LangWatch
A crucial point is that once `langwatch.setup()` runs and attaches its exporter to a `TracerProvider`, **all spans** managed by that provider will be exported to the LangWatch backend by default. This includes:
- Spans created using `@langwatch.trace` and `@langwatch.span`.
- Spans created manually using `langwatch.trace()` or `langwatch.span()` as context managers or via `span.end()`.
- Spans generated by standard OpenTelemetry auto-instrumentation libraries (e.g., `opentelemetry-instrumentation-requests`, `opentelemetry-instrumentation-fastapi`) if they are configured to use the same `TracerProvider`.
- Spans you create directly using the OpenTelemetry API (`tracer.start_as_current_span(...)`).
While seeing all application traces can be useful, you might not want *every single span* sent to LangWatch, especially high-volume or low-value ones (like health checks or database pings).
## Selectively Exporting Spans with `span_exclude_rules`
To control which spans are sent to LangWatch, use the `span_exclude_rules` argument during `langwatch.setup()`. This allows you to define rules to filter spans *before* they are exported to LangWatch, without affecting other exporters attached to the same `TracerProvider`.
Rules are defined using `SpanProcessingExcludeRule` objects.
```python
import langwatch
import os
from langwatch.domain import SpanProcessingExcludeRule
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
# Example: You already have an OTel setup exporting to console
existing_provider = TracerProvider()
existing_provider.add_span_processor(
SimpleSpanProcessor(ConsoleSpanExporter())
)
# Define rules to prevent specific spans from going to LangWatch
# (They will still go to the Console exporter)
exclude_rules = [
# Exclude spans exactly named "GET /health_check"
SpanProcessingExcludeRule(
field_name="span_name",
match_value="GET /health_check",
match_operation="exact_match"
),
# Exclude spans where 'http.method' attribute is 'OPTIONS'
SpanProcessingExcludeRule(
field_name="attribute",
attribute_name="http.method",
match_value="OPTIONS",
match_operation="exact_match"
),
# Exclude spans whose names start with "Internal."
SpanProcessingExcludeRule(
field_name="span_name",
match_value="Internal.",
match_operation="starts_with"
),
]
# Setup LangWatch to use the existing provider and apply exclude rules
langwatch.setup(
api_key=os.getenv("LANGWATCH_API_KEY"),
tracer_provider=existing_provider, # Use our existing provider
span_exclude_rules=exclude_rules,
# Important: Set this if you intend for LangWatch to use the existing provider
# and want to silence the warning about not overriding it.
ignore_global_tracer_provider_override_warning=True
)
# Now, create some spans using OTel API directly
tracer = existing_provider.get_tracer("my.app.tracer")
with tracer.start_as_current_span("GET /health_check") as span:
span.set_attribute("http.method", "GET")
# This span WILL go to Console Exporter
# This span WILL NOT go to LangWatch Exporter
with tracer.start_as_current_span("Process User Request") as span:
span.set_attribute("http.method", "POST")
span.set_attribute("user.id", "user-123")
# This span WILL go to Console Exporter
# This span WILL ALSO go to LangWatch Exporter
```
Refer to the `SpanProcessingExcludeRule` definition for all available fields (`span_name`, `attribute`, `library_name`) and operations (`exact_match`, `contains`, `starts_with`, `ends_with`, `regex`).
## Debugging with Console Exporter
When developing or troubleshooting your OpenTelemetry integration, it's often helpful to see the spans being generated locally without sending them to a backend. The OpenTelemetry SDK provides a `ConsoleSpanExporter` for this purpose.
You can add it to your `TracerProvider` like this:
```python Scenario 1: Managed Provider (Recommended)
import langwatch
import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
# Create your own TracerProvider
my_tracer_provider = TracerProvider()
# Add the ConsoleSpanExporter for debugging
my_tracer_provider.add_span_processor(
SimpleSpanProcessor(ConsoleSpanExporter())
)
# Now, setup LangWatch with your pre-configured provider
langwatch.setup(
tracer_provider=my_tracer_provider,
# If you are providing your own tracer_provider that might be global,
# you might want to set this to True if you see warnings.
# ignore_global_tracer_provider_override_warning=True
)
# Spans created via LangWatch or directly via OTel API using this provider
# will now also be printed to the console.
# Example of creating a span to test
tracer = my_tracer_provider.get_tracer("my.debug.tracer")
with tracer.start_as_current_span("My Test Span"):
print("This span should appear in the console.")
```
```python Scenario 2: Global Provider (Illustrative)
# Ensure necessary imports if running this snippet standalone
import os
import langwatch
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider # Needed for isinstance check
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
# In this case, you might try to get the global provider and add the exporter.
# Note: This can be less predictable if other libraries also manipulate the global provider.
langwatch.setup(
ignore_global_tracer_provider_override_warning=True # If a global provider exists
)
# Try to get the globally configured TracerProvider
global_provider = trace.get_tracer_provider()
# Check if it's an SDK TracerProvider instance that we can add a processor to
if isinstance(global_provider, TracerProvider):
global_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
# Example span after attempting to modify global provider
# Note: get_tracer from the global trace module
global_otel_tracer = trace.get_tracer("my.app.tracer.global")
with global_otel_tracer.start_as_current_span("Test Span with Global Provider"):
print("This span should appear in console if global provider was successfully modified.")
```
This will print all created spans to your console
## Accessing the OpenTelemetry Span API
Since LangWatch spans wrap standard OTel spans, the `LangWatchSpan` object (returned by `langwatch.span()` or accessed via `langwatch.get_current_span()`) directly exposes the standard OpenTelemetry `trace.Span` API methods. This allows you to interact with the span using familiar OTel functions when needed for advanced use cases or compatibility.
You don't need to access a separate underlying object; just call the standard OTel methods directly on the `LangWatchSpan` instance:
```python
import langwatch
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
langwatch.setup() # Assume setup is done
with langwatch.span(name="MyInitialSpanName") as span:
# Use standard OpenTelemetry Span API methods directly on span:
span.set_attribute("my.custom.otel.attribute", "value")
span.add_event("Specific OTel Event", {"detail": "more info"})
span.set_status(Status(StatusCode.ERROR, description="Something went wrong"))
span.update_name("MyUpdatedSpanName") # Renaming the span
print(f"Is Recording? {span.is_recording()}")
print(f"OTel Span Context: {span.get_span_context()}")
# You can still use LangWatch-specific methods like update()
span.update(langwatch_info="extra data")
```
This allows full flexibility, letting you use both LangWatch's structured data methods (`update`, etc.) and the standard OpenTelemetry span manipulation methods on the same object.
## Understanding `ignore_global_tracer_provider_override_warning`
If `langwatch.setup()` detects an existing *global* `TracerProvider` (one set via `opentelemetry.trace.set_tracer_provider()`) and you haven't explicitly passed a `tracer_provider` argument, LangWatch will log a warning by default. The warning states that it found a global provider and will attach its exporter to it rather than replacing it.
This warning exists because replacing a globally configured provider can sometimes break assumptions made by other parts of your application or libraries. However, in many cases, **attaching** the LangWatch exporter to the existing global provider is exactly the desired behavior.
If you are intentionally running LangWatch alongside an existing global OpenTelemetry setup and want LangWatch to simply add its exporter to that setup, you can silence this warning by setting:
```python
langwatch.setup(
# ... other options
ignore_global_tracer_provider_override_warning=True
)
```
---
# FILE: ./integration/typescript/guide.mdx
---
title: TypeScript Integration Guide
sidebarTitle: TypeScript
description: LangWatch TypeScript SDK integration guide
---
LangWatch library is the easiest way to integrate your TypeScript application with LangWatch, the messages are synced on the background so it doesn't intercept or block your LLM calls.
#### Prerequisites
- Obtain your `LANGWATCH_API_KEY` from the [LangWatch dashboard](https://app.langwatch.ai/).
#### Installation
```sh
npm install langwatch
```
#### Configuration
Ensure `LANGWATCH_API_KEY` is set:
### Environment variable
```bash .env
LANGWATCH_API_KEY='your_api_key_here'
```
### Client parameters
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch({
apiKey: 'your_api_key_here',
});
```
## Basic Concepts
- Each message triggering your LLM pipeline as a whole is captured with a [Trace](/concepts#traces).
- A [Trace](/concepts#traces) contains multiple [Spans](/concepts#spans), which are the steps inside your pipeline.
- A span can be an LLM call, a database query for a RAG retrieval, or a simple function transformation.
- Different types of [Spans](/concepts#spans) capture different parameters.
- [Spans](/concepts#spans) can be nested to capture the pipeline structure.
- [Traces](/concepts#traces) can be grouped together on LangWatch Dashboard by having the same [`thread_id`](/concepts#threads) in their metadata, making the individual messages become part of a conversation.
- It is also recommended to provide the [`user_id`](/concepts#user-id) metadata to track user analytics.
## Integration
### Vercel AI SDK
The Vercel AI SDK supports tracing via Next.js OpenTelemetry integration. By using the `LangWatchExporter`, you can automatically collect those traces to LangWatch.
First, you need to install the necessary dependencies:
```bash
npm install @vercel/otel langwatch @opentelemetry/api-logs @opentelemetry/instrumentation @opentelemetry/sdk-logs
```
Then, set up the OpenTelemetry for your application, follow one of the tabs below depending whether you are using AI SDK with Next.js or on Node.js:
### Next.js
You need to enable the `instrumentationHook` in your `next.config.js` file if you haven't already:
```javascript
/** @type {import('next').NextConfig} */
const nextConfig = {
experimental: {
instrumentationHook: true,
},
};
module.exports = nextConfig;
```
Next, you need to create a file named `instrumentation.ts` (or `.js`) in the __root directory__ of the project (or inside `src` folder if using one), with `LangWatchExporter` as the traceExporter:
```typescript
import { registerOTel } from '@vercel/otel'
import { LangWatchExporter } from 'langwatch'
export function register() {
registerOTel({
serviceName: 'next-app',
traceExporter: new LangWatchExporter({
apiKey: process.env.LANGWATCH_API_KEY
})
})
}
```
(Read more about Next.js OpenTelemetry configuration [on the official guide](https://nextjs.org/docs/app/building-your-application/optimizing/open-telemetry#manual-opentelemetry-configuration))
Finally, enable `experimental_telemetry` tracking on the AI SDK calls you want to trace:
```typescript
const result = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Explain why a chicken would make a terrible astronaut, be creative and humorous about it.',
experimental_telemetry: {
isEnabled: true,
// optional metadata
metadata: {
userId: "myuser-123",
threadId: "mythread-123",
},
},
});
```
### Node.js
For Node.js, start by following the official OpenTelemetry guide:
- [OpenTelemetry Node.js Getting Started](https://opentelemetry.io/docs/languages/js/getting-started/nodejs/)
Once you have set up OpenTelemetry, you can use the `LangWatchExporter` to automatically send your traces to LangWatch:
```typescript
import { LangWatchExporter } from 'langwatch'
const sdk = new NodeSDK({
traceExporter: new LangWatchExporter({
apiKey: process.env.LANGWATCH_API_KEY
}),
// ...
});
```
That's it! Your messages will now be visible on LangWatch:
## Example Project
You can find a full example project with a more complex pipeline and Vercel AI SDK and LangWatch integration [on our GitHub](https://github.com/langwatch/langwatch/blob/main/typescript-sdk/example/lib/chat/vercel-ai.tsx).
## Manual Integration
The docs from here below are for manual integration, in case you are not using the Vercel AI SDK OpenTelemetry integration,
you can manually start a trace to capture your messages:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then, you can start an LLM span inside the trace with the input about to be sent to the LLM.
```typescript
import { convertFromVercelAIMessages } from 'langwatch'
const span = trace.startLLMSpan({
name: "llm",
model: model,
input: {
type: "chat_messages",
value: convertFromVercelAIMessages(messages)
},
});
```
This will capture the LLM input and register the time the call started. Once the LLM call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation, e.g.:
```typescript
span.end({
output: {
type: "chat_messages",
value: convertFromVercelAIMessages(output), // assuming output is Message[]
},
metrics: {
promptTokens: chatCompletion.usage?.prompt_tokens,
completionTokens: chatCompletion.usage?.completion_tokens,
},
});
```
### OpenAI
Start by initializing LangWatch client and creating a new trace to capture your messages:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then to capture your LLM calls, you can start an LLM span inside the trace with the input about to be sent to the LLM.
First, define the model and the messages you are going to use for your LLM call separately, so you can capture them:
```typescript
import { OpenAI } from "openai";
// Model to be used and messages that will be sent to the LLM
const model = "gpt-4o"
const messages : OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: "You are a helpful assistant." },
{
role: "user",
content: "Write a tweet-size vegetarian lasagna recipe for 4 people.",
},
]
```
Then, start the LLM span from the trace, giving it the model and input messages:
```typescript
const span = trace.startLLMSpan({
name: "llm",
model: model,
input: {
type: "chat_messages",
value: messages
},
});
```
This will capture the LLM input and register the time the call started. Now, continue with the LLM call normally, using the same parameters:
```typescript
const openai = new OpenAI();
const chatCompletion = await openai.chat.completions.create({
messages: messages,
model: model,
});
```
Finally, after the OpenAI call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation:
```typescript
span.end({
output: {
type: "chat_messages",
value: [chatCompletion.choices[0]!.message],
},
metrics: {
promptTokens: chatCompletion.usage?.prompt_tokens,
completionTokens: chatCompletion.usage?.completion_tokens,
},
});
```
### Azure
Start by initializing LangWatch client and creating a new trace to capture your messages:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then to capture your LLM calls, you can start an LLM span inside the trace with the input about to be sent to the LLM.
First, define the model and the messages you are going to use for your LLM call separately, so you can capture them:
```typescript
import { AzureOpenAI } from "openai";
// Model to be used and messages that will be sent to the LLM
const model = "gpt-4-turbo-2024-04-09"
const messages : OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: "You are a helpful assistant." },
{
role: "user",
content: "Write a tweet-size vegetarian lasagna recipe for 4 people.",
},
]
```
Then, start the LLM span from the trace, giving it the model and input messages:
```typescript
const span = trace.startLLMSpan({
name: "llm",
model: model,
input: {
type: "chat_messages",
value: messages
},
});
```
This will capture the LLM input and register the time the call started. Now, continue with the LLM call normally, using the same parameters:
```typescript
const openai = new AzureOpenAI({
apiKey: process.env.AZURE_OPENAI_API_KEY,
apiVersion: "2024-02-01",
endpoint: process.env.AZURE_OPENAI_ENDPOINT,
});
const chatCompletion = await openai.chat.completions.create({
messages: messages,
model: model,
});
```
Finally, after the OpenAI call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation:
```typescript
span.end({
output: {
type: "chat_messages",
value: [chatCompletion.choices[0]!.message],
},
metrics: {
promptTokens: chatCompletion.usage?.prompt_tokens,
completionTokens: chatCompletion.usage?.completion_tokens,
},
});
```
### LangChain.js
Start by initializing LangWatch client and creating a new trace to capture your chain:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then, to capture your LLM calls and all other chain steps, LangWatch provides a callback hook for LangChain.js that automatically tracks everything for you.
First, define your chain as you would normally do:
```typescript
import { StringOutputParser } from '@langchain/core/output_parsers'
import { ChatPromptTemplate } from '@langchain/core/prompts'
import { ChatOpenAI } from '@langchain/openai'
const prompt = ChatPromptTemplate.fromMessages([
['system', 'Translate the following from English into Italian'],
['human', '{input}']
])
const model = new ChatOpenAI({ model: 'gpt-3.5-turbo' })
const outputParser = new StringOutputParser()
const chain = prompt.pipe(model).pipe(outputParser)
```
Now, when calling your chain either with `invoke` or `stream`, pass in `trace.getLangChainCallback()` as one of the callbacks:
```typescript
const stream = await chain.stream(
{ input: message },
{ callbacks: [trace.getLangChainCallback()] }
)
```
That's it! The full trace with all spans for each chain step will be sent automatically to LangWatch in the background on periodic intervals. After capturing your first LLM Span, go to [LangWatch Dashboard](https://app.langwatch.ai), your message should be there!
On short-live environments like Lambdas or Serverless Functions, be sure to call `await trace.sendSpans();` to wait for all pending requests to be sent before the runtime is destroyed.
## Capture a RAG Span
Appart from LLM spans, another very used type of span is the RAG span. This is used to capture the retrieved contexts from a RAG that will be used by the LLM, and enables a whole new set of RAG-based features evaluations for RAG quality on LangWatch.
To capture a RAG, you can simply start a RAG span inside the trace, giving it the input query being used:
```typescript
const ragSpan = trace.startRAGSpan({
name: "my-vectordb-retrieval", // optional
input: { type: "text", value: "search query" },
});
// proceed to do the retrieval normally
```
Then, after doing the retrieval, you can end the RAG span with the contexts that were retrieved and will be used by the LLM:
```typescript
ragSpan.end({
contexts: [
{
documentId: "doc1",
content: "document chunk 1",
},
{
documentId: "doc2",
content: "document chunk 2",
},
],
});
```
On LangChain.js, RAG spans are captured automatically by the LangWatch callback when using LangChain Retrievers, with `source` as the documentId.
## Capture an arbritary Span
You can also use generic spans to capture any type of operation, its inputs and outputs, for example for a function call:
```typescript
// before the function starts
const span = trace.startSpan({
name: "weather_function",
input: {
type: "json",
value: {
city: "Tokyo",
},
},
});
// ...after the function ends
span.end({
output: {
type: "json",
value: {
weather: "sunny",
},
},
});
```
You can also nest spans one inside the other, capturing your pipeline structure, for example:
```typescript
const span = trace.startSpan({
name: "pipeline",
});
const nestedSpan = span.startSpan({
name: "nested_pipeline",
});
nestedSpan.end()
span.end()
```
Both LLM and RAG spans can also be nested like any arbritary span.
## Capturing Exceptions
To capture also when your code throws an exception, you can simply wrap your code around a try/catch, and update or end the span with the exception:
```typescript
try {
throw new Error("unexpected error");
} catch (error) {
span.end({
error: error,
});
}
```
## Capturing custom evaluation results
[LangWatch Evaluators](/llm-evaluation/list) can run automatically on your traces, but if you have an in-house custom evaluator, you can also capture the evaluation
results of your custom evaluator on the current trace or span by using the `.addEvaluation` method:
```typescript
import { type LangWatchTrace } from "langwatch";
async function llmStep({ message, trace }: { message: string, trace: LangWatchTrace }): Promise {
const span = trace.startLLMSpan({ name: "llmStep" });
// ... your existing code
span.addEvaluation({
name: "custom evaluation",
passed: true,
score: 0.5,
label: "category_detected",
details: "explanation of the evaluation results",
});
}
```
The evaluation `name` is required and must be a string. The other fields are optional, but at least one of `passed`, `score` or `label` must be provided.
---
# FILE: ./integration/typescript/integrations/langchain.mdx
---
title: LangChain
sidebarTitle: TS/JS
description: LangWatch LangChain TypeScript integration guide
---
LangWatch library is the easiest way to integrate your TypeScript application with LangWatch, the messages are synced on the background so it doesn't intercept or block your LLM calls.
## Basic Concepts
- Each message triggering your LLM pipeline as a whole is captured with a [Trace](/concepts#traces).
- A [Trace](/concepts#traces) contains multiple [Spans](/concepts#spans), which are the steps inside your pipeline.
- A span can be an LLM call, a database query for a RAG retrieval, or a simple function transformation.
- Different types of [Spans](/concepts#spans) capture different parameters.
- [Spans](/concepts#spans) can be nested to capture the pipeline structure.
- [Traces](/concepts#traces) can be grouped together on LangWatch Dashboard by having the same [`thread_id`](/concepts#threads) in their metadata, making the individual messages become part of a conversation.
- It is also recommended to provide the [`user_id`](/concepts#user-id) metadata to track user analytics.
## Integration
Start by initializing LangWatch client and creating a new trace to capture your chain:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then, to capture your LLM calls and all other chain steps, LangWatch provides a callback hook for LangChain.js that automatically tracks everything for you.
First, define your chain as you would normally do:
```typescript
import { StringOutputParser } from '@langchain/core/output_parsers'
import { ChatPromptTemplate } from '@langchain/core/prompts'
import { ChatOpenAI } from '@langchain/openai'
const prompt = ChatPromptTemplate.fromMessages([
['system', 'Translate the following from English into Italian'],
['human', '{input}']
])
const model = new ChatOpenAI({ model: 'gpt-3.5-turbo' })
const outputParser = new StringOutputParser()
const chain = prompt.pipe(model).pipe(outputParser)
```
Now, when calling your chain either with `invoke` or `stream`, pass in `trace.getLangChainCallback()` as one of the callbacks:
```typescript
const stream = await chain.stream(
{ input: message },
{ callbacks: [trace.getLangChainCallback()] }
)
```
That's it! The full trace with all spans for each chain step will be sent automatically to LangWatch in the background on periodic intervals. After capturing your first LLM Span, go to [LangWatch Dashboard](https://app.langwatch.ai), your message should be there!
On short-live environments like Lambdas or Serverless Functions, be sure to call `await trace.sendSpans();` to wait for all pending requests to be sent before the runtime is destroyed.
## Capture a RAG Span
Appart from LLM spans, another very used type of span is the RAG span. This is used to capture the retrieved contexts from a RAG that will be used by the LLM, and enables a whole new set of RAG-based features evaluations for RAG quality on LangWatch.
## Capture an arbritary Span
You can also use generic spans to capture any type of operation, its inputs and outputs, for example for a function call:
## Capturing Exceptions
To capture also when your code throws an exception, you can simply wrap your code around a try/catch, and update or end the span with the exception:
## Capturing custom evaluation results
[LangWatch Evaluators](/llm-evaluation/list) can run automatically on your traces, but if you have an in-house custom evaluator, you can also capture the evaluation
results of your custom evaluator on the current trace or span by using the `.addEvaluation` method:
---
# FILE: ./integration/typescript/integrations/vercel-ai-sdk.mdx
---
title: Vercel AI SDK
sidebarTitle: Vercel AI SDK
description: LangWatch Vercel AI SDK integration guide
---
LangWatch library is the easiest way to integrate your TypeScript application with LangWatch, the messages are synced on the background so it doesn't intercept or block your LLM calls.
## Basic Concepts
- Each message triggering your LLM pipeline as a whole is captured with a [Trace](/concepts#traces).
- A [Trace](/concepts#traces) contains multiple [Spans](/concepts#spans), which are the steps inside your pipeline.
- A span can be an LLM call, a database query for a RAG retrieval, or a simple function transformation.
- Different types of [Spans](/concepts#spans) capture different parameters.
- [Spans](/concepts#spans) can be nested to capture the pipeline structure.
- [Traces](/concepts#traces) can be grouped together on LangWatch Dashboard by having the same [`thread_id`](/concepts#threads) in their metadata, making the individual messages become part of a conversation.
- It is also recommended to provide the [`user_id`](/concepts#user-id) metadata to track user analytics.
## Integration
The Vercel AI SDK supports tracing via Next.js OpenTelemetry integration. By using the `LangWatchExporter`, you can automatically collect those traces to LangWatch.
First, you need to install the necessary dependencies:
```bash
npm install @vercel/otel langwatch @opentelemetry/api-logs @opentelemetry/instrumentation @opentelemetry/sdk-logs
```
Then, set up the OpenTelemetry for your application, follow one of the tabs below depending whether you are using AI SDK with Next.js or on Node.js:
### Next.js
You need to enable the `instrumentationHook` in your `next.config.js` file if you haven't already:
```javascript
/** @type {import('next').NextConfig} */
const nextConfig = {
experimental: {
instrumentationHook: true,
},
};
module.exports = nextConfig;
```
Next, you need to create a file named `instrumentation.ts` (or `.js`) in the __root directory__ of the project (or inside `src` folder if using one), with `LangWatchExporter` as the traceExporter:
```typescript
import { registerOTel } from '@vercel/otel'
import { LangWatchExporter } from 'langwatch'
export function register() {
registerOTel({
serviceName: 'next-app',
traceExporter: new LangWatchExporter({
apiKey: process.env.LANGWATCH_API_KEY
})
})
}
```
(Read more about Next.js OpenTelemetry configuration [on the official guide](https://nextjs.org/docs/app/building-your-application/optimizing/open-telemetry#manual-opentelemetry-configuration))
Finally, enable `experimental_telemetry` tracking on the AI SDK calls you want to trace:
```typescript
const result = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Explain why a chicken would make a terrible astronaut, be creative and humorous about it.',
experimental_telemetry: {
isEnabled: true,
// optional metadata
metadata: {
userId: "myuser-123",
threadId: "mythread-123",
},
},
});
```
### Node.js
For Node.js, start by following the official OpenTelemetry guide:
- [OpenTelemetry Node.js Getting Started](https://opentelemetry.io/docs/languages/js/getting-started/nodejs/)
Once you have set up OpenTelemetry, you can use the `LangWatchExporter` to automatically send your traces to LangWatch:
```typescript
import { LangWatchExporter } from 'langwatch'
const sdk = new NodeSDK({
traceExporter: new LangWatchExporter({
apiKey: process.env.LANGWATCH_API_KEY
}),
// ...
});
```
That's it! Your messages will now be visible on LangWatch:
## Example Project
You can find a full example project with a more complex pipeline and Vercel AI SDK and LangWatch integration [on our GitHub](https://github.com/langwatch/langwatch/blob/main/typescript-sdk/example/lib/chat/vercel-ai.tsx).
## Manual Integration
The docs from here below are for manual integration, in case you are not using the Vercel AI SDK OpenTelemetry integration,
you can manually start a trace to capture your messages:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then, you can start an LLM span inside the trace with the input about to be sent to the LLM.
```typescript
import { convertFromVercelAIMessages } from 'langwatch'
const span = trace.startLLMSpan({
name: "llm",
model: model,
input: {
type: "chat_messages",
value: convertFromVercelAIMessages(messages)
},
});
```
This will capture the LLM input and register the time the call started. Once the LLM call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation, e.g.:
```typescript
span.end({
output: {
type: "chat_messages",
value: convertFromVercelAIMessages(output), // assuming output is Message[]
},
metrics: {
promptTokens: chatCompletion.usage?.prompt_tokens,
completionTokens: chatCompletion.usage?.completion_tokens,
},
});
```
On short-live environments like Lambdas or Serverless Functions, be sure to call `await trace.sendSpans();` to wait for all pending requests to be sent before the runtime is destroyed.
## Capture a RAG Span
Appart from LLM spans, another very used type of span is the RAG span. This is used to capture the retrieved contexts from a RAG that will be used by the LLM, and enables a whole new set of RAG-based features evaluations for RAG quality on LangWatch.
## Capture an arbritary Span
You can also use generic spans to capture any type of operation, its inputs and outputs, for example for a function call:
## Capturing Exceptions
To capture also when your code throws an exception, you can simply wrap your code around a try/catch, and update or end the span with the exception:
## Capturing custom evaluation results
[LangWatch Evaluators](/llm-evaluation/list) can run automatically on your traces, but if you have an in-house custom evaluator, you can also capture the evaluation
results of your custom evaluator on the current trace or span by using the `.addEvaluation` method:
---
# FILE: ./integration/typescript/integrations/azure.mdx
---
title: Azure OpenAI
sidebarTitle: TS/JS
description: LangWatch Azure OpenAI integration guide
---
LangWatch library is the easiest way to integrate your TypeScript application with LangWatch, the messages are synced on the background so it doesn't intercept or block your LLM calls.
## Basic Concepts
- Each message triggering your LLM pipeline as a whole is captured with a [Trace](/concepts#traces).
- A [Trace](/concepts#traces) contains multiple [Spans](/concepts#spans), which are the steps inside your pipeline.
- A span can be an LLM call, a database query for a RAG retrieval, or a simple function transformation.
- Different types of [Spans](/concepts#spans) capture different parameters.
- [Spans](/concepts#spans) can be nested to capture the pipeline structure.
- [Traces](/concepts#traces) can be grouped together on LangWatch Dashboard by having the same [`thread_id`](/concepts#threads) in their metadata, making the individual messages become part of a conversation.
- It is also recommended to provide the [`user_id`](/concepts#user-id) metadata to track user analytics.
## Integration
Start by initializing LangWatch client and creating a new trace to capture your messages:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then to capture your LLM calls, you can start an LLM span inside the trace with the input about to be sent to the LLM.
First, define the model and the messages you are going to use for your LLM call separately, so you can capture them:
```typescript
import { AzureOpenAI } from "openai";
// Model to be used and messages that will be sent to the LLM
const model = "gpt-4-turbo-2024-04-09"
const messages : OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: "You are a helpful assistant." },
{
role: "user",
content: "Write a tweet-size vegetarian lasagna recipe for 4 people.",
},
]
```
Then, start the LLM span from the trace, giving it the model and input messages:
```typescript
const span = trace.startLLMSpan({
name: "llm",
model: model,
input: {
type: "chat_messages",
value: messages
},
});
```
This will capture the LLM input and register the time the call started. Now, continue with the LLM call normally, using the same parameters:
```typescript
const openai = new AzureOpenAI({
apiKey: process.env.AZURE_OPENAI_API_KEY,
apiVersion: "2024-02-01",
endpoint: process.env.AZURE_OPENAI_ENDPOINT,
});
const chatCompletion = await openai.chat.completions.create({
messages: messages,
model: model,
});
```
Finally, after the OpenAI call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation:
```typescript
span.end({
output: {
type: "chat_messages",
value: [chatCompletion.choices[0]!.message],
},
metrics: {
promptTokens: chatCompletion.usage?.prompt_tokens,
completionTokens: chatCompletion.usage?.completion_tokens,
},
});
```
On short-live environments like Lambdas or Serverless Functions, be sure to call `await trace.sendSpans();` to wait for all pending requests to be sent before the runtime is destroyed.
## Capture a RAG Span
Appart from LLM spans, another very used type of span is the RAG span. This is used to capture the retrieved contexts from a RAG that will be used by the LLM, and enables a whole new set of RAG-based features evaluations for RAG quality on LangWatch.
## Capture an arbritary Span
You can also use generic spans to capture any type of operation, its inputs and outputs, for example for a function call:
## Capturing Exceptions
To capture also when your code throws an exception, you can simply wrap your code around a try/catch, and update or end the span with the exception:
## Capturing custom evaluation results
[LangWatch Evaluators](/llm-evaluation/list) can run automatically on your traces, but if you have an in-house custom evaluator, you can also capture the evaluation
results of your custom evaluator on the current trace or span by using the `.addEvaluation` method:
---
# FILE: ./integration/typescript/integrations/mastra.mdx
---
title: Mastra
description: Learn how to integrate Mastra, a TypeScript agent framework, with LangWatch.
---
# Observability for Mastra With LangWatch
This guide shows you how to integrate **Mastra** with **LangWatch** for observability and tracing. By following these steps, you'll be able to monitor and debug your Mastra agents in the LangWatch dashboard.
## Integration
Create a Mastra project using the Mastra CLI:
```bash
npx create-mastra
```
Move into the project directory:
```bash
cd your-mastra-project
```
For more information, view Mastra installation instructions [here](https://mastra.ai/docs/getting-started/installation)
Create a project in [LangWatch](https://app.langwatch.ai) and get your API keys from the project settings page.
Create or update your `.env` file with the following variables:
```bash
# Your LLM API key
OPENAI_API_KEY=your-api-key
# LangWatch credentials
LANGWATCH_API_KEY=sk-...
```
Add the `langwatch` package to your project:
```bash
npm install langwatch
```
Add LangWatch to your Mastra instance via the telemetry exporter.
```typescript
import { Mastra } from "@mastra/core/mastra";
import { PinoLogger } from "@mastra/loggers";
import { LibSQLStore } from "@mastra/libsql";
import { LangWatchExporter } from "langwatch";
import { weatherAgent } from "./agents/weather-agent";
export const mastra = new Mastra({
agents: { weatherAgent },
telemetry: {
serviceName: "ai", // this must be set to "ai" so that the LangWatchExporter thinks it's an AI SDK trace
enabled: true,
export: {
type: "custom",
exporter: new LangWatchExporter({
apiKey: process.env.LANGWATCH_API_KEY,
debug: true,
// includeAllSpans: true,
}),
},
},
});
```
Start the Mastra development server:
```bash
npm run dev
```
Head over to the developer playground with the provided URL and start chatting with your agent.
### View traces in LangWatch
Visit your [LangWatch dashboard](https://app.langwatch.ai) to explore detailed insights into your agent interactions. Monitor and analyze every aspect of your AI conversations, from prompt engineering to response quality, helping you optimize your AI applications.
---
# FILE: ./integration/typescript/integrations/open-ai.mdx
---
title: OpenAI
sidebarTitle: TS/JS
description: LangWatch OpenAI TypeScript integration guide
---
LangWatch library is the easiest way to integrate your TypeScript application with LangWatch, the messages are synced on the background so it doesn't intercept or block your LLM calls.
## Basic Concepts
- Each message triggering your LLM pipeline as a whole is captured with a [Trace](/concepts#traces).
- A [Trace](/concepts#traces) contains multiple [Spans](/concepts#spans), which are the steps inside your pipeline.
- A span can be an LLM call, a database query for a RAG retrieval, or a simple function transformation.
- Different types of [Spans](/concepts#spans) capture different parameters.
- [Spans](/concepts#spans) can be nested to capture the pipeline structure.
- [Traces](/concepts#traces) can be grouped together on LangWatch Dashboard by having the same [`thread_id`](/concepts#threads) in their metadata, making the individual messages become part of a conversation.
- It is also recommended to provide the [`user_id`](/concepts#user-id) metadata to track user analytics.
## Integration
Start by initializing LangWatch client and creating a new trace to capture your messages:
```typescript
import { LangWatch } from 'langwatch';
const langwatch = new LangWatch();
const trace = langwatch.getTrace({
metadata: { threadId: "mythread-123", userId: "myuser-123" },
});
```
Then to capture your LLM calls, you can start an LLM span inside the trace with the input about to be sent to the LLM.
First, define the model and the messages you are going to use for your LLM call separately, so you can capture them:
```typescript
import { OpenAI } from "openai";
// Model to be used and messages that will be sent to the LLM
const model = "gpt-4o"
const messages : OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: "system", content: "You are a helpful assistant." },
{
role: "user",
content: "Write a tweet-size vegetarian lasagna recipe for 4 people.",
},
]
```
Then, start the LLM span from the trace, giving it the model and input messages:
```typescript
const span = trace.startLLMSpan({
name: "llm",
model: model,
input: {
type: "chat_messages",
value: messages
},
});
```
This will capture the LLM input and register the time the call started. Now, continue with the LLM call normally, using the same parameters:
```typescript
const openai = new OpenAI();
const chatCompletion = await openai.chat.completions.create({
messages: messages,
model: model,
});
```
Finally, after the OpenAI call is done, end the span to get the finish timestamp to be registered, and capture the output and the token metrics, which will be used for cost calculation:
```typescript
span.end({
output: {
type: "chat_messages",
value: [chatCompletion.choices[0]!.message],
},
metrics: {
promptTokens: chatCompletion.usage?.prompt_tokens,
completionTokens: chatCompletion.usage?.completion_tokens,
},
});
```
On short-live environments like Lambdas or Serverless Functions, be sure to call `await trace.sendSpans();` to wait for all pending requests to be sent before the runtime is destroyed.
## Capture a RAG Span
Appart from LLM spans, another very used type of span is the RAG span. This is used to capture the retrieved contexts from a RAG that will be used by the LLM, and enables a whole new set of RAG-based features evaluations for RAG quality on LangWatch.
## Capture an arbritary Span
You can also use generic spans to capture any type of operation, its inputs and outputs, for example for a function call:
## Capturing Exceptions
To capture also when your code throws an exception, you can simply wrap your code around a try/catch, and update or end the span with the exception:
## Capturing custom evaluation results
[LangWatch Evaluators](/llm-evaluation/list) can run automatically on your traces, but if you have an in-house custom evaluator, you can also capture the evaluation
results of your custom evaluator on the current trace or span by using the `.addEvaluation` method:
---
# FILE: ./integration/rest-api.mdx
---
title: REST API
description: Integrate LangWatch with any language by using the REST API
---
If your preferred programming language or platform is not directly supported by the existing LangWatch libraries, you can use the REST API with `curl` to send trace data. This guide will walk you through how to integrate LangWatch with any system that allows HTTP requests.
**Prerequisites:**
- Ensure you have `curl` installed on your system.
**Configuration:**
Set the `LANGWATCH_API_KEY` environment variable in your environment:
```bash
export LANGWATCH_API_KEY='your_api_key_here'
```
**Usage:**
You will need to prepare your span data in accordance with the Span type definitions provided by LangWatch. Below is an example of how to send span data using curl:
1. Prepare your JSON data. Make sure it's properly formatted as expected by LangWatch.
2. Use the curl command to send your trace data. Here is a basic template:
```bash
# Set your API key and endpoint URL
LANGWATCH_API_KEY="your_langwatch_api_key"
LANGWATCH_ENDPOINT="https://app.langwatch.ai"
# Use curl to send the POST request, e.g.:
curl -X POST "$LANGWATCH_ENDPOINT/api/collector" \
-H "X-Auth-Token: $LANGWATCH_API_KEY" \
-H "Content-Type: application/json" \
-d @- <
On LangChain.js, RAG spans are captured automatically by the LangWatch callback when using LangChain Retrievers, with `source` as the documentId.
### REST API
To track the RAG context when using the REST API, add a new span of type `rag`, you may also refer the LLM generation as the child of it:
```bash
curl -X POST "https://app.langwatch.ai/api/collector" \\
-H "X-Auth-Token: $API_KEY" \\
-H "Content-Type: application/json" \\
-d @- <
### Using Prompts in the Optimization Studio
To get started with prompt versioning in the Optimization Studio:
1. Create a new workflow or open an existing one
2. Drag a signature node onto the workspace
3. Click on the node to access configuration options in the right side panel
4. Make your desired changes to the prompt configuration
5. Save your changes as a new version
### Managing Prompt Versions
Once you've created a prompt version, you can:
- Restore previous versions by clicking the time icon
- Rename your prompts for better organization
- Reuse prompts across different workflows
- View version history and track changes
### Prompt Management Table
The Prompt Management table provides a centralized location to manage all your prompts:
1. Access the prompt table from the main menu
2. Create new prompts or edit existing ones
3. Save different versions of your prompts
4. View all saved versions in a list format
5. See who created each version (team collaboration)
### Team Collaboration
All prompts and their versions are shared across your project team:
- Any prompt created in the table is available in the studio
- Changes made in the studio are reflected in the prompt table
- Team members can see who created each version
- Prompts can be reused across different workflows
### API Integration
LangWatch provides a comprehensive API for managing prompts programmatically. You can use the API to:
- List all prompts for a project
- Create new prompts
- Update existing prompts
- Manage prompt versions
- Delete prompts
To use the Prompts API, you'll need to authenticate using your LangWatch API key. For detailed information about the API endpoints and authentication, see the [Prompts API Reference](/api-reference/prompts/overview).
The API supports the following operations:
- `GET /api/prompts` - Get all prompts for a project
- `POST /api/prompts` - Create a new prompt
- `GET /api/prompts/:id` - Get a specific prompt
- `PUT /api/prompts/:id` - Update a prompt
- `DELETE /api/prompts/:id` - Delete a prompt
- `GET /api/prompts/:id/versions` - Get all versions for a prompt
- `POST /api/prompts/:id/versions` - Create a new version for a prompt
---
# FILE: ./features/batch-evaluations.mdx
---
title: Batch Evaluations
---
If you intend to conduct batch evaluations on the datasets you've created in LangWatch, we offer a Python SKD to facilitate this process. This guide aims to provide comprehensive instructions on leveraging our Python SDK to execute batch evaluations effectively.
### Usage
After adding records to your dataset, created within the dataset section of LangWatch, you can proceed to select the dataset for batch evaluation along with the desired evaluations. You have the option to choose from predefined evaluations or any custom evaluations you've set up in the Evaluation and Guardrails section of LangWatch.
### Screenshots examples
In the below in screenshot you will see the datasets section in LangWatch, you can get your batch evaluation python snippet by clicking on on the Batch Evaluation button.
In the below screenshot you will see where you can select the dataset you want to evaluate on as well as selecting which evaluations you would like to run. Each tab has different evaluation you can choose from.
In the screenshot below, you'll find a Python code snippet ready for execution to perform your batch processing. The parameters passed into the `BatchEvaluation` include your chosen dataset and an array of selected evaluations to run against it.
We've streamlined the process by setting up pandas for you, enabling seamless evaluation of datasets directly on the results object. This means you can leverage the power of pandas' data manipulation and analysis capabilities effortlessly within your evaluation workflow. With pandas at your disposal, you can efficiently explore, analyze, and manipulate your data to derive valuable insights without the need for additional setup or configuration.
### Python snippet
When executing the snippet, you'll encounter a callback function at your disposal. This function contains the original entry data, allowing you to run it against your own Large Language Model (LLM). You can utilize this response to compare results within your evaluation process.
Ensure that you return the `output` as some evaluations may require it. As you create your code snippet in the evaluations tab, you'll notice indications of which evaluations necessitate particular information. Utilize this guidance as a reference to kickstart your workflow effectively.
---
# FILE: ./features/triggers.mdx
---
title: Alerts and Triggers
description: Be alerted when something goes wrong and trigger actions automatically
---
## Create triggers based on LangWatch filters
LangWatch offers you the possibility to create triggers based on your selected filters. You can use these triggers to send notifications to either Slack or selected team email adresses.
#### Usage
To create a trigger in the LangWatch dashboard, follow these steps:
- Click the filter button located at the top right of the LangWatch dashboard.
- After creating a filter, a trigger button will appear.
- Click the trigger button to open a popout drawer.
- In the drawer, you can configure your trigger with the desired settings.
**Trigger actions**
Once the trigger is created, you will receive an alert whenever a message meets the criteria of the trigger. These trigger checks are run on the minute but not instantaneously, as the data needs time to be processed. You can find the created triggers under the Settings section, where you can deactivate or delete a trigger to stop receiving notifications.
**Trigger settings**
---
# FILE: ./features/embedded-analytics.mdx
---
title: Exporting Analytics
description: Build and integrate LangWatch graphs on your own systems and applications
---
## Export Analytics with REST Endpoint
LangWatch offers you the possibility to build and integrate LangWatch graph's on your own systems and applications, to display it to your customers in another interface.
On LangWatch dashboard, you can use our powerful custom chart builder tool, to plot any data collected and generated by LangWatch, and customize the way you want to display it. You can then use our REST API to fetch the graph data.
**Usage:**
You will need to obtain your JSON payload from the custom graph section in our application. You can find this on the Analytics page > Custom Reports > Add chart.
1. Pick the custom graph you want to get the analytics for.
2. Prepare your JSON data. Make sure it's is the same format that is showing in the LangWatch application.
3. Use the `curl` command to get you analytics data. Here is a basic template:
```bash
# Set your API key and endpoint URL
API_KEY="your_langwatch_api_key"
ENDPOINT="https://app.langwatch.ai/api/analytics"
# Use curl to send the POST request, e.g.:
curl -X POST "$ENDPOINT" \
-H "X-Auth-Token: $API_KEY" \
-H "Content-Type: application/json" \
-d @- <
Within this modal, you'll find the JSON payload required for the precise custom analytics
data. Simply copy this payload and paste it into the body of your REST POST request.
Now you're fully prepared to access your customized analytics and seamlessly integrate
them into your specific use cases.
If you encounter any hurdles or have questions, our support team is eager to assist you.
---
# FILE: ./features/annotations.mdx
---
title: Annotations
description: Collaborate with domain experts using annotations
---
# Create annotations on messages
With annotations, you can add additional information to messages. This can be useful to comment on or add any other information that you want to add to a message for further analysis.
We have also implemented the option to add a scoring system for each annotation, more information about this can be found in the [Annotation Scoring](/features/annotations#annotation-scoring) section
If you want to add an annotation to a queue, you can do so by clicking on the add to queue button to send the messages to the queue for later analysis. You can create queues and add members to them on the the main annotations page. More information about this can be found in the [Annotation Queues](/features/annotations#annotation-queues) section.
## Usage
To create an annotation, follow these steps:
1) Click the message you want to annotate on and a [Trace](/concepts#traces) details drawer will open.
2) On the top right, click the annotation button.
3) Here you will be able to add a comment, a link or any other information that you want to add to the message.
Once you have created an annotation, you will see it next to to the message.
# Annotation Queues
To get started with annotation queues, follow these steps:
1) Go to the annotations page.
2) Click the plus button to create a new queue.
3) Add a name for your queue, description, members and click on the "Save" button.
Once you have created your queue, you will be able to select this when creating an annotation and send the messages to the queue or directly to a project member for later analysis.
Once you add an item to the queue, you can view it in the annotations section, whether it's in a queue or sent directly to you.
When clicking on a queue item, you will be directed to the message where you can add an annotation. Once happy with your annotation, you can click on the "Done" button and move on to the next item.
Once you’ve completed the final item in the queue, you’ll see that all tasks are done. That’s it! Happy annotating!
# Annotation Scoring
We have developed a customized scoring system for each annotation. To get started, you will need to create your scores on the settings page.
There are two types of score data you can choose from:
- **Checkbox**: To add multiple selectable options.
- **Multiple Choice**: To add a single selectable option.
After you have created your scores, you can activate or deactivate them on the settings page.
Once your scores are activated, you will see them in the annotations tab. For each annotation you create, the score options will be available, allowing you to add more detailed information to your annotations.
When annotating a message, you will see the score options below the comment input. Once you have added a score, you will be asked for an optional reason for the score.
Thats it! You can now annotate messages and add your custom score metrics to them.
---
# FILE: ./evaluations/custom-evaluator-integration.mdx
---
title: Instrumenting Custom Evaluator
description: Add your own evaluation results into LangWatch trace
---
If you have a custom evaluator built in-house which run on your own code, either during the LLM pipeline or after, you can still capture the evaluation results
and connect it back to the trace to visualize it together with the other LangWatch evaluators.
### Python
You can capture the evaluation results of your custom evaluator on the current trace or span by using the `.add_evaluation` method:
```python
import langwatch
@langwatch.span(type="evaluation")
def evaluation_step():
... # your custom evaluation logic
langwatch.get_current_span().add_evaluation(
name="custom evaluation", # required
passed=True,
score=0.5,
label="category_detected",
details="explanation of the evaluation results",
)
```
The evaluation `name` is required and must be a string. The other fields are optional, but at least one of `passed`, `score` or `label` must be provided.
### TypeScript
You can capture the evaluation results of your custom evaluator on the current trace or span by using the `.addEvaluation` method:
```typescript
import { type LangWatchTrace } from "langwatch";
async function llmStep({ message, trace }: { message: string, trace: LangWatchTrace }): Promise {
const span = trace.startLLMSpan({ name: "llmStep" });
// ... your existing code
span.addEvaluation({
name: "custom evaluation",
passed: true,
score: 0.5,
label: "category_detected",
details: "explanation of the evaluation results",
});
}
```
The evaluation `name` is required and must be a string. The other fields are optional, but at least one of `passed`, `score` or `label` must be provided.
### REST API
## REST API Specification
### Endpoint
`POST /api/collector`
### Headers
- `X-Auth-Token`: Your LangWatch API key.
### Request Body
```javascript
{
"trace_id": "id of the message the evaluation was run on",
"evaluations": [{
"evaluation_id": "evaluation-id-123", // optional unique id for identifying the evaluation, if not provided, a random id will be generated
"name": "custom evaluation", // required
"passed": true, // optional
"score": 0.5, // optional
"label": "category_detected", // optional
"details": "explanation of the evaluation results", // optional
"error": { // optional to capture error details in case evaluation had an error
"message": "error message",
"stacktrace": [],
},
"timestamps": { // optional
"created_at": "1723411698506", // unix timestamp in milliseconds
"updated_at": "1723411698506" // unix timestamp in milliseconds
}
}]
}
```
---
To get started with prompt versioning in the Optimization Studio:
1. Create a new workflow or open an existing one
2. Drag a signature node onto the workspace
3. Click on the node to access configuration options in the right side panel
4. Make your desired changes to the prompt configuration
5. Save your changes as a new version
### Managing Prompt Versions
Once you've created a prompt version, you can:
- Restore previous versions by clicking the time icon
- Rename your prompts for better organization
- Reuse prompts across different workflows
- View version history and track changes
### Prompt Management Table
The Prompt Management table provides a centralized location to manage all your prompts:
1. Access the prompt table from the main menu
2. Create new prompts or edit existing ones
3. Save different versions of your prompts
4. View all saved versions in a list format
5. See who created each version (team collaboration)
### Team Collaboration
All prompts and their versions are shared across your project team:
- Any prompt created in the table is available in the studio
- Changes made in the studio are reflected in the prompt table
- Team members can see who created each version
- Prompts can be reused across different workflows
### API Integration
LangWatch provides a comprehensive API for managing prompts programmatically. You can use the API to:
- List all prompts for a project
- Create new prompts
- Update existing prompts
- Manage prompt versions
- Delete prompts
To use the Prompts API, you'll need to authenticate using your LangWatch API key. For detailed information about the API endpoints and authentication, see the [Prompts API Reference](/api-reference/prompts/overview).
The API supports the following operations:
- `GET /api/prompts` - Get all prompts for a project
- `POST /api/prompts` - Create a new prompt
- `GET /api/prompts/:id` - Get a specific prompt
- `PUT /api/prompts/:id` - Update a prompt
- `DELETE /api/prompts/:id` - Delete a prompt
- `GET /api/prompts/:id/versions` - Get all versions for a prompt
- `POST /api/prompts/:id/versions` - Create a new version for a prompt
---
# FILE: ./features/batch-evaluations.mdx
---
title: Batch Evaluations
---
If you intend to conduct batch evaluations on the datasets you've created in LangWatch, we offer a Python SKD to facilitate this process. This guide aims to provide comprehensive instructions on leveraging our Python SDK to execute batch evaluations effectively.
### Usage
After adding records to your dataset, created within the dataset section of LangWatch, you can proceed to select the dataset for batch evaluation along with the desired evaluations. You have the option to choose from predefined evaluations or any custom evaluations you've set up in the Evaluation and Guardrails section of LangWatch.
### Screenshots examples
In the below in screenshot you will see the datasets section in LangWatch, you can get your batch evaluation python snippet by clicking on on the Batch Evaluation button.
In the below screenshot you will see where you can select the dataset you want to evaluate on as well as selecting which evaluations you would like to run. Each tab has different evaluation you can choose from.
In the screenshot below, you'll find a Python code snippet ready for execution to perform your batch processing. The parameters passed into the `BatchEvaluation` include your chosen dataset and an array of selected evaluations to run against it.
We've streamlined the process by setting up pandas for you, enabling seamless evaluation of datasets directly on the results object. This means you can leverage the power of pandas' data manipulation and analysis capabilities effortlessly within your evaluation workflow. With pandas at your disposal, you can efficiently explore, analyze, and manipulate your data to derive valuable insights without the need for additional setup or configuration.
### Python snippet
When executing the snippet, you'll encounter a callback function at your disposal. This function contains the original entry data, allowing you to run it against your own Large Language Model (LLM). You can utilize this response to compare results within your evaluation process.
Ensure that you return the `output` as some evaluations may require it. As you create your code snippet in the evaluations tab, you'll notice indications of which evaluations necessitate particular information. Utilize this guidance as a reference to kickstart your workflow effectively.
---
# FILE: ./features/triggers.mdx
---
title: Alerts and Triggers
description: Be alerted when something goes wrong and trigger actions automatically
---
## Create triggers based on LangWatch filters
LangWatch offers you the possibility to create triggers based on your selected filters. You can use these triggers to send notifications to either Slack or selected team email adresses.
#### Usage
To create a trigger in the LangWatch dashboard, follow these steps:
- Click the filter button located at the top right of the LangWatch dashboard.
- After creating a filter, a trigger button will appear.
- Click the trigger button to open a popout drawer.
- In the drawer, you can configure your trigger with the desired settings.
**Trigger actions**
Once the trigger is created, you will receive an alert whenever a message meets the criteria of the trigger. These trigger checks are run on the minute but not instantaneously, as the data needs time to be processed. You can find the created triggers under the Settings section, where you can deactivate or delete a trigger to stop receiving notifications.
**Trigger settings**
---
# FILE: ./features/embedded-analytics.mdx
---
title: Exporting Analytics
description: Build and integrate LangWatch graphs on your own systems and applications
---
## Export Analytics with REST Endpoint
LangWatch offers you the possibility to build and integrate LangWatch graph's on your own systems and applications, to display it to your customers in another interface.
On LangWatch dashboard, you can use our powerful custom chart builder tool, to plot any data collected and generated by LangWatch, and customize the way you want to display it. You can then use our REST API to fetch the graph data.
**Usage:**
You will need to obtain your JSON payload from the custom graph section in our application. You can find this on the Analytics page > Custom Reports > Add chart.
1. Pick the custom graph you want to get the analytics for.
2. Prepare your JSON data. Make sure it's is the same format that is showing in the LangWatch application.
3. Use the `curl` command to get you analytics data. Here is a basic template:
```bash
# Set your API key and endpoint URL
API_KEY="your_langwatch_api_key"
ENDPOINT="https://app.langwatch.ai/api/analytics"
# Use curl to send the POST request, e.g.:
curl -X POST "$ENDPOINT" \
-H "X-Auth-Token: $API_KEY" \
-H "Content-Type: application/json" \
-d @- <
Within this modal, you'll find the JSON payload required for the precise custom analytics
data. Simply copy this payload and paste it into the body of your REST POST request.
Now you're fully prepared to access your customized analytics and seamlessly integrate
them into your specific use cases.
If you encounter any hurdles or have questions, our support team is eager to assist you.
---
# FILE: ./features/annotations.mdx
---
title: Annotations
description: Collaborate with domain experts using annotations
---
# Create annotations on messages
With annotations, you can add additional information to messages. This can be useful to comment on or add any other information that you want to add to a message for further analysis.
We have also implemented the option to add a scoring system for each annotation, more information about this can be found in the [Annotation Scoring](/features/annotations#annotation-scoring) section
If you want to add an annotation to a queue, you can do so by clicking on the add to queue button to send the messages to the queue for later analysis. You can create queues and add members to them on the the main annotations page. More information about this can be found in the [Annotation Queues](/features/annotations#annotation-queues) section.
## Usage
To create an annotation, follow these steps:
1) Click the message you want to annotate on and a [Trace](/concepts#traces) details drawer will open.
2) On the top right, click the annotation button.
3) Here you will be able to add a comment, a link or any other information that you want to add to the message.
Once you have created an annotation, you will see it next to to the message.
# Annotation Queues
To get started with annotation queues, follow these steps:
1) Go to the annotations page.
2) Click the plus button to create a new queue.
3) Add a name for your queue, description, members and click on the "Save" button.
Once you have created your queue, you will be able to select this when creating an annotation and send the messages to the queue or directly to a project member for later analysis.
Once you add an item to the queue, you can view it in the annotations section, whether it's in a queue or sent directly to you.
When clicking on a queue item, you will be directed to the message where you can add an annotation. Once happy with your annotation, you can click on the "Done" button and move on to the next item.
Once you’ve completed the final item in the queue, you’ll see that all tasks are done. That’s it! Happy annotating!
# Annotation Scoring
We have developed a customized scoring system for each annotation. To get started, you will need to create your scores on the settings page.
There are two types of score data you can choose from:
- **Checkbox**: To add multiple selectable options.
- **Multiple Choice**: To add a single selectable option.
After you have created your scores, you can activate or deactivate them on the settings page.
Once your scores are activated, you will see them in the annotations tab. For each annotation you create, the score options will be available, allowing you to add more detailed information to your annotations.
When annotating a message, you will see the score options below the comment input. Once you have added a score, you will be asked for an optional reason for the score.
