contexts
(retrieved documents) used by the LLM, you unlock several benefits in LangWatch:
- Specialized RAG evaluators (e.g., Faithfulness, Context Relevancy).
- Analytics on document usage (e.g., which documents are retrieved most often, which ones lead to better responses).
- Deeper insights into the retrieval step of your pipeline.
Manual RAG Span Creation
You can manually create a RAG span by usingtracer.withActiveSpan()
with type: "rag"
. Inside this span, you should perform the retrieval and then update the span with the retrieved contexts.
The contexts
should be a list of LangWatchSpanRAGContext
objects. The LangWatchSpanRAGContext
object allows you to provide more metadata about each retrieved chunk, such as document_id
, chunk_id
, and content
.
Here’s an example:
performRAG
usestracer.withActiveSpan()
withtype: "rag"
.- Inside
performRAG
, we simulate a retrieval step. span.setRAGContexts(ragContexts)
is called to explicitly log the retrieved documents.- The generation step (
generateAnswerFromContext
) is called, which itself can be another span (e.g., an LLM span).
Advanced RAG Patterns
Multiple Retrieval Sources
You can capture RAG contexts from multiple sources in a single span:RAG with Metadata
You can include additional metadata in your RAG contexts:Error Handling
When working with RAG operations, it’s important to handle errors gracefully and capture error information in your spans:Best Practices
- Use Descriptive Span Names: Name your RAG spans clearly to identify the retrieval method or source.
- Include Metadata: Add relevant attributes like retrieval method, thresholds, or source information.
- Handle Errors Gracefully: Wrap RAG operations in try-catch blocks and capture error information.
- Optimize Context Size: Be mindful of the size of context content to avoid performance issues.
- Use Consistent Document IDs: Use consistent naming conventions for document and chunk IDs.
- Control Data Capture: Use data capture configuration to manage what gets captured in sensitive operations.
Related Documentation
For more advanced RAG patterns and framework-specific implementations:- Integration Guide - Basic setup and core concepts
- Manual Instrumentation - Advanced span management for RAG pipelines
- Semantic Conventions - RAG-specific attributes and naming conventions
- LangChain Integration - Automatic RAG instrumentation with LangChain
- Capturing Metadata - Adding custom metadata to RAG contexts
For production RAG applications, combine manual RAG spans with Semantic Conventions for consistent observability and better analytics.