Skip to main content
LangWatch offers robust integration with Google Vertex AI, allowing you to capture detailed information about your Vertex AI API calls automatically. The recommended approach is to use OpenInference instrumentation, which provides comprehensive tracing for Google Vertex AI API calls and integrates seamlessly with LangWatch.

​
Using OpenInference Instrumentation

The recommended approach for instrumenting Google Vertex AI calls with LangWatch is to use the OpenInference instrumentation library, which provides comprehensive tracing for Google Vertex AI API calls.

​
What OpenInference Captures

The OpenInference Vertex AI instrumentation automatically captures:
  • LLM Calls: All text generation, chat completion, and embedding requests
  • Model Information: Model name, version, and configuration parameters
  • Input/Output: Prompts, responses, and token usage
  • Performance Metrics: Latency, token counts, and cost information
  • Error Handling: Failed requests and error details
  • Context Information: Session IDs, user IDs, and custom metadata

​
Installation and Setup

​
Prerequisites

  1. Install the OpenInference Vertex AI instrumentor: 
    pip install openinference-instrumentation-vertexai
  2. Install LangWatch SDK: 
    pip install langwatch
  3. Set up your Google Cloud credentials: 
    export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/service-account-key.json"
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"
The LangWatch API key is configured by default via the LANGWATCH_API_KEY environment variable.

​
Basic Setup

There are two main ways to integrate OpenInference Vertex AI instrumentation with LangWatch: You can pass an instance of the VertexAIInstrumentor to the instrumentors list in the langwatch.setup() call. LangWatch will then manage the lifecycle of this instrumentor. 
import langwatch
from vertexai.language_models import TextGenerationModel
import os

# Example using OpenInference's VertexAIInstrumentor
from openinference.instrumentation.vertexai import VertexAIInstrumentor

# Initialize LangWatch with the VertexAIInstrumentor
langwatch.setup(
    instrumentors=[VertexAIInstrumentor()]
)

# Initialize Vertex AI
from vertexai import init
init(project=os.getenv("GOOGLE_CLOUD_PROJECT"), location=os.getenv("GOOGLE_CLOUD_LOCATION"))

model = TextGenerationModel.from_pretrained("text-bison@001")

@langwatch.trace(name="Vertex AI Call with OpenInference")
def generate_text_with_openinference(prompt: str):
    # No need to call autotrack explicitly, the OpenInference instrumentor handles Vertex AI calls globally.
    response = model.predict(prompt)
    return response.text

if __name__ == "__main__":
    user_query = "Tell me a joke about Python programming."
    response = generate_text_with_openinference(user_query)
    print(f"User: {user_query}")
    print(f"AI: {response}")

​
2. Direct Instrumentation

If you have an existing OpenTelemetry TracerProvider configured in your application, you can use the instrumentor’s instrument() method directly. LangWatch will automatically pick up the spans generated by these instrumentors as long as its exporter is part of the active TracerProvider. 
import langwatch
from vertexai.language_models import TextGenerationModel
import os
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter

from openinference.instrumentation.vertexai import VertexAIInstrumentor

langwatch.setup()

# Initialize Vertex AI
from vertexai import init
init(project=os.getenv("GOOGLE_CLOUD_PROJECT"), location=os.getenv("GOOGLE_CLOUD_LOCATION"))

model = TextGenerationModel.from_pretrained("text-bison@001")

# Instrument Vertex AI directly using the OpenInference library
VertexAIInstrumentor().instrument()

@langwatch.trace(name="Vertex AI Call with Direct OpenInference Instrumentation")
def get_story_ending(beginning: str):
    response = model.predict(
        f"You are a creative writer. Complete the story: {beginning}"
    )
    return response.text

if __name__ == "__main__":
    story_start = "In a land of dragons and wizards, a young apprentice found a mysterious map..."
    ending = get_story_ending(story_start)
    print(f"Story Start: {story_start}")
    print(f"AI's Ending: {ending}")

​
Which Approach to Choose?

  • OpenInference Instrumentation is recommended for most use cases as it provides comprehensive, automatic instrumentation with minimal setup
  • Direct OpenTelemetry Setup is useful when you need fine-grained control over the tracing configuration or are already using OpenTelemetry extensively
Both approaches effectively send Vertex AI call data to LangWatch for monitoring and analysis.