How It Works
- Create variants as different versions of the same prompt
- Switch between versions at runtime with an A/B testing strategy
- Track performance using LangWatch’s built-in analytics
- Compare results to see which version performs better
Implementation
Create Prompt Variants
Create different versions of your prompt for testing:- TypeScript SDK
- Python SDK
Run A/B Tests
Use the captured version numbers to switch between prompt versions at runtime (random sampling):- TypeScript SDK
- Python SDK
Track Performance
LangWatch automatically tracks performance metrics for each prompt version:- Response latency - Which version is faster?
- Token usage - Which version is more efficient?
- Cost per request - Which version is more cost-effective?
- Quality scores - Which version produces better responses?