Unit Tests

LangEvals is a tool for developers by developers. And we, as developers, strongly believe that software needs to be tested, including the LLMs.

You can easily integrate LangEvals with PyTest and leverage the power of evaluators combined with reproducible testing. This integration provides extensive coverage of edge cases and ensures an increased level of certainty in your chatbot’s behavior.

​

Simple Assertions

For straightforward cases where the expected output is clear, such as extracting entity from text, LangEvals can help you assert the correctness of the output. This method is perfect for tests where outputs are unambiguous and easily comparable to expected results.

​

Unit Test Helpers

You can easily combine LangEvals with such decorators as parametrize, flaky and pass_rate to grow your single test case in an extensive comparison accross models with specific pass rates.

​

Setting Up CI/CD

Integrating LangEvals and PyTest into your CI/CD pipeline ensures continuous validation of your models. By running tests automatically on each commit, you can detect issues early and maintain model performance and reliability. This setup streamlines the development process and helps deliver robust, well-tested LLM applications.

​

Examples and Tutorials

For more detailed examples and tutorials on writing unit tests and using LangEvals in your projects, please refer to our Tutorials. There, you will find practical examples demonstrating how to use LangEvals with PyTest for various testing scenarios, from simple assertions to advanced evaluations using custom and out-of-the-box evaluators.