Build better prompts, faster
Evalit is a lightweight Python toolkit to:- Manage versioned prompts with an easy local store
- Run A/B tests between prompt variants
- Evaluate performance beyond simple win rates
Quickstart
Install, run an experiment, and find the winning prompt.
Python SDK
Explore the PromptManager, Experiment, and Evaluator APIs.
What’s inside
Prompt management
Create, version, and activate prompts.
A/B testing
Compare variants with a controlled budget.
Evaluation model
IRT-inspired scoring using logistic regression.