Build better prompts, faster
What’s inside

Build better prompts, faster

Evalit is a lightweight Python toolkit to:

Manage versioned prompts with an easy local store
Run A/B tests between prompt variants
Evaluate performance beyond simple win rates

Quickstart

Install, run an experiment, and find the winning prompt.

Python SDK

Explore the PromptManager, Experiment, and Evaluator APIs.

What’s inside

Prompt management

Create, version, and activate prompts.

A/B testing

Compare variants with a controlled budget.

Evaluation model

IRT-inspired scoring using logistic regression.