Why and when?
- You, the user, wants an LLM to do a task well (e.g. classification, ranking, code gen)
-
Before GEPA:
- Look at your data, understand the task details (hopefully label and build an eval)
- Repeat until satisfied:
- Write the prompt
- Run inference
- Analyze failure modes manually (maybe you have eval number)
-
After GEPA
- Look at your data, understand the task details (hopefully label and build an eval)
- Write a GEPA optimization loop by discussing with an agent
- That’s it :)
-
Why prompt tuning?
- Ease of use, for a given model instance, I can switch between many different prompts on the fly
- I can tune external models (Gemini, GPT-5)
- No train-inference mismatch, I can pick my SGL settings in advance (nvpf4, fp8 kv cache, specific kernels), and tune on that
- You can use reasoning
- This lowers throughput significantly, but can be necessary for hard problems, such as math correctness
- Not possible with embedding-based classifiers
Rough description of how GEPA works
Use-cases
- LLM quality classifier
- For web-text, you can use human labels for “readability”
- For STEM, you can tune on GPT5 rubric + GPT5 formatting critic
- For web-code, you can tune on human labels to give low/medium/high
- Reverse-engineering human preferences/priors that are under-specified
- You can label 100 samples (good/bad) , then tuned GPT5 to match human labels. You can then read the prompt to see if your priors can be made explicit.
- Bootstrapping LLM/VLM labelling for training an embedding classifier