Evaluation

4 stories tagged Evaluation on Learning Fanatic.

Lesson 5.1: Testing and comparing prompts with rubrics and test sets Learn to build a small test set, score outputs with a rubric, use a model as a grader carefully, and tell whether one prompt really beats another.
Lesson 5.2: Choosing and comparing models for your task Learn to compare models on your own task by quality, cost, speed and risk, work out cost per outcome, and route easy work to cheaper models.
Lesson 3.2: Writing instructions and system prompts for agents Learn to write agent instructions that work: clear structure, specific rules with reasons, the right place for each rule and tests that tell versions apart.
Lesson 4.3: Building with language models in a customer setting A demo that wows is not a system a customer can rely on. Learn to start simple, ground answers, test on real cases and keep a person in the loop.