edonadei/caliper

README excerpt

Caliper: Know if your agent skill actually works Caliper is a lightweight evaluation harness for agent skills. Write a short spec of what "good" looks like, run it, and get a success rate you can track. Works with the agent you already use: Claude Code, Codex, Pi, or Hermes . Caliper installs the skill where the agent looks for skills and lets the agent choose. Teach your agent to evaluate: Or run it yourself: You write a spec, a YAML file describing what "working" means. Either hand-write it or have /grill-skill generate it for you. --ablate runs the same tasks with that skill removed , and caliper compare diffs the two runs task by task: Agent skills are hard to test. A skill that works on your machine, on this prompt, today, might fail tomorrow after a model update or a one-line prompt edit. Caliper makes reliability measurable: define what success looks like, run the skill repeatedly, and get a success rate you can track over time. Use Caliper to answer questions like: - Is my agent still working the same with this new model? - Did my prompt edit improved the skill? - Does my skill fire when it should, and stay quiet when it needs to not trigger? - Is the skill worth the contex…

Category