prime-radiant-inc/superpowers-evals
72 stars · Last commit 2026-07-11
Behavioral eval lab (Quorum) for the superpowers project that drives real coding-agent CLIs (Claude, Codex, Gemini, Kimi, and more) through a QA agent and grades them on workflow compliance against scenario criteria and deterministic post-checks.
README preview
# Superpowers Evals Behavioral eval lab for [superpowers](https://github.com/obra/superpowers). **Quorum** drives real coding-agent CLIs (Claude, Codex, Antigravity, Gemini, Kimi, OpenCode, Pi, and Copilot) through a Gauntlet QA agent and grades them against scenario acceptance criteria plus deterministic post-checks. Code, CLI, paths, and inline prose all use lowercase `quorum`; the capitalized form `Quorum` appears in headings and the actor table. This is not a generic benchmark suite. It is an eval lab for workflow compliance: skill triggering, worktree behavior, subagent coordination, verification reflexes, review quality, and cost-shaping patterns. ## Safety Model quorum has two very different execution modes: - **Static/unit checks** are safe for public CI. They run `biome`, `tsc`, and `bun test`. They do not call model APIs and do not launch agent CLIs.