How much does your AI actually know about Ethereum?

Models, agents, skills and harnesses, scored on the four pillars of Ethereum knowledge: concepts, transactions, building and security. Every eval targets something LLMs get wrong today. Bring your own AI.

Results

One card per configuration. A model alone and the same model with a skill installed are two different cards, because the gap between them is the point. Scores are out of 20 per pillar.

Loading results…

Run it on your AI

Point any agent at the suite and it runs itself: executes the evals, has a blind judge grade them, and prints a card like the ones above. Works in Claude Code, Codex, or OpenCode for open models.

The evals

Twenty per pillar, eighty total. Each one is keyed to a mistake seen in real agent output. quiz is a question with a written answer. goal is a build task judged on the files left behind. deterministic evals have a computed answer and a regex or exact grader. judge evals are graded by a blind LLM against written conditions. Ported evals link to their source task.

Sources

• BuidlGuidl/ethskills-evals — goal and quiz tasks with a blind LLM judge, one per ethskills claim. Most of the building and security evals come from here.
• clawdbotatg/eth-evals — 242 closed-book tasks, deterministic graders, computed calldata. Most of the concepts and transactions evals come from here.
• ethskills.com — the skill library these evals grade against. Every eval names the claim it tests.