53 tasks10 agents1,240 trials

The Open Benchmark for AI on Ethereum

We evaluate agents used by developers, and the models behind them, on real Ethereum work. Every task, trial and trajectory is public, allowing anyone to run the same tasks.

We evaluate three pillars: Concepts, Build, and Act.

Results

The same work, with and without Ethereum skills.

Compare configurations →

Measures how agents do Ethereum work with web access, the way developers use them

Configuration scores, sorted by overall descending
Configurationmodel · agent
Fable 5.2Claude Code · effort high
Opus 5.5Claude Code · effort high
GPT-6.2 SolCodex · effort high
GPT-6 AstraCodex · effort high
GPT-6.2 AstraCodex · effort high
GPT-6.1 SolCodex · effort high
Gemini 4 ArgonOpenCode · effort high
DeepSeek V4.1 FlashOpenCode · effort high
Kimi K3.1 ProOpenCode · effort high
GLM 5.3OpenCode · effort high
How to read it: Each row is an agent or a model, scored on three pillars: Concepts, Build, and Act. Overall is the average of the pillar scores. Click a pillar cell to see its tasks and every trial behind them.

Results by task

A detailed view of the Leaderboard. See how many trials each agent passed on every task, grouped by pillar.

How to add a task ↗

53 tasks · Internet

Passed trials / scored trials for each task and configuration
Task Fable 5.2Claude Code · effort high Opus 5.5Claude Code · effort high GPT-6 AstraCodex · effort high GPT-6.1 SolCodex · effort high GPT-6.2 AstraCodex · effort high GPT-6.2 SolCodex · effort high DeepSeek V4.1 FlashOpenCode · effort high Gemini 4 ArgonOpenCode · effort high Kimi K3.1 ProOpenCode · effort high GLM 5.3OpenCode · effort high
Concepts25 tasks
Accounts 1271 permit2 cross account replayconcepts/accounts-1271-permit2-cross-account-replay
Accounts 4337 unused gas penaltyconcepts/accounts-4337-unused-gas-penalty
Accounts 7739 typeddatasign domainconcepts/accounts-7739-typeddatasign-domain
Agent registriesconcepts/agent-registries
Compiler legacy init orderconcepts/compiler-legacy-init-order
Compiler memory bytes deleteconcepts/compiler-memory-bytes-delete
Compiler transient clear collisionconcepts/compiler-transient-clear-collision
Compiler tuple assignment self transferconcepts/compiler-tuple-assignment-self-transfer
Evm transfer refund to safeconcepts/evm-transfer-refund-to-safe
Evm try catch empty returnconcepts/evm-try-catch-empty-return
Forwarder multicall suffix oz5concepts/forwarder-multicall-suffix-oz5
Governance arbitrum block clockconcepts/governance-arbitrum-block-clock
Lending small position close factorconcepts/lending-small-position-close-factor
Permit2 7702 signatureconcepts/permit2-7702-signature
Permit2 zero expirationconcepts/permit2-zero-expiration
Signatures oz ecdsa signature as keyconcepts/signatures-oz-ecdsa-signature-as-key
Signatures permit2 signaturetransfer frontrunconcepts/signatures-permit2-signaturetransfer-frontrun
Signatures permit2 witness stringconcepts/signatures-permit2-witness-string
Signatures usdc contract permitconcepts/signatures-usdc-contract-permit
Smart accounts p256 precompileconcepts/smart-accounts-p256-precompile
Smart accounts userop hash v08concepts/smart-accounts-userop-hash-v08
Swaps permit2 zero expirationconcepts/swaps-permit2-zero-expiration
Upgrades stateless parent gapconcepts/upgrades-stateless-parent-gap
Wallet atomic readyconcepts/wallet-atomic-ready
Wallet batch status 600concepts/wallet-batch-status-600
Build25 tasks
Aave savings pagebuild/aave-savings-page
Audit 1inch swapvmbuild/audit-1inch-swapvm––––––––––
Audit automata dcapbuild/audit-automata-dcap––––––––––
Audit balancer v3 vaultbuild/audit-balancer-v3-vault––––––––––
Audit broadcaster proversbuild/audit-broadcaster-provers––––––––––
Audit cap labsbuild/audit-cap-labs––––––––––
Audit confidential contractsbuild/audit-confidential-contracts––––––––––
Audit intuitionbuild/audit-intuition––––––––––
Audit ion protocolbuild/audit-ion-protocol––––––––––
Audit onchainidbuild/audit-onchainid––––––––––
Audit pods vaultbuild/audit-pods-vault––––––––––
Audit taraxa bridgebuild/audit-taraxa-bridge––––––––––
Audit the compactbuild/audit-the-compact––––––––––
Audit thenabuild/audit-thena––––––––––
Audit trex v5build/audit-trex-v5––––––––––
Audit uniswap token launcherbuild/audit-uniswap-token-launcher––––––––––
Audit uniswap tribunalbuild/audit-uniswap-tribunal––––––––––
Creator coinsbuild/creator-coins
Keeper depositsbuild/keeper-deposits
Signed votesbuild/signed-votes
Swap pagebuild/swap-page
Token governorbuild/token-governor
Token launchpadbuild/token-launchpad
Token lending marketbuild/token-lending-market
Voting pagebuild/voting-page
Act3 tasks
Treasury eth saleact/treasury-eth-sale
Vault v2 concentration rebalanceact/vault-v2-concentration-rebalance
Vault v2 market exitact/vault-v2-market-exit
Trials passed per task. Click a scored cell for its instruction and trials. Errors are excluded from counts.