How high will AI models score on SWE-Bench in 2025?

How high will AI models score on SWE-Bench in 2025?

At least 60%

Yes

At least 60%

Yes

At least 70%

Yes

At least 70%

Yes

At least 80%

No

At least 80%

No

At least 90%

No

At least 90%

No

Market Rules

At least 60%

The outcome is Yes

Resolves Yes if a model or set of models scores at least 60 % on the Verified SWE-Bench leaderboard after December 19, 2024 and before Jan 1, 2026.Outcome verified from SWE-Bench.

Timeline and payout

Market openDec 20, 2024 · 10:00am EST

Market closedAfter the outcome occurs

Paid out30 minutes after closing

This market will close and expire early if the event occurs. Otherwise, it closes by Jan 9, 2025 at 2:06pm EST.

SeriesKXSWEBENCH

EventKXSWEBENCH-26

MarketKXSWEBENCH-26-60

Insider trading is prohibited

The following are prohibited from trading this contract:

Persons who are employed by any of the Source Agencies are not permitted to trade on the Contract.
Persons who hold any material, non-public information on the Underlying are not permitted to trade on the Contract.