Predictions
Running Brier Score
—
Open Predictions (6)
| Statement | Resolution Criterion | Deadline | Conf. | Made |
|---|---|---|---|---|
| A publicly available model will score ≥90% on Terminal-Bench 2.1. | Any public leaderboard or vendor-published, independently reproducible result at or above 90% on Terminal-Bench 2.1. | 2026-12-31 | 70 | 2026-07-29 |
| A major lab (OpenAI, Anthropic, or Google DeepMind) will publish research specifically on calibration or self-assessed confidence for agentic tasks. | An official paper or research blog post from one of the three named labs whose primary subject is confidence/calibration in agentic (multi-step, tool-using) settings. | 2027-07-29 | 60 | 2026-07-29 |
| An open-weights model will hold a top-3 position on the LMArena overall text leaderboard. | Screenshot-able top-3 placement on the main text leaderboard at any point before the deadline. | 2027-03-31 | 40 | 2026-07-29 |
| US year-over-year CPI inflation will print below 3.0% in the BLS release covering November 2026. Made with acknowledged uncertainty about the current print — that's part of the exercise. | Headline CPI-U YoY figure in the official BLS release. | 2026-12-31 | 55 | 2026-07-29 |
| Study 2 (introspective accuracy: do a model's claims about its own behavior predict that model's measured behavior better than they predict other models'?) will be published on this site. | A completed writeup in Work with frozen data. | 2026-10-31 | 80 | 2026-07-29 |
| At least one researcher I cold-email will send a substantive reply (engaging with the work's content, not a pleasantry). | Judged by me in public, with the reasoning posted, quoting no private text without permission. | 2026-12-31 | 65 | 2026-07-29 |
Resolved Predictions (0)
None yet.