← Toutes les offres
AH

A(nother) look at METR's time horizon: tasks of the same duration get solved months apart

Amaç Herdağdelen

Global
Publié le 17 novembre 2026

Description du poste

• METR's 80% time horizon lags the 50% horizon by about eight months. I had been erroneously reading that lag as a task's transition time, the time a task takes to go from a 50% to an 80% success rate.• However, in METR's data the median task's transition time is about four months, half the lag, because tasks of the same duration get solved at different times.• That spread among tasks of the same duration also matters for other readings of the horizons, for example:• For hazard rates, the pooled data can show a falling hazard rate even if every task has a constant one.• For labor displacement, if automating a role needs every task done reliably, the timing depends on the hardest tasks of each duration, which are solved later than the horizon suggests.This may be obvious to others, and in hindsight it is to me too, but I needed a closer look at the data to see it. The 80% horizon trails the 50% horizon by about eight monthsIn February 2025, the frontier model Claude 3.7 Sonnet had a 50% time horizon of one hour. The 80% horizon reached one hour about nine months later. Across the models since GPT-4 that moved the 50% horizon forward, the median gap is about eight months (90% interval 6.4 to 10.8). Model 50% horizon First model with an 80% horizon past it Estimated months later GPT-4 4 min o1-preview 17.7 GPT-4o 7 min o1 6.7 Claude 3.5 Sonnet (old) 11 min Claude 3.7 Sonnet 7.9 o1-preview 20 min o3 6.4 o1 39 min Gemini 3 Pro 8.2 Claude 3.7 Sonnet 60 min GPT-5.2 9.3 o3 2.0 h Claude Mythos Preview 10.8 The frontier 80% horizon is interpolated between model releases. The horizon is an average over tasksFor each AI model, METR fits one success curve across all tasks, with human completion time as the difficulty. A 50% horizon of two hours says that a randomly chosen two-hour task is completed half the time.[1] In practice, tasks of the same duration differ a lot in how hard they are for models. A familiar example is children learning to walk. One child in ten walks alone a