Description du poste
Since DeepSeek R1 came out, many people have felt that Chinese models are catching up to the US frontier. Using a new benchmark, Frontier Clock, I find the opposite: while noisy, the US-China gap has grown by around 2.4 months per year.
Inspired by Epoch AI's “Chinese AI models have lagged the US frontier by 7 months on average since 2023” and Parv Mahajan's “The US-China AI gap is very uncertain,” I built an AI benchmark that measures the gaps between the US, China, non-China open weights, and individual labs. Instead of giving each model a traditional benchmark score, Frontier Clock assigns a “frontier-equivalent date”: the date when the world’s best publicly released model first had the same capability level as that model. It uses Epoch AI’s ECI benchmark data. The “Method” section in frontierclock.org discusses how the data is used.
This graph from the website plots each model’s frontier-equivalent date on the y-axis and release date on the x-axis:
Frontier Clock classifies models into three groups: “US Closed,” “China,” and “Open weights (non-China).” For each group, it displays the “gap to the frontier,” how many months each group’s best model is behind the world's best publicly released model, over time, from the ECI data.
The faint blue horizontal line over the X-axis represents the “US Closed” group. Closed-weight US models have always been at the frontier, so the value is always 0 months. The red line is China, and the teal line is non-Chinese open-weight models.
The red Chinese-model line is noisy, but the trendline shows it is increasing over time by 2.4 months per year. There is no sign that China is catching up; if anything, the US-China gap is growing. This trend has been discussed before.
How confident are we about this? Frontier Clock estimates uncertainty by randomly resampling across Epoch’s 47 benchmarks and fitting 90% confidence intervals. These intervals only capture uncertainty from which benchmarks are selected, so the true uncertainty is pr