FINAL-Bench: VIDRAFT's Daily-Updated Global Weather Model Leaderboard Pits AI Against Traditional NWP on Equal Terms
TL;DR: VIDRAFT has launched FINAL-Bench, a daily-updated weather model leaderboard covering 24 cities across 6 continents that compares AI and traditional numerical weather prediction (NWP) under identical conditions using common-case methodology. The benchmark evaluates max temperature, min temperature, max wind speed, and precipitation across lead times of 1–7 days against ERA5 reanalysis data. Developers evaluating weather APIs or building forecast-dependent applications now have a persistent, rolling accuracy reference rather than a one-time publication snapshot.
What it is
FINAL-Bench is a publicly accessible weather model leaderboard introduced by VIDRAFT on November 13, 2026. Key structural facts:
- Coverage: 24 cities across 6 continents
- Variables tracked: Maximum temperature, minimum temperature, maximum wind speed, precipitation
- Lead times: 1-day through 7-day forecasts, evaluated separately per lead time
- Update cadence: Daily rolling — future cells are forecasts; a cell moves into verified history only after the corresponding ground-truth data arrives
- Verification baseline: ERA5 reanalysis data (not individual surface station observations)
- Error metric: RMSE (root mean squared error)
- Evaluation period shown at publication: February 1 – November 7, 2026
The leaderboard also documents VIDRAFT's own Darwin fusion forecast system alongside third-party models, and separately references a WeatherBench 2 comparison covering deterministic forecast settings from 2020.
The leaderboard code is released under Apache License 2.0. The compiled records and the model weights are under separate licensing terms — developers should check the original repository for specifics before assuming commercial reuse.
How it works
Common-case methodology is the core fairness mechanism. Rather than averaging each model's error over whatever dates it happened to produce output, FINAL-Bench only scores a city/date/lead-time triple when all models under comparison have a forecast for it. This prevents a model that skips difficult or data-sparse dates from appearing artificially accurate relative to one with fuller coverage.
Darwin fusion engine (VIDRAFT's own system) works at a conceptual level as follows:
- Ingest forecasts from multiple global NWP/AI sources
- Apply performance-weighted blending — weights derived from each upstream model's historical accuracy per lead time and region
- Apply localized corrections for temperature and wind speed, trained only on verification data available prior to each forecast issuance (no future data leakage, per the authors)
Darwin-27B decision core is a 27-billion-parameter model layered on top of raw forecast fields. Its role is explicitly downstream of the forecast itself: it converts meteorological variables into decision-relevant outputs — go/no-go signals, threshold-exceedance probabilities, or categorical likelihoods. This separation between forecast accuracy and decision quality is architecturally intentional and means you should evaluate the two stages independently.
ERA5 as the reference baseline is an important design choice to understand. ERA5 is a reanalysis product (observations fused with atmospheric modeling into a spatially consistent historical estimate), not raw station measurements. Leaderboard RMSE values therefore measure divergence from ERA5, which may differ from what a local sensor would record — especially relevant for localized extremes, orographic wind, or convective precipitation events.
Benchmarks & results
All numbers below are from the source article for the period Feb 1 – Nov 7, 2026, across the 24-city common-case set:
| Forecast variable | Lead time | Darwin MAE | Comparison model (AIFS) |
|---|---|---|---|
| Maximum temperature | 7-day | 2.09 °C | 2.41 °C |
| Precipitation | 1-day | 4.15 mm | 4.33 mm |
- Darwin is claimed by the authors to achieve lowest RMSE across all four variables at all lead times (1–7 days) within the 24-city common-case scope.
- Important caveat: VIDRAFT both develops Darwin and operates the leaderboard. These figures are author-reported and have not been independently co-verified by all compared meteorological agencies.
- For WeatherBench 2, the authors report Darwin's errors fall below the best values in the official WeatherBench 2 scoreboard across 15 cells (3 variables × 5 lead times, 2020 deterministic setting). Forecast files have been submitted to the WeatherBench 2 team for external review — submission is pending, not yet officially accepted.
- Global average RMSE improvement does not imply superiority for every region, season, or extreme-event type. Precipitation averages in particular can mask performance on high-impact rainfall days when most days are low-volume.
How to try it
The leaderboard dashboard and source code are publicly available. The source article references the Apache 2.0-licensed leaderboard repository via VIDRAFT's official release. At time of writing, specific Hugging Face model repo slugs, pip-installable packages, or OpenAI-compatible API endpoints are not confirmed in this source article. Developers should check:
- VIDRAFT's official GitHub organization for the FINAL-Bench leaderboard code
- The Darwin-Prediction-Weather system documentation linked from the November 13, 2026 announcement for data access and licensing terms
Do not assume model weights or API endpoints are publicly accessible until confirmed in official documentation.
FAQ
Q: Why use ERA5 instead of station observations as the verification baseline?
A: ERA5 provides a spatially and temporally consistent global grid, making it practical to compare models across 24 cities with a single reference. The tradeoff is that ERA5 is itself a modeled estimate, so leaderboard RMSE reflects agreement with that estimate — not necessarily with what a local instrument would read. For applications requiring hyperlocal accuracy, ERA5-based rankings are a starting point, not a final validation.
Q: Can I trust Darwin's self-reported #1 ranking since VIDRAFT runs both the model and the leaderboard?
A: Treat it as a reproducible, auditable claim rather than a certified result. The methodology (common cases, ERA5 baseline, per-lead-time separation, no-future-data correction policy) is documented and the data is public enough to re-derive. The WeatherBench 2 submission is pending external review. Independent replication by a third party would be the gold standard before making infrastructure decisions based solely on leaderboard position.
Q: Is Darwin-27B a replacement for the forecast engine, or a separate layer?
A: Architecturally separate. The 27B model processes forecast fields into decision outputs (thresholds, probabilities, go/no-go). Forecast RMSE and decision quality should be benchmarked independently — a low temperature RMSE does not automatically mean the downstream decision recommendations are calibrated for your specific application thresholds.
Q: How should I interpret the precipitation numbers compared to temperature numbers?
A: Don't cross-compare variables by raw RMSE magnitude — units differ (°C vs. mm). A smaller numerical error in precipitation does not mean precipitation forecasts are more reliable than temperature forecasts. Evaluate each variable's RMSE in its own context and against your application's specific action threshold.
Originally reported by AI郵報 AI Post Hub (중화권) (2026-11-15) — source article.
