Model performance
How the predictions have scored against actual results.
Winner called correctly
73.5%
9,287 matches, 1871-03-27 to 2026-10-17
Log loss
0.618
Lower is better; a confident wrong call costs most
Margin error
21.5 pts
Typical error in the predicted margin, 1871-03-27 to 2026-10-17
Recent form
78.8%
Weighted toward the last 90 days · an effective 134 matches
The last two years and the two before them are within sampling error of each other.
Calibration
When the rating says 70%, does it happen 70% of the time? This table and the two after it score the rating engine on its own, each match predicted from the ratings held before kick-off. The published forecast adds a capped adjustment on top, and is checked against the held-out matches further down.
| Model said | Matches | Predicted | Actual | Gap | Within margin of error |
|---|---|---|---|---|---|
| 50-60% | 2204 | 55.0% | 56.4% | +1.4pp | Yes (±2.1pp) |
| 60-70% | 2033 | 65.0% | 67.5% | +2.5pp | No (±2.1pp) |
| 70-80% | 2039 | 75.0% | 78.7% | +3.7pp | No (±1.9pp) |
| 80-90% | 1788 | 84.8% | 89.0% | +4.2pp | No (±1.7pp) |
| 90-100% | 891 | 93.3% | 96.2% | +2.9pp | No (±1.7pp) |
Calibration across the fixture list
The same check, grouped by how far apart the two ratings were — from the closest fixtures to the most lopsided.
| Rating gap | Matches | Predicted | Actual | Gap | Within margin of error |
|---|---|---|---|---|---|
| 0-43 | 929 | 58.6% | 60.6% | +2.0pp | Yes (±3.2pp) |
| 43-89 | 929 | 58.7% | 60.3% | +1.6pp | Yes (±3.2pp) |
| 90-136 | 929 | 59.4% | 62.0% | +2.6pp | Yes (±3.2pp) |
| 136-183 | 929 | 62.3% | 64.4% | +2.1pp | Yes (±3.2pp) |
| 183-237 | 929 | 67.0% | 70.2% | +3.2pp | No (±3.1pp) |
| 237-297 | 929 | 71.4% | 75.9% | +4.5pp | No (±3.0pp) |
| 297-369 | 929 | 75.5% | 79.0% | +3.5pp | No (±2.8pp) |
| 369-460 | 928 | 80.6% | 83.0% | +2.4pp | Yes (±2.6pp) |
| 460-619 | 928 | 85.4% | 89.9% | +4.5pp | No (±2.3pp) |
| 619-2084 | 928 | 89.2% | 89.9% | +0.7pp | Yes (±2.0pp) |
Calibration against the record behind it
The same check once more, grouped by how many matches the thinner of the two sides had played. A rating built on three results is a number the record cannot pin down, and the rankings table withholds a place for that reason. This asks whether the rating deserved the confidence it carried.
| Matches on record | Matches | Predicted | Actual | Gap | Within margin of error |
|---|---|---|---|---|---|
| 0-19 | 1573 | 70.3% | 71.3% | +1.0pp | Yes (±2.3pp) |
| 20-59 | 1932 | 70.4% | 73.0% | +2.6pp | No (±2.1pp) |
| 60-149 | 2364 | 70.5% | 73.8% | +3.3pp | No (±1.9pp) |
| 150 or more | 3418 | 71.4% | 74.6% | +3.2pp | No (±1.5pp) |
The 20-59 and 60-149 and 150 or more bands fall outside the sampling error, the favourite winning more often than the rating claimed.
Does the model beat the rating?
Trained on matches before 2018-11-17 and scored on the 1394 it never saw. The published forecast is the blend.
| Measure | Rating only | Blended (published) |
|---|---|---|
| Winner called correctly | 77.0% | 76.5% |
| Log loss | 0.498 | 0.493 |
| Brier score | 0.165 | 0.162 |
| Margin error (points) | 22.200 | 21.263 |
| Total error (points) | 16.388 | 16.388 |
Calibration of the published forecast
The calibration tables above score the rating alone. This one scores the blend, over the same held-out matches, and it is the one to hold a published percentage to.
| Forecast said | Matches | Predicted | Actual | Gap | Within margin of error |
|---|---|---|---|---|---|
| 50-60% | 243 | 54.6% | 55.1% | +0.5pp | Yes (±6.4pp) |
| 60-70% | 257 | 64.5% | 68.9% | +4.4pp | Yes (±6.0pp) |
| 70-80% | 264 | 74.8% | 72.0% | -2.8pp | Yes (±5.3pp) |
| 80-90% | 293 | 85.4% | 85.7% | +0.3pp | Yes (±4.1pp) |
| 90-100% | 309 | 95.5% | 94.5% | -1.0pp | Yes (±2.4pp) |
Where it was most wrong
The matches where the favourite was surest and still did not win.
| Date | Match | Result | Favourite given |
|---|---|---|---|
| 1993-10-31 | Luxembourg v Georgia | 10-10 | Georgia 98% |
| 2011-07-17 | Australia v Samoa | 23-32 | Australia 94% |
| 2024-02-25 | France v Italy | 13-13 | France 94% |
| 2016-02-27 | Brazil v USA | 24-23 | USA 94% |
| 2009-06-13 | New Zealand v France | 22-27 | New Zealand 94% |
| 2024-11-16 | France v USA | 17-36 | France 94% |
| 2018-04-28 | Ukraine v Hungary | 19-19 | Ukraine 93% |
| 2005-09-10 | Serbia v Bosnia & Herzegovina | 22-22 | Serbia 93% |
| 2012-06-05 | Australia v Scotland | 6-9 | Australia 92% |
| 2018-09-15 | New Zealand v South Africa | 34-36 | New Zealand 92% |
The backtest figures score the rating engine alone. The forward holdout scores the blended forecast that is actually published.
The adjustment heads were fitted on 7,429 matches to 2016-11-19, and the 1,858 after them, to 2026-10-17, decided when fitting stopped. Last updated 19/10/2026. How it works explains what these figures measure.