Model performance
How the predictions have scored against actual results.
Winner called correctly
70.9%
3,673 matches, 1871-03-27 to 2026-08-15
Log loss
0.667
Lower is better; a confident wrong call costs most
Margin error
16.8 pts
Typical error in the predicted margin, 1871-03-27 to 2026-08-15
Recent form
78.3%
Weighted toward the last 90 days · an effective 59 matches
The last two years and the two before them are within sampling error of each other.
Calibration
When the model says 70%, does it happen 70% of the time?
| Model said | Matches | Predicted | Actual | Gap | Within margin of error |
|---|---|---|---|---|---|
| 50-60% | 917 | 54.9% | 55.2% | +0.3pp | Yes (±3.3pp) |
| 60-70% | 842 | 65.1% | 65.9% | +0.8pp | Yes (±3.3pp) |
| 70-80% | 778 | 74.9% | 77.5% | +2.6pp | Yes (±3.1pp) |
| 80-90% | 637 | 84.9% | 84.5% | -0.4pp | Yes (±2.8pp) |
| 90-100% | 369 | 93.5% | 94.3% | +0.8pp | Yes (±2.6pp) |
Calibration across the fixture list
The same check, grouped by how far apart the two ratings were — from the closest fixtures to the most lopsided.
| Rating gap | Matches | Predicted | Actual | Gap | Within margin of error |
|---|---|---|---|---|---|
| 0-37 | 368 | 55.1% | 59.5% | +4.4pp | Yes (±5.2pp) |
| 37-81 | 368 | 55.2% | 56.0% | +0.8pp | Yes (±5.2pp) |
| 81-125 | 368 | 58.5% | 56.8% | -1.7pp | Yes (±5.1pp) |
| 125-170 | 367 | 62.4% | 64.0% | +1.6pp | Yes (±5.1pp) |
| 170-219 | 367 | 67.1% | 68.4% | +1.3pp | Yes (±4.9pp) |
| 219-275 | 367 | 71.4% | 72.8% | +1.4pp | Yes (±4.7pp) |
| 275-345 | 367 | 75.8% | 76.0% | +0.2pp | Yes (±4.5pp) |
| 345-438 | 367 | 80.9% | 79.8% | -1.1pp | Yes (±4.1pp) |
| 438-599 | 367 | 86.0% | 87.5% | +1.5pp | Yes (±3.6pp) |
| 599-2856 | 367 | 90.9% | 88.3% | -2.6pp | Yes (±3.0pp) |
Does the model beat the rating?
Trained on matches before 2022-11-05 and scored on the 551 it never saw. The published forecast is the blend.
| Measure | Rating only | Blended (published) |
|---|---|---|
| Winner called correctly | 72.8% | 73.5% |
| Log loss | 0.571 | 0.563 |
| Brier score | 0.191 | 0.188 |
| Margin error (points) | 25.308 | 24.875 |
| Total error (points) | 16.617 | 16.617 |
Where it was most wrong
The matches where the favourite was surest and still did not win.
| Date | Match | Result | Favourite given |
|---|---|---|---|
| 2024-11-02 | Belgium v Switzerland | 48-7 | Switzerland 98% |
| 2024-10-16 | Paraguay v Colombia | 49-18 | Colombia 97% |
| 2019-10-12 | New Zealand v Italy | 0-0 | New Zealand 96% |
| 2024-11-16 | France v USA | 17-36 | France 96% |
| 2015-09-19 | South Africa v Japan | 32-34 | South Africa 94% |
| 2024-02-25 | France v Italy | 13-13 | France 94% |
| 2024-11-16 | France v Japan | 20-36 | France 93% |
| 2009-06-13 | New Zealand v France | 22-27 | New Zealand 93% |
| 2012-06-05 | Australia v Scotland | 6-9 | Australia 93% |
| 2022-03-19 | Wales v Italy | 21-22 | Wales 92% |
These figures score the rating engine alone. The holdout below scores the blended forecast that is actually published.
Trained on 3,673 matches up to 2026-08-15, last updated 8/19/2026. How it works explains what these figures measure.