World Monitor — real-time global intelligence dashboard with live markets, geopolitical data, and infrastructure monitoring

Forecast accuracy

Forecast accuracy scorecard

World Monitor logs every forecast it publishes and aims to score each one once its outcome is known. While the audit below is open, this page publishes no scores.

Under audit since 2026-10-07. An audit found three errors in how forecasts were scored. Some outcomes were recorded as "did not happen" without reading the data that decides them. Some forecasts were counted more than once. Some were scored at a probability other than the one published. The scores previously shown here were not reliable and are withdrawn while corrections are made. Forecasts are still being published and logged, and their outcomes will be rescored once the fixes land.

The findings and the fixes are tracked in issue #8990. The download below keeps the raw figures and marks them as under audit.

Methodology

This is the scoring method as documented. The audit found places where the scoring did not follow it.

Brier/log score over resolved YES/NO published forecast windows; VOID and pending entries are counted for coverage but excluded from accuracy math. The ledger totals and the maturity funnel count published forecasts only: shadow bets, scored for evidence but never shown, are left out of them. Each window is one question, and it is scored on the probability published when the window opened; a later re-emission of the same question does not change it. While an outcome-fitted calibration gate passes, that probability is calibrated, and the API does not mark which ones are; after a switch between raw and calibrated publication, the rolling window mixes forecasts published under both. 120 forecasts scored against a data feed we could not read correctly are voided and left out of every score (issue #5233). 322 shadow bet windows that opened on a base-rate placeholder instead of a model forecast are left out of every count, VOID included (issue #8990). Since 8 October 2026, 2 of 3 resolved published forecasts were void (66.7%, 95% interval 20.8% to 93.9%). That is too few to compare with the target of under 15%, which needs at least 30 resolved. Judged forecasts resolve days after hard ones and have voided more often, so early readings run low. This counts published forecasts first issued on or after that date, so it leaves out unpublished shadow bets and the voids relabelled in the issue #8990 audit.

Recently resolved forecasts, unverified

Not yet rechecked. These outcomes were recorded by the scoring system the audit found errors in, so some may be wrong. The 20 most recently resolved attributable published forecasts, newest first, leaving out experimental, synthetic and unattributed origins, voids included. The chance is the probability each forecast was scored at. This may differ from the probability first published. A hard forecast is settled by reading a World Monitor data feed; a judged one by AI judges reading archived news, and the linked item is one they cited. Dates are UTC.
ForecastMadeChance scoredOutcomeResolvedHow it was settled
Within the 30d horizon, did Ethiopia experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Active armed conflict: Ethiopia"?2026-09-1678%Happened2026-10-18Judged against archived news: Ethiopia launches drone attack against Eritrean troops who crossed into Tigray
Grain/Energy price impact from Kerch Strait disruption2026-09-1735%Did not happen2026-10-17Commodity prices read 666.5
Within the 7d horizon, did Syria experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Escalation risk: Syria"?2026-10-0856.8%Void2026-10-17The judges disagreed
GPS interference in Red Sea shipping zone2026-10-071.8%Happened2026-10-16GPS jamming reports read 3
Will "Military posture escalation: Korean Peninsula" (military, Korean Peninsula) resolve YES within its 7d horizon?2026-10-0841%Happened2026-10-16Judged against archived news: North Korea test-fires submarine-launched cruise missiles amid rising tensions
Within the 7d horizon after this forecast, did public threat-intelligence sources report at least 5 new malicious cyber threat indicators (malware hosts, command-and-control servers, phishing or scanning IPs) attributed to Canada?2026-10-089.8%Void2026-10-16Held out of scoring while the judges' evidence is being fixed
GPS interference in Baltic Sea shipping zone2026-10-0650%Happened2026-10-15GPS jamming reports read 340
GPS interference in Black Sea shipping zone2026-10-0650%Happened2026-10-15GPS jamming reports read 197
GPS interference in Eastern Mediterranean shipping zone2026-10-0650%Happened2026-10-15GPS jamming reports read 76
GPS interference in Persian Gulf shipping zone2026-10-0650%Happened2026-10-15GPS jamming reports read 85
Within the 7d horizon after this forecast, did public threat-intelligence sources report at least 2 new malicious cyber threat indicators (malware hosts, command-and-control servers, phishing or scanning IPs) attributed to Russia?2026-10-072.2%Void2026-10-15Held out of scoring while the judges' evidence is being fixed
Supply chain disruption: Kerch Strait2026-10-0635%Did not happen2026-10-14Shipping chokepoint status read 70
Supply chain disruption: Strait of Hormuz2026-10-0640.8%Happened2026-10-14Shipping chokepoint status read 85
Within the 7d horizon, did Germany see materially elevated malicious cyber activity versus its recent baseline, consistent with "Cyber threat concentration: Germany"?2026-10-0725%Void2026-10-14Held out of scoring while the judges' evidence is being fixed
Within the 7d horizon, did United States see materially elevated malicious cyber activity versus its recent baseline, consistent with "Cyber threat concentration: United States"?2026-10-0750%Void2026-10-14Held out of scoring while the judges' evidence is being fixed
Will "Military posture escalation: Middle East" (military, Middle East) resolve YES within its 7d horizon?2026-10-0572%Happened2026-10-13Judged against archived news: Oil Climbs as U.S. Sends Another Aircraft Carrier to Middle East - WSJ
Will "Military posture escalation: Northern Europe" (military, Northern Europe) resolve YES within its 7d horizon?2026-10-0572%Void2026-10-13Both judges found the evidence insufficient
Within the 7d horizon, did Ukraine experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Escalation risk: Ukraine"?2026-10-0580%Happened2026-10-13Judged against archived news: Russian strike kills at least 33 in eastern Ukraine in deadliest single attack this year - CNN
Within the 7d horizon, did China see materially elevated malicious cyber activity versus its recent baseline, consistent with "Cyber threat concentration: China"?2026-10-0434.6%Void2026-10-12Held out of scoring while the judges' evidence is being fixed
Within the 7d horizon, did Netherlands see materially elevated malicious cyber activity versus its recent baseline, consistent with "Cyber threat concentration: Netherlands"?2026-10-0450%Void2026-10-12Held out of scoring while the judges' evidence is being fixed

Market alerts: did the news follow?

The audit did not cover this section. Market alerts are scored from a separate ledger, and these figures have not been rechecked.

World Monitor raises a market alert when a market or a prediction market makes an unusual move. Some alerts fire when there is no news behind the move, and one type fires when related news is already out. Each alert is checked 6 hours later. It counts as a hit if an established news outlet published a new story about the same company, commodity or topic in that time. The same check also runs on the same market for a stretch of the same length one day earlier, when no alert was raised, and that gives the base rate. An alert type is useful only when its hit rate is clearly above the base rate on the same alerts. The figures cover the last 30 days and were generated 2026-10-19 05:05:09 UTC.

Market-alert hit rates by alert type. Each figure is shown once 30 alerts are behind it and reads Not yet measurable below that. The two comparison columns count only the alerts whose earlier window could also be checked, so both rates describe the same alerts. The typical wait is shown once 30 alerts were followed by news. Prediction-market alerts are shown only once their earlier windows have been checked, because the topic words in a prediction question turn up in the news often anyway.
Alert typeAlerts scoredNews followedNews followed, compared alertsNews a day earlier, same marketsTypical wait for the news (median)
Prediction market moved on a quiet news day1Not yet measurableNot yet measurableNot yet measurableNot yet measurable
Market moved alongside related news13327.8% of 133 alerts28.4% of 67 alerts31.3% of 67 earlier windows1 h 27 min
Market moved with no news found3851.3% of 385 alerts3.2% of 157 alerts3.2% of 157 earlier windowsNot yet measurable
Energy price rose with no pipeline news15Not yet measurableNot yet measurableNot yet measurableNot yet measurable

How an alert is scored

An emission resolves HIT when a story first tracked by the English news digest within six hours after the emission has at least one Tier 1 or Tier 2 source and its title names the same entity: for a market, an alias match for the symbol's registry entity or a whole-word match of the market name of four or more characters; for a prediction market, a topic keyword that appears in the question itself. It resolves MISS when six hours pass with no such story. It resolves VOID, and is not counted, when the story archive for its window expired before the resolver read it. Lead time is the gap between emission and the matching story's first tracking. The base rate applies the same rule to a control window of the same length for the same entity that starts 24 hours before each scored emission, counted only when the archive covers the control window, no alert for that entity was open during it, and the emission and the control fall on the same side of a weekend in UTC.

Open the live forecast panel in World Monitor →

Related reference

Download: scorecard.json. Source: World Monitor forecast scorecard snapshot. The raw figures in the download were generated 2026-10-18 06:00:29 UTC and read on 2026-10-19, and are under audit. Live results come from the credentialed forecast scorecard endpoint, which this page freezes so it can be read without one.