Simulating Nations: Useful for Asking What If, Unfit for Predicting What Next

The problem

The Afghan republic fell in 10 days in August 2021¹. The inspector-general report that examined the collapse, published in November 2022, found that the Afghan government did not believe the United States would actually leave, which left it unprepared for the withdrawal¹. The misjudgment ran in the other direction as well. In a news report whose date is not shown, the Associated Press described US intelligence agencies as having underestimated Ukraine and its president and overestimated Russia, while correctly predicting that Vladimir Putin would order an invasion². The same report said the agencies had begun a review of how they judge a foreign government's "will to fight" and "capacity to fight"².

The difficulty is not that forecasting is hopeless. A peer-reviewed study of a government-funded forecasting tournament reported in its 2015 abstract that a group of "superforecasters" kept high accuracy across hundreds of questions on a wide array of topics (abstract only)³. The harder case is how a particular state, with its own leaders, history and incentives, would react to a particular shock. That is the problem this piece takes up.

What already works

Agents that play nations. A study presented at the ACM FAccT conference in June 2024 built simulations with eight nation agents, each driven by the same language model, which is software trained on large amounts of text to produce humanlike responses. It tested five models across three starting scenarios, ran 10 simulations for each combination, and let each simulation run for 14 turns of 14 days⁴. In a neutral scenario, one model's escalation score rose from 10.15 to 26.02, an increase of 256%⁴. Sudden jumps of more than 50% in a single turn appeared in some runs of two of the models, while a third showed significantly fewer⁴. Rare nuclear actions appeared as outliers, and a base model without safety training carried out nuclear strikes at an average of 2.48 per nation, against 7.38 messages sent⁴.

In plain terms, models given the role of a government tend to build up arms and escalate, and the escalations are hard to predict. The authors added that their results "should be viewed within the context of our particular methodology" and do not show how decision-making agents would act in general⁴.

Agents compared with human experts. A preprint with partly overlapping authors compared 214 national security experts, organised in 48 teams, with simulated teams in a fictional US-China crisis. Humans and models agreed on about half of 21 possible actions, and the frequency of actions matched on 16 for one model, 10 for another and 9 for a third⁵. The simulated players "rarely disagree", a pattern the authors call "farcical harmony", and the models showed no significant difference even for extreme traits such as "pacifist" or "aggressive sociopath"⁵.

Agents in simulated nuclear crises. A February 2026 preprint by a King's College London author ran three frontier models through 21 games and more than 300 turns. In its abstract, strategic nuclear attack is rare, threats more often provoke counter-escalation than compliance, and no model ever chose accommodation or withdrawal even under acute pressure, only reduced levels of violence⁶. In the 21 simulated crises, no model chose accommodation or withdrawal, even under acute pressure, only reduced levels of violence⁶. The same abstract says such simulation is useful only if calibrated against known patterns of human reasoning⁶.

Agents that copy individuals. A separate research group built agents from two-hour interviews with 1,052 people and tested them against what those same people answered in a social survey. Interview-based agents reached 0.83 of the accuracy the participants achieved when repeating their own answers two weeks later, against 0.74 for agents built only from demographic details⁷. On incentivised economic games the same paper found no significant differences between agent types and called the games "a boundary case"⁷.

For a non-specialist, the individual result is encouraging for predicting what a survey respondent would say, and the games result is a warning that strategic choices are harder.

What does not exist. No study was found that tests agents standing in for governments against real government behaviour, such as votes, responses to sanctions or crisis decisions. The nation-level work above shows how models behave, not whether states behave that way. A calibrated method for building a state agent, and a way of checking it against events it was not shown, would have to be invented.

The possibility

One way this could work is as a structured "what if" tool for analysts. A team would describe a crisis, build agents for each government from documented behaviour such as past statements, votes and decisions, run the crisis many times with small variations, and read the range of outcomes as a set of scenarios to examine, not as a prediction. The most useful output could be the surprising branches that a human team had not considered.

The role of AI is specific. The language-model part would be the agents themselves: generating a plausible statement or move in the voice of a government, given a scenario. Almost everything else is ordinary systems and human work: assembling and cleaning the record of past behaviour, designing scenarios, running many repetitions, recording results, and judging which outputs deserve attention. If the agents are replaced with simple rules or with human role-players, the surrounding process would still stand, which is a reason to ask what the language model adds.

This piece does not suggest that such a tool could say what a given state will do. The evidence reviewed above suggests that, at best, it could widen the set of possibilities that analysts examine.

What would have to be true

1. Calibration against something that really happened. The 2026 preprint says simulation is useful only if calibrated against known patterns of human reasoning⁶. For states, this would mean rebuilding an actual past crisis from the information available at the time and checking whether the agents produced the observed outcome among their range of results.
2. Validation on events the system was not shown. Calibration alone could fit the past. A blind test on later events would be needed, and none was found for state-level agents.
3. Agents that do not default to escalation or to agreement. Models escalated in simulated crises⁴⁶ and agreed with one another more than human experts did⁵. Both patterns would need to be measured and corrected, or the output would reflect the model's habits more than any government's.
4. Cultural coverage beyond one tradition. A study in PNAS Nexus reported in its abstract that all tested models showed cultural values resembling English-speaking and Protestant European countries, and that for later models, prompting with a cultural identity improved alignment for 71 to 81% of countries and territories (abstract only)⁸. The individual-simulation paper lists English-only data as a limitation⁷. A state agent built on thin coverage of a country's political culture could misread it.
5. A way to treat the output as a hypothesis. Outputs would need to be used to generate questions for analysts, not to settle them.

What could stop it

What the evidence shows

• The authors of the nation-agent study caution that their results should be read within their particular methodology and not as a guide to how high-stakes agents would behave in general⁴.
• In the 2026 preprint, no model chose accommodation or withdrawal⁶. A tool that cannot produce one of the most common real-world outcomes, a state backing down, would be unreliable for the cases that matter most.
• In the expert comparison, adding extreme persona traits did not significantly change model behaviour⁵. Agents that cannot represent a leader's character would struggle to represent a particular government's.
• The individual-simulation paper reports that economic games were a boundary case, that it tested one model family, and that data were English-only and US-based⁷.
• The study of the Afghan collapse shows how much turns on one set of actors misreading another's intentions¹, and the intelligence review shows that expert human assessment of other states has also missed².

Reasoning

The challenge question is what would make this less likely, less useful or more harmful than it first appears. Several answers can be reasoned without new facts.

There may be no way to test. A country-level simulation can be compared with one history, and each crisis happens once. If the simulation matches an event it was tuned on, that may show little; if it misses an event, there may be no way to tell whether the model, the data or chance is to blame.

States could change course when they know they are being modelled. If a government learned that adversaries run simulations of its behaviour, it could act differently, which would make a model built from past behaviour less reliable exactly when it is used.

The risk could fall on the wrong people. A tool that confirms what planners already believe, in fluent language and with many runs behind it, could add confidence without adding accuracy. The people harmed by a bad decision would not be those who chose to trust the simulation.

Incentives could work against honesty. A team under pressure to show a result could tune the agents until they produce an expected answer.

Simpler solutions could do the job. Structured human wargames, red teams and area experts already exist, and the human experts in the comparison study serve as the benchmark⁵. A simulation would need to show an advantage in cost, speed or range of scenarios before it replaces them, and the evidence reviewed does not yet show that.

Signs to watch

• A peer-reviewed version of the 2026 nuclear-crisis study, and replication with other models⁶.
• Publication of any findings from the US intelligence review of how agencies judge a foreign government's will and capacity to fight; no timetable was announced in the report seen².
• Blind back-tests in which state-level agents are scored against held-out historical events.
• Benchmarks or studies that test language models on political cultures outside the English-speaking world, following the cultural alignment work in PNAS Nexus⁸.
• Follow-up work that tests individual-level agents on strategic games with real stakes, building on the boundary case reported in⁷.