Seldon Vault Turns Asimov's Psychohistory into a Multi-Agent AI Forecasting Service

Seldon Vault is a free multi-agent service that publishes forecasts of world events every day and then checks how accurate they turned out to be. The project is inspired by the psychohistory of Isaac Asimov's Foundation, but it is built from very real components: LLMs, news feeds, Bayesian updates and open datasets.

How Seldon Vault works

The service does not work as a single "oracle" but as a small analytical bureau. First it collects signals from open sources. A separate processor then filters out noise and decides what counts as breaking news and what is a long-term structural trend. This first stage uses an inexpensive model, so resources are not spent on everything indiscriminately and the stream of meaningless hypotheses does not balloon. The filter exists so that fresh news does not get the same weight as a random plant.

• world media, Reddit, Telegram and Bluesky
• prediction markets Polymarket and Metaculus
• FRED macro data and the Fear & Greed index
• databases of conflicts, disasters and geopolitical events

After filtering, the signals go to seven analysts at once: one looks at geopolitics, another at the economy, a third at technology, followed by sociology, climate, military analysis and cybersecurity. They interpret the same news event differently, then offer their own forecasts with probabilities. A final arbiter assembles the top 5 conclusions from this pool and publishes bilingual forecast cards with a probability estimate ranging from 5% to 95%.

The skeptic, cascades and metrics

The most interesting part of the architecture is a dedicated skeptic agent. Its job is not to help the analysts but to break their conclusions: look for counterexamples, check facts through external search and find weak points in the reasoning. In the article, the author calls this institutionalized doubt, the idea that it is more useful for a system to first try to prove itself wrong than to declare itself right prematurely. In effect, it is a built-in self-criticism mechanism that cuts down attractive but weak analysis before it reaches the final feed.

An analyst must look for reasons why he is wrong before reporting to

his superiors that he is right.

Another idea in Seldon Vault is cascading narratives. When the system sees several related forecasts, it builds a cause-and-effect chain: for example, new export sanctions could lead to a chip shortage, production delays and a cooling of adjacent markets. When the first event in such a chain comes true, the probabilities of the others are recalculated automatically. So that a single trigger does not swing the whole model, the influence decays at each step and is limited to a few levels of depth.

Forecasts that survive this check are not frozen forever. Every six hours the service runs a new cycle, revises probabilities using Bayesian logic and caps the daily shift, so that it does not react hysterically to every loud headline. In parallel, the system calculates the Brier Score, a basic accuracy metric for probabilistic forecasts, and accumulates statistics for each agent. This feedback returns to the prompts so that the models calibrate their confidence better over time.

Where the system stumbles

The project's author says outright that there are plenty of weak points. The first problem is LLM hallucination: a model can confidently cite an event that never happened, and if external search does not catch the error, it will end up in the final forecast. The second is the habit of models and people to drift toward the safe 45–55% zone. Formally it looks tidy, but in practice too many "50%" answers turn forecasting into a polite way of saying "I don't know."

There are also more fundamental limitations. Black swans, by definition, are hard to predict from historical patterns, and news, Reddit or Telegram show not reality itself but someone's already filtered version of events. So even a rich set of sources does not guarantee objectivity. The author honestly admits that the service launched very recently, and only after several months will the accumulated Brier Score show whether this scheme can catch trends better than chance.

What this means

Seldon Vault is interesting not for a promise to "predict the future" but for its attempt to turn LLM forecasts into a verifiable system with roles, a clash of opinions and a quality metric. If this approach survives the first months and keeps reasonable accuracy, analytical teams will get a useful tool for monitoring risks and weak signals. For the market this matters more than yet another chatbot giving confident but unverifiable answers in corporate analytics and media.