Seldon Vault turned Asimov's psychohistory into a multi-agent AI forecasting service

Seldon Vault — a free multi-agent service that publishes forecasts of world events daily and then checks how accurate they turned out to be. The project is inspired by psychohistory from Isaac Asimov's "Foundation," but built from very real components: LLM, news streams, Bayesian updates, and open datasets.

How Seldon Vault Works

The service operates not like a single "oracle," but as a small analytical bureau. First, it collects signals from open sources, then a separate processor filters out noise and decides what counts as urgent news versus a long-term structural trend. At this first stage, a cheaper model is used so resources aren't wasted on everything indiscriminately and the stream of meaningless hypotheses doesn't bloat. Such a filter is needed so that fresh news doesn't get the same weight as a random plant.

• global media, Reddit, Telegram, and Bluesky
• prediction markets Polymarket and Metaculus
• macro data from FRED and the Fear & Greed index
• databases of conflicts, disasters, and geopolitical events

After filtering, signals go straight to seven analysts: one looks at geopolitics, another at economics, a third at technology, then sociology, climate, military analytics, and cybersecurity join in. They interpret the same news story differently and then offer their forecasts with probabilities. A final arbiter collects the top-5 conclusions from this set and publishes bilingual forecast cards with probability estimates ranging from 5% to 95%.

Skeptic, Cascades, and Metrics

The most interesting part of the architecture is a separate skeptic agent. Its job is not to help analysts but to break their conclusions: find counterexamples, verify facts through external search, and spot weak points in reasoning. In the article, the author calls this institutionalized doubt — the idea that it's more useful for a system to first prove itself wrong than to prematurely declare itself right. Effectively, it's a built-in self-criticism mechanism that cuts beautiful but weak analysis before it reaches the final feed.

The analyst must search for reasons why he is wrong before reporting

to management that he is right.

Another idea in Seldon Vault is cascading narratives. If the system sees several related forecasts, it builds a cause-and-effect chain: for example, new export sanctions might lead to a chip shortage, production delays, and cooling in adjacent markets. When the first event in such a chain comes true, the probabilities of the rest are automatically recalculated. To prevent one trigger from destabilizing the entire model, influence decays at each step and is limited to a few levels of depth.

Forecasts that survive this check aren't frozen forever. Every six hours, the service runs a new cycle, reviews probabilities using Bayesian logic, and limits daily shifts so it doesn't react hysterically to every headline. In parallel, the system calculates Brier Score — a baseline metric for the accuracy of probabilistic forecasts — and accumulates statistics for each agent. This feedback returns to the prompts so models calibrate confidence better over time.

Where the System Stumbles

The project author directly writes that there are plenty of weak points. The first problem is LLM hallucinations: the model can confidently cite an event that never happened, and if external search doesn't catch the error, it lands in the final forecast. The second is the habit of models and people to drift toward the safe zone of 45–55%. Formally it looks neat, but in practice too many "50%" turn forecasting into a polite way of saying "I don't know."

There are also more fundamental limitations. Black swans are by definition poorly suited to prediction based on historical patterns, and news from Reddit or Telegram show not reality itself but someone's already-filtered version of what's happening. So even a rich set of sources doesn't guarantee objectivity. The author himself honestly admits: the service was just launched, and only after several months will the accumulated Brier Score show whether this scheme can catch trends better than chance.

What It Means

Seldon Vault is interesting not for the promise to "predict the future," but for the attempt to turn LLM forecasts into a verifiable system with roles, conflict of opinions, and a quality metric. If such an approach survives the first months and maintains adequate accuracy, analytical teams will gain a useful tool for monitoring risks and weak signals. For the market, this matters more than yet another chatbot with confident but unverifiable answers in corporate analytics and media.