Alignment Forum Proposes 4 Steps: Next-Generation AI Only After Vetting the Current One

A short but telling post has appeared on the AI Alignment Forum about what a minimum standard for the pace of AI development should look like if the industry agrees not to a full pause, but only to a slowdown. The thesis is simple: a lab has no right to train the next generation of models, in terms of capabilities, until it has resolved all significant alignment problems in the current one. The author lays out a process in four stages and calls it a minimum, not a sufficient condition.

The logic of the proposal

The analogy comes from aviation. You cannot launch an aircraft that flies at Mach 3 until you understand why the machine crashes at Mach 2. With any other dangerous technology, nobody would accept a "shrug and move on" approach. In a footnote, the author concedes that the cause of the Mach 2 crashes might stop being a problem at Mach 3, but says it would be insane to assume so.

The author also stresses that the formula suits both optimists and pessimists about "prosaic" alignment, meaning refinement of current methods. The optimists expect that a moderate slowdown is enough to close the problems at each generation. The pessimists expect the requirement to lead to a prolonged pause and give time to search for fundamentally new approaches.

The four stages of the cycle

• Audit during training. Several independent alignment organizations check and red-team the new model across many checkpoints, with full access to data and the best available tools. Problems are taken seriously even when there is no metric and something is simply "qualitatively off," and even when a good property cannot be reliably proven. Early problems do not necessarily block the process, but a convincing safety case is needed for why the later stages of training will eliminate them.
• Limited launch. The model is available only to a group of trusted users, with tasks that do not affect training and monitoring. The organizations watch for new problems.
• Broad deployment, internal or external. Allowed only after a significant period of limited launch without incidents.
• Next generation. Training starts only after prolonged operation without significant problems.

If a serious problem arises at the second or third stage, the scheme is strict: the model is pulled, a root-cause analysis is carried out, and the team establishes why the problem was not caught during training. Then they look for a way to treat the disease, not the symptom, train the next model of the same generation, and start over.

An important detail from a footnote: "better metrics than previous models" does not count as an adequate criterion. If, after launch, a user points to strange but alarming behavior that was not noticed earlier, that is a critical failure of both the model and the process.

What was left out

The author admits that the proposal is a heuristic, not a concrete set of rules. He names four obvious gaps:

• how to define a "model generation" within a single lab and across different ones;
• what to do if a fix for an alignment problem also noticeably raises capabilities;
• how to formalize and verify compliance with the agreement if it cannot rely only on predefined metrics;
• where the severity threshold sits at which a model is stopped, given that a lab will have an incentive to set that threshold too high.

These are the main points of failure. Without a shared definition of a generation, any model can be called an "intermediate version." And when the lab itself sets the severity threshold, the process turns into self-certification.

Practical takeaway

The proposal has no legal force and for now remains a post on a specialist forum. But it is useful as a template of questions worth asking labs. Do you have external auditors with access to training data? How long does the limited launch last, and what counts as an incident? What exactly will happen to the release schedule if a problem is found after launch? Regulators and corporate customers can build such questions into vendor requirements right now, without waiting for an industry-wide agreement. For the labs themselves, it is a benchmark by which an outside observer will judge how seriously they take their own safety claims.