AutoIndexer Teaches a Language Model to Edit Its Own Output with Three Types of Edits

Researchers have published AutoIndexer, a generative model that can go back to text it has already written and change it with three operations: inserting, replacing and deleting tokens. Instead of moving strictly left to right, the model works like a text editor with a cursor. The preprint 2610.12545 appeared in arXiv cs.CL on 12 October 2026, and the code is in the open repository AutodeskAILab/autoindexer.

The underlying problem is familiar to anyone who has worked with LLMs. An autoregressive model emits one token after another, and what is written cannot be corrected. That is awkward for data where the order of dependencies is not obvious, such as cross-references or anything tied to what appears later in the text. A person in that situation writes a draft, goes back and fixes it. Diffusion language models partly close the gap, but according to the authors they still cannot flexibly insert into finished context or delete an arbitrary number of tokens from it.

How the chain of edits works

The authors call the approach Chain of Edits. The model predicts special marker tokens that open and close an edit. At the same time, it moves the cursor to the right position in the existing sequence. One mechanism thus covers all three operations: inserting new tokens, replacing existing ones and removing unneeded ones.

The training data is built in a non-obvious way. Reference answers are taken and corrupted with a chain of reverse operations, meaning insertions, replacements and deletions are introduced artificially. The model learns to restore the original by undoing these distortions. Editing here is a direct outcome of the training task, not an add-on layered over a finished model.

This kind of training has a side effect that the authors single out. Because the model is used to seeing "dirty" context, it is more robust to errors in that context and better suited to editing tasks, in particular code correction.

What is claimed

• Operations: insertion, replacement and deletion of tokens in already generated context.
• Mechanism: marker tokens that open and close an edit, plus cursor movement through the existing sequence.
• Training: perturbing reference labels with reverse chains of edits; the model learns to undo them.
• Result: the ability to edit context after generation, which, according to the authors, most autoregressive and diffusion models lack.
• Quality: no drop on standard LLM benchmarks is claimed.
• Availability: the code is published on GitHub.

The abstract gives no model sizes, no specific metrics and no benchmark names. The phrase "without loss of quality" needs to be checked against the tables in the full text of the paper. There are at least two open questions. First, what editing costs in tokens and latency: edit markers and cursor moves make the output longer. Second, how well the method transfers to larger models, given that training relies on synthetic corruptions.

Where this can be tested in practice

The nearest practical area is code correction. Edits there are pinpoint, and an error is often discovered only after a function has been written to the end. The second area is structured documents, where references and tables of contents depend on content that does not yet exist when generation starts. The third is agentic pipelines, in which a model edits its own draft instead of restarting generation from scratch.

A team building developer tools or document workflows should take the repository and run it on its own code correction tasks. The comparison should be against a baseline autoregressive model on three measures: the share of errors fixed, the number of tokens generated and latency. If editing really costs less than regeneration, the approach makes commercial sense. If not, it remains an interesting research technique.