EDU-Jev 4B grades student work with 67.6% accuracy, beating commercial APIs by 9.4 points

A team from King's College London (Jiazheng Li and Yulan He) has introduced EDU-Jev, a family of compact models for automated grading of student work. The 0.8B, 2B and 4B-parameter models do not write out reasoning. They take the task context and a question with predefined options, then return a probability distribution over those options in a single forward pass. On twelve educational benchmarks, the 4B version reached an average accuracy of 67.6%. That is 9.4 percentage points above the strongest of five commercial APIs for this kind of decision, and 18.7 points above the best open 4B model of the same type.

Why grading needs "System One"

Most decisions in education have the same shape: give an answer 0 to 3 points against a rubric, check whether the text meets a given criterion, or find the first incorrect step in a solution. The options are known in advance. Generative models usually handle this by reasoning and then emitting a final label, and the authors point to three problems with that approach.

The output has to be parsed, and it can fall outside the allowed options. Cost grows with output length, and reasoning models burn many tokens even on simple cases. And confidence expressed in words does not have to match the real probability of error, which is exactly what you need in order to send doubtful cases to a human reviewer.

The "typed decisions" interface was introduced earlier by TypeSafe AI in its hosted model Jev (tested in the paper through API version 1.14). For comparison, there is an independent public benchmark, JevBench. According to the authors, open alternatives are trained mostly on general data and handle rubric-based grading poorly.

Pointer head and soft labels

Instead of a language head, EDU-Jev uses a pointer head. In the prompt, a marker token follows each answer option, and a decision token comes at the end. A two-layer MLP with shared weights reads the hidden state of each marker and outputs a logit, and the softmax is computed over exactly the declared options. The model can only make a wrong choice; it cannot produce an invalid answer. In controlled experiments at 0.8B, a "letter" head that reads the logits of the A/B/C labels was significantly worse, and it gave the same answer 3.5 points less often when the options were shuffled.

Training is a single stage with soft cross-entropy. The target is the distribution of annotator scores when there were several annotators, and one-hot otherwise. The base models are Qwen3.

5 with full fine-tuning. The dataset is about 1.5 billion tokens, or 4.

6 million decisions: grading of short answers and essays, step-by-step solution checking, student errors, and tutor dialogues, mostly in English and Chinese. The warmup-stable-decay learning-rate schedule ends on a not-yet-seen tail of the epoch, and the final weights are an average of three checkpoints (halfway, three-quarters and the end of training). According to the authors, this costs a few tenths of a point on in-distribution data and gains about one point out of distribution.

Reinforcement learning on the pointer output did not outperform simply continuing regular supervised training.

What the measurements showed, and where the limits are

Across 20 datasets, split into general decisions, reasoning and education, the 4B model scored 67.6%, against 45.2–48.9% for open 4B models and 49.2–58.2% for commercial APIs. The 2B and 0.8B versions scored 63.2% and 62.8%, so they also beat all baseline models on average across education. On four educational datasets from other domains, the 4B model matched the strongest API. The 2B model processes about 150 decisions per second on a single H100.

The model did not become universal. On general decisions, the 4B trails the APIs by 3.2–6.7 percentage points, and on reasoning it trails three of them by 2.2–3.7 points. Among open 4B models, it matched JevK5-4B and trailed OpenJev-4B by 1.3 points. Scaling up the model gave a larger gain on transfer and general reasoning than on accuracy within the grading domain itself.

There are also methodological caveats. Calibration was checked empirically, using expected calibration error (ECE) over 15 bins and without temperature scaling. This is not a guarantee on new data. The confidence intervals account only for the sampling of tasks, not for variance between runs. Some of the training labels are synthetic: they were produced by third-party models. Grading without a rubric, reference answer or key points is intentionally unsupported. Finally, the model does not explain its grade, so scenarios that need a justification for a student or teacher will require a separate layer.

For teams building automated grading, the takeaway is practical. If decisions come down to choosing from a fixed set against a rubric, a small model with typed output can be more accurate and cheaper than a generative judge. The probabilities are usable for routing uncertain cases to a human.