Logits

Logits are the raw, unnormalized real-valued scores output by the final linear layer of a neural network before softmax normalization; in language models, one logit per vocabulary token represents that token's relative predicted likelihood.

In language models, logits are the output of the final linear projection layer, which maps the model's hidden-state vector to a real-valued score for each entry in the vocabulary—commonly 32,000 to over 100,000 tokens for modern models. These scores are unbounded real numbers with no direct probabilistic interpretation in isolation: what matters is their relative magnitudes. A higher logit means the model ranks that token above alternatives, but the absolute value is meaningless without comparison to the rest of the vocabulary vector.

To convert logits into a probability distribution, the softmax function is applied: each logit is exponentiated and divided by the sum of all exponentiated values across the vocabulary, producing non-negative values that sum to 1.0. The term has roots in statistics, where a logit denotes the log-odds of a binary probability log(p/(1−p)), but in modern deep learning it is used more broadly to mean any pre-softmax score in a classification or generation setting. All major token sampling strategies—temperature scaling, top-k filtering, top-p nucleus sampling, repetition penalties, and logit bias—operate on this vector before softmax is applied, making logits the central control point in the inference pipeline.

Logits matter because they are the layer at which practitioners intervene to shape model behavior without retraining. Logit bias (adding a constant to specific token logits) enforces output constraints such as JSON formatting or content restrictions. Classifier-free guidance in language and diffusion models blends conditional and unconditional logit vectors by a guidance scale factor. Constrained decoding sets logits for grammatically invalid continuations to negative infinity, guaranteeing syntactically valid structured output. Returning log-probabilities (log-softmax of the logits) to API consumers enables downstream scoring, uncertainty estimation, and re-ranking of candidate completions.

As of 2025–2026, several production APIs expose logprob information to developers: the OpenAI API returns top-k log-probabilities per token upon request, enabling confidence measurement and token-level analysis. Interpretability research—particularly mechanistic interpretability work at Anthropic and DeepMind—uses the "logit lens" technique, which projects intermediate layer hidden states through the output matrix to observe how token predictions evolve across transformer layers, treating logit space as a diagnostic window into model computation.