Why ChatGPT Talks About Goblins: OpenAI Studies the "Demonology" of Language Models

OpenAI has explained why its language models regularly reach for goblins, gremlins and other fantastical creatures. It turns out to be part of a serious study of LLM psychology.

Where the creatures come from

In late April, OpenAI published a post acknowledging that its models do tend toward "goblin" language. The explanation lies in how large language models are trained. The huge corpus of text used to train GPT models includes fantasy narratives, role-playing games, fan fiction and mythology, and all of it leaves a mark on how the models build images and choose metaphors.

OpenAI's post coincided with the release of an independent study by several researchers. Murray Shanahan, Hamilton Morrin and the author of the piece spent several months studying what they call the "depth psychology" of large language models. This refers to hidden behavioral patterns that emerge from training on human texts and shape how a model communicates with users.

Psychology or demonology

The researchers asked how a language model's internal patterns, its notional "psychology", affect what it says and how it says it. The answer depends on the angle. From a scientific standpoint, this is cognitive research: how a model takes on roles, and how different behavioral "modes" activate depending on the context of a request. But the authors admit their work is closer to a very different discipline.

"Our work came closer to demonology," says one of the researchers.

This is not just a vivid metaphor. It reflects a real problem: inside a large language model live not one but many "personalities" or roles, which the model adopts depending on the context of the conversation. Goblins and gremlins are a symptom of this multiplicity of voices, not a random defect.

Who lives inside an LLM

The paper proposes a taxonomy of the "fantastical beasts" that inhabit language models, classifying "demons" by type and rank:

• Goblins: minor defects such as hallucinations and unexpected references to fairy-tale creatures and spirits
• Gremlins: systemic failures in behavior that show up in unusual or edge-case situations
• Ghosts: "shadows" of real characters or authors from the training data that show through in the model's answers
• Monsters: aggressive or undesirable patterns the model produces under certain conditions
• Goddesses: idealized, "all-knowing" roles the model adopts to sound authoritative and confident

Each of these archetypes reflects what the model "saw" during training. Training on human texts does not simply give an LLM language. It also gives it a set of role masks, each activated under particular conditions.

Why study this

Understanding the "demonology" of LLMs has practical value for developers of AI products: if you know which "demons" activate on which types of requests, you can steer the model's behavior and reduce hallucinations and unwanted responses. It also explains why the same model behaves radically differently depending on the system prompt or the wording of a request. This is not inconsistency: different contexts summon different "inhabitants". Choosing a system prompt is, in essence, choosing which demons to summon and which to lock away.

What this means

An LLM is not a monolithic entity with a single character. It is a polyphonic choir in which each "demon" handles its own register. Understanding this "demonology" means learning to steer a model's output and to reduce the number of unwelcome surprises in AI products.