OpenMythos: an open-source PyTorch reconstruction of the Claude Mythos architecture with 770M parameters

Anthropic has never published a technical report on Claude Mythos. That hasn't stopped the research community, and a project called OpenMythos has now appeared on GitHub, trying to answer the question: how is this architecture actually built?

The project's author, Kye Gomez, approached the task from first principles. He did not rely on internal leaks or insider information, only on peer-reviewed publications and publicly available research on large language model architectures. The result is a full reconstruction of the alleged Claude Mythos architecture, implemented in PyTorch.

The project's main technical finding: a model with 770 million parameters delivers performance comparable to standard transformer architectures with 1.3 billion parameters. If that observation holds up, it points to a fundamentally different approach to scaling: not by adding more parameters, but by organizing them more efficiently.

Among the architectural choices Gomez reconstructed in OpenMythos are specialized attention mechanisms, non-standard normalization schemes and, apparently, a departure from the classic decoder-only design of GPT-style models. According to the author, it is the combination of these choices that produces the anomalously high parameter efficiency.

It is important to understand that OpenMythos is not reverse engineering in the technical sense, and it is not an attempt to reproduce the weights of a real Anthropic model. It is a theoretical hypothesis turned into working code. Gomez says so explicitly in the documentation: the project reflects one possible interpretation of how such an architecture could be built, and does not claim to reproduce the original exactly.

Still, the very existence of such a project is telling. Anthropic's secrecy about the architectural details of Claude has become the norm: the company publishes research on safety, interpretability and reinforcement learning, but avoids disclosing technical details of its flagship models. That creates an information vacuum, which the community fills on its own.

OpenMythos is not the first attempt of this kind. The community has previously worked on reconstructing the architectures of GPT-4 and Gemini Ultra, relying on indirect clues from benchmarks, error patterns and publications by affiliated authors. The difference is that here the architecture is one Anthropic has never officially announced at all: Claude Mythos appears only in rumors and unconfirmed leaks.

For machine learning practitioners, OpenMythos is interesting above all as a source of architectural ideas. Even if the reconstruction turns out to be inaccurate, its specific solutions, covering attention organization, normalization schemes and the approach to scaling, may prove useful regardless of where they came from.

The project is released under an open license, and the code is available on GitHub. That means anyone can run the architecture, check the claimed characteristics and propose their own interpretations. In effect, Gomez has opened a collective investigation, and the community will now carry it forward.