A 49,000-parameter neural network instead of Playwright tests for a browser strategy game

A developer of a browser space strategy described on Habr how he replaced part of Playwright E2E-tests with a tiny neural network autopilot — 49 thousand parameters, pure TypeScript without ONNX dependencies — and went through five iterations from an accuracy ceiling of 67% to a model that froze solid at 99.7%.

What classical tests fundamentally cannot check

Playwright, Selenium and analogues are good at catching interface bugs: the button clicked, the form disappeared, the modal closed. But they have a fundamental ceiling — they cannot assess the strategic effectiveness of game mechanics.

For a browser strategy, this is a key blind spot. The central question when developing sounds like this: "But wouldn't it be more profitable for a player to simply deposit their starting capital and wait — instead of doing anything at all in the game?" No Playwright scenario will play thousands of matches with different strategies, calculate the final profitability of each and compare it to passive waiting. This is exactly where a niche opens up for a trainable agent.

How the autopilot works

The autopilot receives a snapshot of the current game state — a set of numerical features: resources, fleet positions, timestamps — and outputs the next action: click a button, wait for an event, switch mode. Key implementation parameters:

• 49 thousand parameters — the model fits in several megabytes, no GPU needed
• Pure TypeScript — no ONNX runtime, native C library, or Python environment
• Run directly in the browser — the agent lives in the same process as the game
• Quality metric — the final effectiveness of the strategy relative to passive "deposit"

Such an agent can run in accelerated mode, execute thousands of matches overnight and automatically detect balance degradation with each change to game logic.

Five iterations from 67% to freezing at 99.7%

The first ceiling hit at 67% accuracy: the model overfit on the most frequent actions and ignored rare but critically important moves — a classic symptom of an unbalanced dataset.

After reworking the dataset and weighting the classes, accuracy went up. But when the metric reached 99.7%, something unexpected happened: the neural network completely froze and stopped making any decisions — and still showed near-perfect accuracy. "Doing nothing" turned out to be statistically correct strategy given how the target labels were encoded. Additional tuning of the loss function and forced action diversity in the training set were required.

"Unit tests, integration tests and

Playwright E2E catch bugs in the interface, but are helpless against the question: wouldn't it be more profitable to simply deposit the starting capital and wait?" — the author explains the project motivation.

What this means

The experiment points to a real niche that classical testing frameworks do not cover: behavioral testing of systems with long-term dependencies. Micro-models on TypeScript without external dependencies — a practical alternative to manual scenarios where it's important to check not "does the button work," but "does the system behave correctly in thousands of matches." Relevant for browser games, trading simulators, and any products where mechanics determine user strategic behavior.