A Neural Network for Browser Testing Instead of Playwright: 99.7% Accuracy

A developer described on Habr how they replaced part of their Playwright E2E tests with a tiny neural-network autopilot, with 49,000 parameters and pure TypeScript with no ONNX dependencies, and went through five iterations from 67% accuracy to 99.7% in the final model.

What classic tests fundamentally cannot check

Playwright, Selenium and similar tools are good at catching interface bugs: a button was clicked, a form was submitted, a modal window closed. But they hit a fundamental ceiling: they cannot assess the strategic effectiveness of game mechanics.

For a browser strategy game, this is the key blind spot. The central question in development is: "Wouldn't it be more profitable for the player to simply put the starting capital in a deposit and wait, instead of doing anything in the game at all?" No Playwright scenario will play thousands of games with different strategies, calculate the final return of each and compare it with passive waiting. This is where a trainable agent finds its niche.

How the autopilot works

The autopilot takes a snapshot of the current game state, a set of numeric features such as resources, fleet positions and timestamps, and outputs the next action: click a button, wait for an event, switch a mode. Key implementation parameters:

• 49,000 parameters: the model fits in a few megabytes, no GPU needed
• Pure TypeScript: no ONNX runtime, native C library or Python environment
• Runs directly in the browser: the agent lives in the same process as the game
• Quality metric: the strategy's final effectiveness relative to the passive "deposit"

Such an agent can run in accelerated mode, play thousands of games overnight and automatically flag balance degradation with every change to the game logic.

Five iterations, from 67% to freezing at 99.7%

The first ceiling came at 67% accuracy: the model overfit to the most frequent actions and ignored rare but critically important moves, a classic symptom of an imbalanced dataset.

After reworking the dataset and weighting the classes, accuracy climbed. But when the metric reached 99.7%, something unexpected happened: the network froze completely and stopped making any decisions, yet still showed near-perfect accuracy. "Doing nothing" turned out to be the statistically correct strategy given how the target labels were encoded. Further tuning of the loss function and forced action diversity in the training set were needed.

"Unit tests, integration tests and

Playwright E2E catch bugs in the interface, but are powerless against the question: wouldn't it be more profitable to simply put the starting capital in a deposit and wait?" the author explains the project's motivation.

What this means

The experiment points to a real niche that classic testing frameworks do not cover: behavioral testing of systems with long-term dependencies. Dependency-free TypeScript micro-models are a practical alternative to manual scenarios where the goal is to check not "does the button work" but "does the system behave correctly across thousands of games." This is relevant for browser games, trading simulators and any product where the mechanics shape users' strategic behavior.

How is a neural network better than classic

Playwright tests? Classic tests catch interface bugs (a button was clicked, a form closed) but cannot assess the strategic effectiveness of game mechanics. A neural network can do this.

What accuracy does a neural network achieve in browser testing?

According to the developer's results, the neural network reached 99.7% accuracy with 49,000 parameters after five training iterations.