SberZdorovye: Neural Network Nondeterminism Is a Pipeline Failure, Not a Property of the Model

Ruslan Cherkas, an architect at SberZdorovye, has come out against the popular claim that neural networks are nondeterministic by nature. His core argument: if the input data, model weights and environment are fixed, the system must produce the same result every time, and any discrepancy signals a failure in the pipeline, code or infrastructure.

Where the dispute comes from

The trigger was a situation familiar from ML practice: a team tries to reproduce an experiment but gets different metrics or a different model response. Such cases are often blamed on the nature of neural networks themselves, especially with LLMs, GPU training and complex chains of libraries and services. Cherkas takes issue with exactly this explanation and proposes a stricter view: an engineering defect, not an unavoidable quirk of the technology.

In his reasoning, a mathematical model cannot be "random in itself" if all its arguments are known and do not change. For a neural network, that means fixed input, fixed weights and identical execution conditions. In that setup, the formula should lead to the same output every time. If it does not, then somewhere between the data, hardware, libraries and algorithm there is an unaccounted-for variable that the team simply does not control.

Four sources of failure

The author goes through the most common explanations used to excuse a drifting result and reduces them to four classes of problems. His overall view is blunt: nondeterminism is not a useful "feature" when it appears without any change in input conditions. This matters not only for research but also for running a model in production, where any unexplained discrepancy quickly becomes a risk.

• Input uncertainty — the data itself, initial weights, the seed or internal states change randomly.
• Hardware faults — equipment defects, differences in operation order or an unstable runtime environment affect the result.
• Software discrepancies — library versions, optimization settings, caching or other environment details differ.
• Algorithmic errors — floating computation order, thread races and incorrect parallelization break repeatability.

Cherkas also stresses that appeals to "external factors" such as quantum effects do not relieve the developer of responsibility. If a factor affects the model's output, it must either be included in the arguments or isolated. Otherwise it is not a philosophical question about the nature of AI but an ordinary implementation error.

The article also contains a short statement of the author's position:

"Nondeterminism is an error that must and can be eliminated."

How to achieve repeatability

The practical takeaway is simple: first acknowledge the problem, then locate its source. If a model behaves differently on an identical run, the team should break the incident down by layers: check the data, compare the weights, fix the seed, and verify that the hardware, library versions and the full runtime environment are identical. For production systems this is no longer a matter of convenience but of trust in the result and the ability to investigate failures properly.

The author also warns against carelessly paying for speed by losing the order of computations. If parallelization or optimization changes the order of operations so that the result starts to drift, such an implementation cannot be considered correct for critical scenarios. This applies especially to systems where business decisions, medical recommendations, safety or other high-stakes processes depend on the model. In these cases, a deterministic pipeline should be a distinct engineering goal, not a side effect of a lucky configuration.

What this means

The SberZdorovye piece is useful because it moves the conversation about "neural network magic" into the realm of ordinary engineering. The more actively companies embed models in important processes, the less acceptable it becomes to blame unpredictability on the abstract nature of AI. In practice, the teams that win will be those that can demonstrate reproducibility, describe the sources of randomness and prove that the system remains controllable even in complex scenarios.