It's Not AI That Devalued Seniors: Why Hiring Can't Tell a Novice With an LLM From an Expert

The debate over whether seniors are needed in the age of neural networks is usually built on impressive numbers. In a study of 5,172 support employees, an AI assistant raised productivity by 15% on average, with the biggest gains among the less experienced. In an experiment with 453 professionals writing business texts, time dropped by 40% while quality rose by 18%. The conclusion suggests itself: the model has leveled novice and expert. But what if this is a property not of the model, but of the way we compare them?

What Expertise Really Is

From the standpoint of cognitive psychology, an expert differs from a novice not in working memory capacity, but in the contents of long-term memory. They have seen similar tasks many times and store ready-made solution schemas. This shows up in three ways. First, an expert recognizes meaningful configurations: a chess player memorizes a position from a real game at a glance, and the same effect has been found in programmers. Second, they classify problems by deep structure rather than surface features: physicists grouped problems by the law needed, while students grouped them by what was drawn in the statement. Third, they immediately see the first step of the solution.

Hence the key idea: an LLM replaces execution, while expertise is recognition. It is recognition that is needed to verify the model's answer and to notice that a task has gone beyond the limits of its abilities. In an experiment with 758 consultants, the gain lay inside this boundary, and beyond it the share of correct solutions fell by 19%.

Why Hiring Doesn't See the Difference

Walk through a typical funnel. The job posting is often written by someone who doesn't know the subject area, and it describes the role through a stack, years and buzzwords. The ATS matches keywords with keywords: surface with surface. Screening is done by a non-expert who is persuaded by a smooth story. Research on unstructured interviews shows that people confidently evaluate even random answers, and the prediction from the interview only gets worse. STAR without scales measures the ability to tell stories. Algorithmic tasks and live coding are routine, that is, they lie within the model's capabilities, and with an LLM in the interview a novice will do no worse than a senior.

As a result, no stage tests recognition, verification, and work beyond the model's capabilities. Hiring looks only where a novice with an LLM and an expert really are equal.

The Cost of Error and What to Change

There are two consequences. The first is direct errors: a novice in a role that requires verification makes mistakes beyond the model's limits and doesn't notice. The second is a broken pipeline. Experts grow out of novices, and a novice hired as a model operator does not build their own schemas. This is Lisanne Bainbridge's "ironies of automation" of 1983, transferred to LLMs.

Adaptations with a solid evidence base:

• Job analysis instead of keywords: an expert drafts 5–10 typical tasks of the role and marks which ones the LLM does, which the LLM does with human verification, and which lie beyond the model.
• Structured interview with scales: identical questions, described answer levels, less influence of impressions. The validity of STAR with scales reached 0.63 in a meta-analysis.
• Work samples on real tasks. But a take-home sample is easy to do with an LLM, and then it stops distinguishing a novice from an expert.

Adaptations justified by theory but not tested in hiring: short "first step" probes (a log, a diff, an architecture diagram with the question "what do you do first?"), "find the error in a plausible solution from the model" tasks, the candidate choosing their own strong area, and live coding without an observer followed by discussion.

Limitations and Practical Takeaway

There is an honest caveat: if a role consists almost entirely of routine tasks within the model's capabilities, the devaluation of experience is real, and hiring has nothing to do with it. Moreover, probes for recognition and verification of an LLM answer have not yet been validated, and all these methods require experts' time.

For companies the conclusion is simple: saving on experience today turns into a shortage of reviewers tomorrow. For developers and product managers: value grows where you can recognize and verify a result, not where you get an answer fastest. If you are hiring, start with a list of the role's tasks and add to the process at least one probe where the candidate must find an error in a model's answer.