Doubletapp: Weak Datasets Keep AI From Lifting NPS, CTR and Conversion

Doubletapp has published an interview on why AI projects more often break down on data than on the choice of model. Ilnur Faiziev, head of the Data LLM unit, explained how dataset quality directly affects support NPS, catalog CTR and purchase conversion.

Where the metrics get lost

The interview's central point is simple: businesses don't buy a model as such, they buy an improvement in a specific number. In support, that means how fast a request is resolved and how satisfied the customer is. In online retail, it means product-card click-through and the share of orders. In knowledge-base search, it means answer accuracy. If a dataset is noisy, poorly labeled or disconnected from real scenarios, the model starts making mistakes exactly where each mistake costs the business money. So the conversation about data is not academic here; it is a direct conversation about revenue, costs and service quality.

The material also describes less obvious cases. For an industrial company, AI can look up answers in internal regulations and reduce the number of production errors. For computer vision tasks, it can determine steel quality from process parameters and help keep results stable. The logic is the same in every case: a business metric sits at the top, the quality of the ML system below it, and beneath that a dataset that either strengthens the model or quietly drags it down.

• NPS and response time in support
• CTR and conversion in e-commerce
• Accuracy of search across an internal knowledge base
• Fewer errors in production processes
• Recognition quality in computer vision systems

When a dataset is a must

According to Faiziev, a quality dataset is needed in two typical situations. The first is when a company is only comparing AI with manual work and wants to know whether the solution can be launched in production at all. The second is when the system already works but its metrics no longer satisfy the business: answers are irrelevant, recommendations don't lead to purchases, and speed or accuracy has hit a ceiling. In both cases, without a measurable current quality level and a clear target metric, working with data turns into guesswork.

"Datasets are needed at two stages of product development."

The interview puts particular emphasis on economics. A dataset is not endless custom development but rather a finite artifact that can be prepared, validated and loaded into a training or fine-tuning pipeline. Yes, the model audit has to be repeated regularly, but collection and labeling are usually cheaper to hand to specialists in that process. If everything stays in-house, engineers spend weeks selecting examples, setting up the environment, controlling quality and managing annotators. For a business, that is often more expensive than it looks at the start.

Why crowdsourcing is weakening

The interview is also interesting because it records a shift in the market. Mass crowdsourcing worked well in the era of simple tasks such as "cat or dog." Today the models handle such scenarios fairly confidently themselves, so human labeling is moving into expert domains. For a code assistant for a rare language, complex industrial validation or a domain-specific knowledge base, what's needed is not just a large stream of workers but people who truly understand the task context and can spot subtle errors.

A hybrid scheme is still possible: the simple part of the pipeline can go to mass labeling and the complex part to an expert team. But then the business takes on a new burden: decomposing the task, finding different contractors, passing context between them and adding quality control at the handoffs. That is why, in Doubletapp's assessment, the market remains relatively narrow for now, centered on large LLM companies and projects where metric improvements are easy to convert into money.

What this means

For the market, this signals that competitive advantage in AI is increasingly shifting from picking the loudest model to the quality of applied data. Large players still need big datasets, but the next wave of demand may come from small teams with niche AI products. They will first test an MVP on ready-made data, and once they see the economics, they will start buying targeted datasets for their weak spots, and that is where real metric growth will appear.