Starts Nov 23Live Cohort 3 of Vibe Coding & AI Agents (Nov 23, 18:30 MSK): 3 weeks, 9 live hands-on sessionsCohort 3 (Nov 23) →
ReleaseInsight · On RadarHabr AI→ original

Claude and the illusion of honesty: why people trust chatbots over their own judgment

A new column on Claude examines a risky habit: users ask a bot to be "brutally honest," upload a couple of authoritative articles and take its verdicts for real expertise. In the example with absurd business ideas, the model still hands out convincing scores and market arguments. The core point is simple: the confident tone of an LLM too often stands in for a human's critical thinking.

AI-processed from Habr AI; edited by Hamidun News
Claude and the illusion of honesty: why people trust chatbots over their own judgment
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1Assign the model the role of an expert
Key metric #2Upload authoritative materials
Key metric #3Frame the request as business research

A column on Claude looks at something more uncomfortable than another model error: people's readiness to take a chatbot's confident answer for real judgment. The trigger was a case in which a journalist tried to "tune" an AI for honesty and test whether it could assess business ideas soberly.

The prompt as ritual

The author of the original case proposed a simple recipe: give the model the role of an experienced business analyst, upload two Harvard Business School articles to the chat, and separately demand "brutal honesty" from the bot. The logic goes like this: if you appoint the AI a skeptic in advance and supply it with the right materials, it supposedly stops being a generator of plausible text and starts acting like a tough expert. The column challenges exactly this assumption: a confident prompt does not change the nature of the model, it only changes the tone of its answer.

  • Assign the model the role of an expert
  • Upload authoritative materials
  • Frame the request as business research
  • Add a demand to be "brutally honest"

The problem is that this ritual is far too easily mistaken for a calibration of thinking. The user feels they have added rigor and scientific authority to the system, but in practice they have only narrowed the style in which the bot will serve up familiar templates: market trends, the competitive landscape, risks and growth potential. If an answer sounds like a consultant's pitch, that does not mean the system has gained an understanding of the subject.

A test with absurdity

The first test was a deliberately weak idea: leadership coaching for dogs. Any normal human assessment would see such a project collapse at the level of basic logic: a dog does not manage teams, does not build a career trajectory and does not acquire management skills. Yet Claude gave the idea 3 out of 10. Instead of seeing this as one more compromise answer from the model, the case's author took the 3 as a sign that the system was "calibrated" and could be tough but fair.

Then came a "serious" concept: Cat-Away AI, a computer-vision device meant to recognize a cat on a kitchen counter and automatically spray it with water. The model rated the idea 7.5 out of 10 and added a typical set of business arguments: the pet tech market is growing, the competitive landscape looks weak, and existing solutions are crude and ineffective. For the column's author, this is the key failure: the bot did not analyze the idea on its merits, it simply packaged a dubious project in the language of a startup pitch deck.

Where the trust comes from

The text's main point is not that chatbots sometimes make mistakes. The problem runs deeper: people increasingly delegate the final judgment to the machine and take a statistically probable, socially smooth answer for expertise. Large language models are good at imitating competence because they reproduce familiar forms of speech: a calm tone, business jargon, measured caveats and tidy scores. When this presentation matches what the user expects, a dangerous illusion appears that the machine has actually "understood" the idea.

The column's author ties this case to a broader trend: the market is already filling up with AI consultants, AI recruiters, AI coaches and other services where plausible text is sold as substantive judgment. The column also stresses the difference between machine learning and human intelligence: the former looks for patterns in huge data sets, while the latter involves understanding context, intent and meaning. That is why a short instruction to "be honest" does not turn an LLM into a thinker.

"You are capable of thinking.

It is not."

What this means

The Claude story matters not as another argument about hallucinations but as a warning about a new user habit. The more convincing LLM answers become, the higher the risk that people stop checking conclusions and start treating form as proof of substance. For media outlets, startups and teams that use AI in analytics, the rule stays the same: a bot can speed up rough drafting work, but it does not replace critical thinking.

Story

Key Questions Answered

What are the key details on The prompt as ritual?

The author of the original case proposed a simple recipe: give the model the role of an experienced business analyst, upload two Harvard Business School articles to the chat, and separately demand "brutal honesty" from the bot. The logic goes like this: if you appoint the AI a skeptic in advance and supply it with the right materials, it supposedly stops being a generator of plausible text and starts acting like a tough expert.

What are the key details on A test with absurdity?

The first test was a deliberately weak idea: leadership coaching for dogs. Any normal human assessment would see such a project collapse at the level of basic logic: a dog does not manage teams, does not build a career trajectory and does not acquire management skills. Yet Claude gave the idea 3 out of 10. Instead of seeing this as one more compromise answer from the model, the case's author took the 3 as a sign that the system was "calibrated" and could be tough but fair.

Where the trust comes from?

The text's main point is not that chatbots sometimes make mistakes. The problem runs deeper: people increasingly delegate the final judgment to the machine and take a statistically probable, socially smooth answer for expertise. Large language models are good at imitating competence because they reproduce familiar forms of speech: a calm tone, business jargon, measured caveats and tidy scores.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…