ChatGPT and Claude bots can't see 40% of the text on every eighth pricing page
A product page can describe the product in detail, and ChatGPT still answers in generic terms. ChatGPT and Claude bots download JavaScript but don't execute it, so text that the page renders in the browser never reaches them. Profound checked 77 pricing pages: on 10 of them, one in eight, the bots couldn't see at least 40% of the main text. Keep the product name, price and audience in the raw HTML, not in scripts.
AI-processed from Habr AI; edited by Hamidun News
A product page can spend three screens explaining everything, and ChatGPT, GigaChat or Alice AI will still answer in generic terms or confuse the product with a competitor. One technical reason: ChatGPT and Claude bots download JavaScript files (the code that builds the page in the visitor's browser) but don't run them. If a script draws the price or the description, the bot sees an empty space.
Profound checked 77 public pricing pages: on 10 of them, roughly one in eight, the bots couldn't see at least 40% of the main text. A page can be written in any way at all; for AI, that part isn't there. The second reason is that a neural network doesn't read the whole site, only several retrieved fragments.
The practical conclusion: key facts about the product should sit in the raw HTML and be repeated in every meaningful paragraph.
What the neural network actually reads
A neural network doesn't load the whole site. It receives several retrieved fragments, and if those fragments don't directly say what the product is and who it is for, details from other pages won't help it.
The services use different sources. ChatGPT has its own bots. According to OpenAI's documentation, OAI-SearchBot handles search, and sites appear in results through it. GPTBot collects data for training and is unrelated to search, while ChatGPT-User opens a page when a user asks a question. Their robots.txt settings are independent. Alice AI doesn't crawl sites itself: it breaks a question into sub-questions, sends them to Yandex Search and relies on highly ranked pages. For Alice, a weakly ranked page has almost no chance of becoming a source.
Bots don't see text that scripts generate
A study by Vercel and MERJ, published in December 2024, found that ChatGPT and Claude bots receive the HTML (the page source before scripts run) and JavaScript files, but they don't execute the scripts themselves. JavaScript files made up 11.50% of ChatGPT crawler requests and 23.84% of Claude requests. The bot downloads them, but for it they are just program text. If the product description loads through a script, it isn't in the raw HTML, and therefore isn't there for the bot either.
Profound measured the scale of the problem: on 10 of 77 pricing pages, the bot couldn't see at least 40% of the text. On Profound's own site, prices were also drawn by a script, and AI crawlers didn't notice them. Switching to server-side rendering (SSR) helped: the server assembles the page, so the bot gets finished text right away.
If there is no pricing page at all, the neural network takes the numbers from third parties. For one product in the study, AI answers quoted prices from roughly $100 to $2,400 per user per month, often based on Reddit discussions. A 24-fold gap for the same product gives a buyer nothing reliable to rely on.
Schema.org markup doesn't rescue the situation either. Ahrefs tested ChatGPT, Claude, Perplexity, Gemini and Google AI Mode: all five, when reading in real time, took only the visible HTML and ignored JSON-LD, Microdata and RDFa. On 1,885 pages that added JSON-LD between August 2025 and March 2026, citations in AI Mode and ChatGPT didn't increase, while in AI Overviews they fell by 4.6%, although the authors can't confidently tie this to the markup. The markup can stay, but it shouldn't be the only copy of a fact.
Every paragraph should name the product
Even visible text reaches the neural network in pieces. Anthropic illustrates this with an example: the fragment "revenue grew 3% over the quarter" names neither the company nor the period. If a 50–100-token context note is prepended to each fragment before indexing, search errs almost half as often: 2.9% failed searches instead of 5.7%. A site can't add a note to a paragraph that another system has cut out, but it can write the product name into every meaningful paragraph itself.
Compare: "A solution that helps you grow and save time" fits a thousand products. "Kassir is a cloud cash register for small online stores: it accepts payment, prints the receipt and sends it to the customer" names the product, the audience and the task, even when it's taken out of context.
Six checks
- Open robots.txt and find the OAI-SearchBot block. A line reading Disallow: / shuts the site off from ChatGPT search. If there is no separate block, the rules for User-agent: * apply.
- Fetch the page without JavaScript (the source, or curl with grep) and look for the name, price and audience. Zero matches means a script renders the text. Check tabs and sliders the same way, and duplicate facts from PDFs and images as text.
- If the text isn't in the HTML, switch to server-side rendering or prerendering.
- Compare JSON-LD with the visible text: every value should also appear on the page.
- Read each paragraph about the product on its own, without the heading. If it isn't clear what the product is and who it is for, add that information.
- Ask ChatGPT, GigaChat, Alice AI and Perplexity about the product and compare the answers with the site. If an answer is wrong, open the sources and find the third-party page with the error.
Start with item 2: while the text isn't in the raw HTML, rewriting paragraphs is premature. Item 5 comes next.
Limitations
The data is incomplete. The Vercel and MERJ study dates from 2024, the bots may have changed, and whether GigaChat and Perplexity execute scripts is unknown. Profound collected answers from July 29 to August 10, 2026, and only about prices, while Ahrefs measured citation frequency but not the accuracy of paraphrasing. Text in the HTML and the product name in paragraphs don't guarantee a correct neural-network answer, but without them the model is left to assemble the product from generic phrases and other people's pages.
- Nov 21, 2026OpenAI Revised Its Prompts for GPT-6.2: Fewer Questions, Fewer Tests
- Nov 21, 2026CoreWeave Owes $47 Billion on Nvidia Chips and Lost $709 Million in One Quarter
- Nov 21, 2026Apple Is Building an AI Agent and Making It Harder for Outsiders to Get Access: Permission Now Takes 14 Steps, Not 4
- Nov 21, 2026Developer Skips Pinecone: Searching 72,450 Code Chunks in 50 ms
Key Questions Answered
What the neural network actually reads?
A neural network doesn't load the whole site. It receives several retrieved fragments, and if those fragments don't directly say what the product is and who it is for, details from other pages won't help it. The services use different sources. ChatGPT has its own bots. According to OpenAI's documentation, OAI-SearchBot handles search, and sites appear in results through it. GPTBot collects data for training and is unrelated to search, while ChatGPT-User opens a page when a user asks a question.
What are the key details on Bots don't see text that scripts generate?
A study by Vercel and MERJ, published in December 2024, found that ChatGPT and Claude bots receive the HTML (the page source before scripts run) and JavaScript files, but they don't execute the scripts themselves. JavaScript files made up 11.50% of ChatGPT crawler requests and 23.84% of Claude requests. The bot downloads them, but for it they are just program text. If the product description loads through a script, it isn't in the raw HTML, and therefore isn't there for the bot either.
What are the key details on Every paragraph should name the product?
Even visible text reaches the neural network in pieces. Anthropic illustrates this with an example: the fragment "revenue grew 3% over the quarter" names neither the company nor the period. If a 50–100-token context note is prepended to each fragment before indexing, search errs almost half as often: 2.9% failed searches instead of 5.7%. A site can't add a note to a paragraph that another system has cut out, but it can write the product name into every meaningful paragraph itself.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.