Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableThe Verge→ original

Apple wants 200,000 Americans to rate medical AI answers in an iOS app

Apple has launched the Apple Health AI Usage Study to find out how useful and understandable people find answers from a medical AI. Participants ask a question in a separate iOS app, such as "why am I sleeping badly," receive an answer with links to clinical sources, and rate its accuracy, thoroughness and relevance. The study is looking for up to 200,000 Americans over 21. The takeaway for developers of health AI is that checking whether an answer is correct is not enough. It also needs to be understandable.

AI-processed from The Verge; edited by Hamidun News
Apple wants 200,000 Americans to rate medical AI answers in an iOS app
Source: The Verge. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceThe Verge ↗
Editor-in-Chief
Key metric #1An iPhone 15 Pro and iOS 26 are required; Apple Intelligence is not used in the app
Key metric #2Participants can optionally share HealthKit data: sleep, activity, lab results
Key metric #3Apple promises not to train AI models on the collected data

People have long been asking chatbots about symptoms, test results and medications, but the creators of these services usually know only one thing: the model responded. Whether users understood the answer, found it useful, or were able to act on it remains outside view. Apple decided to measure exactly that. The company has launched the Apple Health AI Usage Study, a research project in which up to 200,000 Americans over 21 will ask a medical AI questions in a dedicated app and rate the answers themselves. The figure of 200,000 here refers not to the size of a training set, but to the number of people whose ratings Apple will collect.

How the study works

For the experiment, Apple released a separate iOS app. Participants type their question, for example "why am I sleeping badly." Dictation via the iOS keyboard is possible, but there is no Siri integration. In response, the person receives an explanation in plain language and links to peer-reviewed clinical sources.

The answers are not generated by Apple Intelligence. The company is running the study together with a clinical AI platform used by medical professionals. The platform prepares answers based on medical literature, including material from The New England Journal of Medicine.

After each answer, participants score it on three criteria: accuracy, thoroughness and relevance. Apple is collecting the reactions of ordinary people, not the opinions of expert physicians. This differs from familiar benchmarks, where AI is tested on medical licensing-exam questions.

Enrollment has been open since the announcement. The requirements are:

  • age 21 or older;
  • residence in the US;
  • iPhone 15 Pro or newer;
  • iOS 26 or later.

What happens with personal data

During sign-up, participants can optionally give the app access to HealthKit data: activity, sleep, and clinical lab results. In that case, the answers take the individual's context into account. A question about poor sleep sounds different if the assistant can see your readings from the past few weeks.

Apple states that participants' information remains confidential and that the collected data will not be used to train AI models. This cannot be verified from the outside, so people will have to trust the company's statement. But the wording itself matters: the study is presented as research into how people behave, not as a way to gather training material.

Apple's Vice President of Health and Fitness, Sumbul Desai, said that health is deeply personal, so a dedicated scientific study is the most thoughtful way to understand how people perceive medical AI in everyday life. According to her, the results will help advance the scientific discussion and build reliable tools.

What this means for developers and product managers

The first takeaway concerns metrics. If you are building a medical or health-adjacent AI product, fact-checking alone is not enough. An answer can be correct but too dry, too long, or off topic. Apple's criteria (accuracy, thoroughness, relevance) work well as a ready-made template for your own user evaluation.

The second takeaway concerns sources. An answer with links to clinical publications lets a person verify what was said. In a field where mistakes are costly, this becomes a baseline requirement for the product.

The third takeaway concerns personal context. Data on sleep, activity and lab results can make answers more useful, but it raises the bar for privacy. Separate consent for each data type and a clear promise not to train models on that data are a reasonable minimum.

There are limitations too. The study runs only in the US and only on pricier iPhones, so the sample will skew toward owners of recent devices. Apple has not yet announced when results will be published, nor said whether the findings will turn into a feature in the company's own products. But if the study reaches its stated 200,000 ratings, it could reveal a rare picture: what ordinary people actually consider a good medical answer from AI.

Story

Key Questions Answered

How the study works?

For the experiment, Apple released a separate iOS app. Participants type their question, for example "why am I sleeping badly." Dictation via the iOS keyboard is possible, but there is no Siri integration. In response, the person receives an explanation in plain language and links to peer-reviewed clinical sources. The answers are not generated by Apple Intelligence. The company is running the study together with a clinical AI platform used by medical professionals.

What happens with personal data?

During sign-up, participants can optionally give the app access to HealthKit data: activity, sleep, and clinical lab results. In that case, the answers take the individual's context into account. A question about poor sleep sounds different if the assistant can see your readings from the past few weeks. Apple states that participants' information remains confidential and that the collected data will not be used to train AI models.

What this means for developers and product managers?

The first takeaway concerns metrics. If you are building a medical or health-adjacent AI product, fact-checking alone is not enough. An answer can be correct but too dry, too long, or off topic. Apple's criteria (accuracy, thoroughness, relevance) work well as a ready-made template for your own user evaluation. The second takeaway concerns sources. An answer with links to clinical publications lets a person verify what was said.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…