How Meta's AI System for Tracking Emotions and Health Works

Meta has received a patent for an AI system that uses computer vision and biometric sensors to track a user's emotions and manage their medication intake. The system offers recommendations based on an analysis of mood and behavior.

How the device works

According to the patent, the gadget continuously records the user's surroundings (camera and microphone) and uses computer vision and biometric sensors to recognize their emotional state. The system analyzes facial expressions, tone of voice, heart rate, and other markers of mental state, then makes suggestions: anything from a medication reminder to recommendations like "you seem upset right now — maybe go for a walk?" The device collects and combines many types of user data, from video content to biometrics, to build a model of the person's mental and physical state.

• Continuous video recording of the user's surroundings
• Emotion analysis via facial recognition, tone of voice, and biometrics
• Medication adherence monitoring (via the camera or built-in sensors)
• Personalized recommendations based on the user's current state
• Integration with medical history and treatment schedules

Healthcare applications

A device like this could potentially improve the lives of millions of people living with chronic illness, depression, and anxiety. The main goal is to improve patients' adherence to their treatment plans: research shows that up to 50% of patients skip doses, either because they forget or out of inertia. An AI assistant could step in with a reminder at the critical moment when motivation fades, or even recognize that a patient is "forgetting" on purpose — and suggest talking to a doctor about side effects.

Real-time emotion monitoring would also give doctors invaluable data on a patient's mental state between visits. Today, doctors get that information only from subjective patient questionnaires or the patient's own descriptions. For people with depression, bipolar disorder, or trauma, this could mean more accurate diagnosis and treatment.

Privacy and control concerns

But the patent immediately drew pointed questions from critics and regulators: a device that constantly watches, listens, and analyzes could turn into a tool of total control. What data is stored in Meta's cloud? How does the company encrypt and protect it? Could AI recommendations gradually evolve into manipulation — for example, "you're sad, maybe buy something?" followed by targeted ads matched to your emotional profile?

Big Tech's track record shows that health and mental-state data is the most sensitive and potentially the most profitable. A company that knows your weak spots in real time can monetize them inappropriately. There's also an open question: did the patient give informed consent to constant video surveillance, or is it a condition the company dictates?

It's still unclear whether such a device will ever reach the market. Many of Meta's ambitious patents remain conceptual and never make it to consumers.

What it means

Big tech companies are pushing deeper into healthcare and biomonitoring, blurring the line between usefulness and invasiveness. If devices like this do appear, questions about privacy, consent, and data collection must be resolved before launch; otherwise, the risk of healthcare becoming a tool of total surveillance is too great. Regulators need clear rules — and they should be adopted now, before the device itself exists.

*Meta is designated an extremist organization and banned in Russia.

How does the AI system recognize emotions?

The system analyzes facial expressions, tone of voice, heart rate, and other markers of mental state using computer vision and biometric sensors.

When will this system be available to use?

For now, this is only a Meta patent. No plans to build it into products have been announced.