TheChek News Flow

So far, Apple has pitched its upcoming Vision Pro mixed-reality headset as a tool for working and enjoying various forms of entertainment. But inside the company, employees have also discussed another, more specialized use of the product as a way to diagnose and treat mental health issues, according to people with direct knowledge of the matter.

Such features would tap the device’s array of cameras and sensors to measure a person’s facial expressions, using them to detect depression, anxiety, stress or post-traumatic stress disorder, the people said. The Vision Pro could display images and sounds that might improve the wearer’s emotions, one of the people said. Various rules restrict companies from marketing medical functions in their devices without an OK from regulators. Over time, Apple has received clearance from regulators for health features in the Apple Watch—such as the ability to detect an irregular heart rhythm—which has broadened its appeal beyond a mere fashion accessory.

• Apple has explored whether Vision Pro could help treat mental health issues

• Outward-facing cameras could detect changes in movement

• Eye-tracking cameras could measure emotions, screen for heart conditions

Among the ideas employees have discussed for the Vision Pro is using its eye-tracking features and external cameras to measure a person’s affect, a psychological term referring to how a person expresses feelings or emotions, the people with knowledge of the matter said. A “flat affect” or emotionless expression can be a sign that a person has schizophrenia, autism, depression, brain damage or even PTSD, for example. Members of Apple’s Vision Products Group, including its leader, Mike Rockwell, have devoted extensive time to discussing the healthcare possibilities of the headset and have hired health experts to explore such features for the device, one of the people said.

Therapeutic functions could be especially important in the case of Vision Pro, a product that will initially cost $3,500—14 times the price tag of the cheapest Apple Watch.

The idea of using headsets to treat mental health disorders isn’t a new one. For years, researchers have studied the use of virtual reality headsets to diagnose and treat phobias and PTSD in patients—for example, by exposing military veterans to combat simulations.

Joe Jerome, a visiting professor at the University of Tampa who previously worked on augmented reality and VR policy at Meta Platforms, said the new sensors available on the latest headsets has made it easier to explore health and wellness uses for the devices.

“Without question, health and wellness use cases are really important to consumers of a whole bunch of different demographics,” Jerome said. “Could this boost sales? Sure, but a cheaper headset does it better.”

Multiple people who have worked on Vision Pro over the years said teams inside Apple have extensively discussed or explored health, wellness and fitness features as recently as this year. It couldn’t be learned whether Apple will include mental health features in the initial Vision Pro, which it has said will go on sale next year, or in future models. When the company announced the headset in June, it didn’t demonstrate health, wellness and fitness uses for the product, with the exception of a meditation experience.

An Apple spokesperson declined to comment.

Apple has shown a broader interest in exploring functions in the Apple Watch and iPhone to improve mental health. Earlier this year, it released new versions of its Health and Mindfulness apps that allow users to log their feelings and receive insights on their state of mind. Some of those features came about after Apple partnered with the University of California, Los Angeles, on a three-year study beginning in 2020 that involved thousands of participants using their Apple devices to track their physical activity, heart rate, sleep and other daily routines.

In 2019, Apple CEO Tim Cook told CNBC that the company’s “greatest contribution to mankind” would be related to health. Former Apple employees who worked in its health group said healthcare is one of the few industries with the size and scale to significantly move the needle on Apple’s revenues, assuming the company’s products could eventually play a substantial role in delivering healthcare services or related functions like billing for those services.

The Information previously reported that Apple had built demonstrations of fitness and wellness apps that guided users in the practice of tai chi and yoga while using the headset’s cameras to measure body movements and breathing. Former Apple employees who worked on the Vision Pro said the company explored using the Vision Pro to enhance the experience of using a rowing or cycling machine.

Any features that Apple claims can diagnose or treat physical or mental health issues would need to undergo clinical studies and have the clearance or approval of health regulators, they said. For instance, the U.S. Food and Drug Administration has cleared the company’s claims that the Apple Watch can detect atrial fibrillation or irregular heart rhythms, which can be a precursor to blood clots, strokes or heart failure.

Apple employees are concerned about the legal liability of making health-related claims, which could limit the scope of these features. For example, when wearers use the electrocardiogram app on the Apple Watch, the app tells them the product can’t check for heart attacks. When they activate the blood-oxygen app, it notes that the measurements aren’t intended for medical use.

Researchers have also begun examining whether wearable consumer devices could end up contributing to health anxiety for the people who wear them and encourage unnecessary visits to doctors’ offices. In 2020, researchers at the Mayo Clinic published a paper about a study of 264 patients who had sought medical treatment after receiving alerts from their Apple Watches about abnormal pulses. Only 11.4% of them received “clinically actionable cardiovascular diagnoses.”

Pupil Dilation

Not everyone inside Apple is optimistic about the prospects of using Vision Pro to treat mental health issues. Some people involved in Apple’s health efforts were skeptical about whether the devices could be as effective as other treatments, such as medicine, and cautioned that such features might never materialize.

Still, those doubts haven’t stopped Apple from brainstorming health-related uses for the headset.

Its employees have also discussed using the Vision Pro to sense weight fluctuations in a person’s body and to evaluate a person’s movements over time; the latter information could be used to screen for ailments such as Parkinson’s disease, the people said. In the near term, those functions are unlikely to appear on the headset. Apple cut full body tracking from the Vision Pro several years ago because engineers couldn’t make it reliable enough (The Information previously reported that Apple told developers full-body tracking wouldn’t be available when the Vision Pro ships next year.).

Another idea the team has discussed is using the headset’s eye-tracking cameras to detect pupil dilation, which could provide a clue to a person’s mood, one person who worked on the device said. The device’s infrared cameras could also detect swelling in the blood vessels of the eyes, a possible early sign of heart failure, another person said.

Inside Apple, mental health is among four pillars of the company’s overall health ambitions, along with nutrition, exercise and sleep, according to former Apple employees involved in its health efforts.

For years, Apple health employees have discussed ways to use the iPhone’s front-facing camera and sensors to track and improve people’s moods, though such features have yet to materialize. If they do emerge in the future, Apple would likely record and analyze users’ data on their devices instead of uploading that information to the cloud, to adhere to its own privacy standards, former employees said.

Other Apple health employees have explored how a user’s app usage and the words they type can be analyzed to detect early signs of some forms of dementia such as Alzheimer’s disease. In 2021, for example, Apple announced a partnership with neuroscience company Biogen to examine how the Apple Watch and iPhone could monitor cognitive performance and help detect a decline in cognitive health.

Last month, Biogen told MobiHealthNews it was pulling out of the study due to a change in its research and development priorities. At the time, Biogen noted that the initial results of sensor data gleaned from Apple devices showed that they could “reliably measure cognition and behavior in real-world unsupervised settings.”