On December 4, 2024, United Healthcare CEO Brian Thompson was shot and killed on the streets of New York City. The killing, which took place during broad daylight, received national media attention. While far less brazen, United Healthcare has used Artificial Intelligence over the past 5 years to do something just as deadly. Since 2019, the company has used AI to wrongly deny millions of insurance claims, which has potentially led to thousands of deaths.
In May of 2022, 91-year-old Gene B. Lokken fell and fractured his ankle. After spending a week in the hospital, he declined so quickly that he was admitted to hospice, where he spent the next month. Six weeks after his fall, Lokken made an unexpected recovery and was transferred to rehab. Although he required intensive physical therapy, his improvements were remarkable, considering that he laid in a hospice bed less than two months prior.
On June 20, United Healthcare abruptly notified the 91-year-old that they would no longer cover his care. They wrote, “More inpatient days at the skilled nursing facility are not medically necessary.” Lokken’s family and doctors were shocked.
Lokken’s story was one of many detailed in a class action lawsuit raised against United Healthcare in December of 2023. The decision to deny his coverage was made by nH Predict, an artificial intelligence model that United Healthcare acquired in 2020 for $1.5 billion. nH Predict determines what care patients are eligible to receive, comparing their personal details — such as age, pre-existing conditions and diagnosis — with a database consisting of the records of over six million previous patients. It then predicts the type and length of care required.
Since United Healthcare began using nH Predict, their denial rate has skyrocketed. They denied 10% of requests in 2020, before they started using nH Predict. Just two years later, their denial rate more than doubled to 22%. Over 9 million people are enrolled in United Healthcare’s Medicare Advantage plan. The 12% increase in denials robs health insurance from hundreds of thousands of people.
It is not as if nH Predict is simply more skilled at catching legitimate reasons to deny health insurance claims than the humans who previously had the job. On the contrary, nH Predict’s error rate is astonishingly high. 90% of appealed denials are overturned. That statistic is worth repeating: 90% of denials are deemed unfair and overturned upon being reviewed. Some families, such as Gene Lokken’s, are not as “lucky.” They appealed, submitting a request from his doctor, but Lokken was once again denied. The reasoning the company included with their verdict? He was not entitled to skilled-care because he was “self feeding” and “required minimal help for hygiene and grooming.” Told by his doctors that staying was medically necessary, Lokken remained in the facility for a year until he passed away. His family paid upwards of $144,000 in out-of-pocket costs for his stay.
Few people would find issue in a healthcare company employing artificial intelligence to expedite claims, but that is clearly not the purpose of nH Predict. Multiple news sources have reported that employees who disagreed with, and sent information to patients contradicting nH Predict, were fired. The model is simply used to issue as many denials as possible, which is invaluable for United Healthcare. A human would struggle, for example, to end the coverage for a 74-year-old who is 20 days into a 100 day stroke recovery program for reasons that are “confidential,” as was alleged in the lawsuit. Each denial, unfair or not, saves the company thousands of dollars. A 2023 report found that less than 1% of in-network denials are appealed.
Despite the limited news coverage that surrounds it, nH Predict symbolizes what people fear most about AI: It is incompetent, unchallenged and affects millions of people. United Healthcare is the largest insurance company in the United States. Next time you hear of someone receiving a denial, remember that an unfair AI model was likely responsible. As automation rises, as long as there are no regulations, instances like these will become common across all sectors.