This week at the HLTH conference in Las Vegas, FDA Commissioner Robert Califf sat down with John Brownstein, chief innovation officer at Boston Children’s Hospital, to discuss the current state of healthcare AI, as well as his agency’s role in regulating such a dynamic field.
“I don’t need to tell anybody here that the field has exploded. There’s a real challenge in figuring out how to avoid too much regulation in a field where a lot of creativity is occurring that can do a lot of human good — but also figuring out where to use regulation to help the industry hopefully move in directions that actually improve the health of patients. We’re a regulatory agency, but we’re also a public health agency,” Califf stated.
Below are some of his main concerns when it comes to healthcare AI.
The FDA can’t keep an eye on everything that’s going on out there
People have some misconceptions when it comes to the FDA’s regulatory strategy, Califf noted.
“In almost every industry that we regulate, people have this view that the FDA is sort of there all the time regulating it. Well, no. The way it really works is we have laws, rules and guidances. We’re sort of the referee in the system. But the first line of defense is the industry doing what they’re supposed to do in the first place,” he explained.
In the healthcare AI space, industry stakeholders — including health systems, AI developers and healthcare data companies — need to come together and ensure that the “life cycle of algorithms is properly conducted,” Califf remarked.
The FDA can’t ensure this on its own, he pointed out.
“If you said, ‘Well, the FDA has got to keep an eye on 100% of it,’ then we’d need an FDA two or three times bigger than it currently is,” Califf said.
Providers don’t seem capable of ongoing AI validation
After an AI model is deployed, it continually evolves, Califf noted.
“It can evolve and get worse, or it can evolve and get better,” he stated. “So I love the term local, recurrent validation. What that means is, if you want to know that your AI is actually doing what you thought it was doing, you actually need to validate it in the situation in which it’s being used. I would just say, right now, I don’t know of a single health system in the U.S. which is capable of doing that validation.”
In order to address this problem, healthcare needs to fix its data blocking problem, Califf declared.
To conduct ongoing validation research, providers need to be able to track the health of their patients, no matter where they receive care or where their records end up being stored, he pointed out. Given the current state of interoperability in healthcare, this is impossible for providers, he said.
Is AI worsening health disparities?
When it comes to the deeply embedded health inequities in the U.S. healthcare system, Califf feels that AI is “furthering the divide big time.”
“I’m very worried that health systems are using AI mostly to segregate patients into those that are profitable and those that are not. Seems to be the biggest use of AI right now in American healthcare. What we need is for AI to bring up the people who are currently disadvantaged. And it’s so obvious that could happen, but you have to focus on doing that,” he declared.
Primary care is a “disaster” in the U.S.
When looking at the world’s high-income countries, the U.S. ranks dead last in health status, Califf pointed out.
“I see a lot of the glitz at a meeting like this, but our fundamental problems are blocking and tackling. It’s like we’re trying to throw the touchdown, but we’re not even doing the basic blocking and tackling. Primary care in this country is a disaster right now, and so people are not getting the fundamental things they need,” he explained.
He said he would love to see more technology aimed at helping Americans do fundamental things like get vaccinated or control their blood pressure.
“There’s a reason Costa Ricans live five years longer than the U.S. They have primary care clinics in every neighborhood. Imagine what we could do if we turned technology loose on those sorts of things,” Califf remarked.
Photo: HLTH
