By Jennifer Bresnick, Health IT Analytics | April 15, 2019
Access to large volumes of quality data is vital for the success of artificial intelligence, but the healthcare industry has a lot of work to do before data becomes a truly valuable commodity.
Few adages sum up the challenges of bringing artificial intelligence to the healthcare industry better than Amara’s law, which states that people tend to overestimate the effect of a new technology in the short term but underestimate the impacts in the long run.
Closely correlated with the equally popular Gartner hype cycle, Roy Amara’s insight into the ups and downs of innovation shows the dangers of thinking short-term about technologies that have huge potential but may not yet offer clear and immediate return on investment.
Healthcare organizations have actually struggled somewhat to understand the immediate value of AI as a way to solve their business problems.
While AI has indeed produced the expected wholesale transformation of other industries, the majority of hospitals and health systems have not yet been able to take advantage of it.
The hype is strong and persistent, and an active vendor community is doing its best to push adoption of game-changing technologies as quickly as possible.
Yet providers are still wrestling with the troublesome data siloes and competing priorities that have stymied so much of their technological progress, and few have the capacity to plan for a seemingly distant future.
But at the 2019 World Medical Innovation Forum, clinicians, researchers, and developers from Partners HealthCare and Harvard Medical School certainly weren’t underestimating the longer-term potential of artificial intelligence.
In fact, they were concerned that the healthcare system is not doing nearly enough to cut through the current hype and adequately prepare itself for the fundamental changes that AI will bring over the next several decades.
Providers, payers, and patients will need to make major adjustments to the way they interact with one another and with the concepts of “disease” and “wellness,” said expert after expert at the three-day event.
Institutions will need to reframe the way they think about the big data that will become increasing vital for cutting costs and improving outcomes: what types of data are useful, how data should be shared, and how algorithms can avoid recreating human bias in their results.
And regulators will have to keep pace with the blistering rate of change by adjusting privacy and security frameworks, paying more attention to real-world evidence, and continuously monitoring the safety and effectiveness of algorithms designed to evolve.
“We are at a turning point in terms of how we can use data to support improvements to the healthcare system,” said CMS Administrator Seema Verma to HealthITAnalytics.com. “The timing is right for true digital transformation.”
“Artificial intelligence is maturing. Data interoperability is starting to improve. The drivers of consumer-focused healthcare are getting stronger. We need to start having these conversations, and we will need to keep having them for some time.”
Implementing wholesale changes in a system that still retains many of its pre-Enlightenment era conventions is not going to be easy.
But artificial intelligence will not fulfill its potential if its stakeholders are not able to look beyond the near-term hype and make a concerted effort to create an equitable, interoperable, and research-driven healthcare system for the future.
GETTING PROACTIVE BY USING INFORMATION FROM EVERY INTERACTION
Programming innovations aside, artificial intelligence requires a fresh look at the role of data in treating – or better yet, preventing – both common and uncommon diseases, said presenters at the event.
“I would like to see a complete reframing of the healthcare challenge, especially in terms of rewarding early detection and prevention,” said Noubar Afeyan, founder and CEO of Flagship Pioneering, a venture capital company.
“We should be thinking about healthcare the same way we think about national security. We don’t have a Department of Restoring Peace in this country – we have a Department of Defense, because the goal is to keep the enemy away from our shores, not deal with them once they have invaded.”
The country’s intelligence services help to identify and neutralize enemies before they strike, he continued. Artificial intelligence should be playing the same part in healthcare: getting ahead of the modifiable roots of diseases through more proactive monitoring and surveillance.
In order to do so, the healthcare system will need to learn how to absorb even more information – and different information – about individuals, their lifestyles, and their environments, said Calum MacRae, MD, PhD, chief of cardiovascular medicine at Brigham & Women’s Hospital (BWH), in an interview.
“In medicine, we tend to put a lot of faith in the types of information we have always put our faith in,” MacRae said. “But we know that clinical data only represents a fraction of the information that really predicts outcomes.”
The industry has widely embraced the idea that up to 80 percent of outcomes are related to non-clinical factors such as economic status, educational attainment, diet, exercise, and other social determinants of health.
But tracking and addressing these common influences on personal health is a major challenge for providers.
Fee-for-service reimbursement models, a frustrating legislative environment, and deeply entrenched social inequities are part of the problem.
“In medicine, we tend to put a lot of faith in the types of information we have always put our faith in.”
So is the fact that the industry simply does not yet have the capacity to capture, analyze, and leverage the non-traditional data sets that contain vital clues to how and when chronic conditions develop.
“We need to start to use the data that we generate in our daily lives: GPS data from our phones; accelerometry data from our watches; our television viewing habits; our credit card receipts,” stressed MacRae.
“We need to start thinking about how, in a secure and controlled way, we can use that information to inform our behaviors years before we ever need to see a healthcare provider.”
Developing the technology and processes to collect and incorporate these novel data sets is definitely a major obstacle. But it will be much easier for health IT developers and providers to deploy innovative screening and risk stratification strategies once underlying attitudes have changed.
“We need to retrain the system,” said MacRae. “Much of what we see as bugs are actually features, but we do not currently have the right culture – or, frankly, the right reimbursement models – to use those data points to advance the practice of medicine.”
AVOIDING BIAS WHILE ARCHITECTING A CULTURE OF AI-DRIVEN CARE
There is no magic cure for what ails healthcare, and no single solution for turning the reactive “sick care” system into the equivalent of the Department of Wellness.
Instead, changes will happen step by step, starting with how this first generation of researchers and clinicians develop artificial intelligence tools and address fundamental challenges such as data access, algorithmic bias, and instilling confidence in physicians.
Over the past two to three years, an explosion of innovation has produced thousands of pilot programs, research collaborations, and hopeful startup companies all looking to give birth to blockbuster AI algorithms with outsized impacts on patient care.
Many of these have proven that, in targeted and well-controlled applications, AI is becoming sophisticated enough to meaningfully augment clinical workflows, especially for pathologists, radiologists, and other diagnostic specialties.
For Constance Lehman, MD, PhD, chief of the breast imaging division at Massachusetts General Hospital (MGH) and a professor of radiology at Harvard Medical School, artificial intelligence has the potential to address one of the most significant challenges in mammography: reducing inter-reader variation.
“Some structures in the body lend themselves well to precise measurements, which makes it easier to see when changes occur. But in other areas, radiologists are looking at very subtle patterns of tissue structure,” she explained.
“Mammography is probably one of the most extreme examples of that. We look for differences in texture and shading in the normal healthy glandular tissue, not specific structures, and every mammogram is like a unique fingerprint for every woman.”
“Human brains just don’t do a great job of processing those signals, and that results in variation of interpretation.”
Forty percent of certified breast imaging radiologists perform outside of recommended ranges for acceptable specificity, she said, citing her previously published research.
And while some radiologists designate less than 10 percent of breast tissue as dense, a key risk factor for breast cancer development, others will label more than 80 percent of mammograms in the same way.
Lehman has developed a deep learning tool that is more accurate than humans at identifying individuals at high risk of developing breast cancer, the nation’s second leading cancer diagnosis for women.
Deep learning algorithms have the potential to be less biased than humans, who can be influenced by preconceived notions, external information about a patient’s history, or even the financial impact of their decision when reading an image.
“Computers don’t have that same bias unless we want them to,” said Lehman. We can give an algorithm supplemental risk information if we think it will help with decision-making, but we can also have them simply read the image and only the image to give a more objective assessment of the data at hand.”
Striking the balance between informed insight and impartiality will be crucial for fostering trust among clinicians, who are generally skeptical of adding anything new to their decision-making processing.
Unintentionally or invisibly biased algorithms represent what may be the single most impactful point of failure for AI tools, Lehman said.
“We’ve seen many problems that aren’t really talked about a lot in the AI world. They tend to get polished up or swept under the rug a little bit,” she said, citing the example of a new algorithm that claims to identify hip fractures better than human radiologists.
“But it turns out the model learned that if you have an x-ray that was taken with a portable machine, it’s more likely that you’re going to have a fracture on the image,” she said. “That’s because patients who are in too much pain to get out of bed will have their images taken by portable machines, and those patients are more likely to have fractures.”
Even Lehman herself ran into the problem of her algorithm being too smart for its own good, illustrating just how difficult it can be to stamp out bias – even when you’re looking for it.
“We had an early experience with training our model where we fed it all the cancer cases first before we gave it the non-cancer cases, just because that was easier for us to organize,” she said. “The algorithm quickly learned that images with certain dates had cancer, and images outside of that range did not. It used that as a factor in making its determination, which was not what we intended.”
“These algorithms are very, very intelligent – they’re designed to be. And they will outsmart a poor data strategy every time.”
Lehman works in an academic research environment, where commercialization is not necessarily an immediate pressure. But in the startup world, where time-to-market is critical for success, developers may not even be paying adequate attention to creating bias-free products.
“These algorithms are very, very intelligent – they’re designed to be. And they will outsmart a poor data strategy every time.”
“It’s definitely the Wild West,” Lehman observed. “It’s astounding to me how little attention is being paid to these critically important quality assessments.”
“So that’s where we need to develop close partnerships with our colleagues in industry, as well as in government and in the academic centers, to make sure we are implementing these tools in a responsible manner.”
Ensuring that developers are not replicating existing biases in the healthcare system will be vital for generating trust and acceptance of AI tools.
ACCESSING THE LARGE-SCALE DATA REQUIRED FOR AI SUCCESS
Part of the bias problem stems from the lack of diverse data available for training algorithms, MacRae pointed out.
Collecting large-scale data that is complete, accurate, up-to-date, and representative of typical populations is a perennial challenge for analytics professionals, and data-thirsty AI models are putting the squeeze on an already bone-dry pipeline.
“One of the problems is that we’re still recruiting patients at an individual level instead of creating an environment where data contribution, in an anonymized and secure way, is an expectation of interacting with the healthcare system,” he asserted.
“It’s almost inconceivable that we study only a very small subset of individuals and then make inferences for the whole population. We should be doing the exact opposite. We should be studying large populations and then applying those insights to individuals.”
Failing to include enough meaningful data during training and validation can have significant downstream effects on the accuracy and applicability of algorithms, agreed Alistair Erskine, MD, MBA, chief digital health officer at Partners HealthCare.
“The source of the data, as well as its volume and quality, can dramatically impact the model you end up building,” he said.
“Models have to be retested on parallel datasets that include patients in other locations, with other genders and races, and other clinical comorbidities before you can really validate your results. If you don’t pay attention to these steps, you are introducing potential bias and impacting the decisions that come out the other end.”
Getting access to additional high-quality data for validation purposes can be even more difficult than pulling together a training data set to begin with, noted Lehman.
“It’s very easy to say that you’ve externally validated your models. But if you’re doing the validation at the hospital down the street because it’s easy to get their data, you’re probably validating the model on a very similar population to the one that trained it. That isn’t going to create a tool that will produce good results for everyone,” she said.
“For our breast cancer risk identification algorithm, which was developed at Mass General, we’re doing validation at places like Henry Ford Hospital in Detroit, which has a much higher percentage of African American women.”
“If we want our models to be robust across rages, ages, and different socioeconomic situations, we need to develop those partnerships and make that extra effort to incorporate other populations,” she stressed.
CHANGING THE PARADIGM OF DATA COLLECTION FOR AI DEVELOPMENT
Preventing bias and unleashing data for researchers will require some wholesale changes to the way the healthcare industry – and its consumers – view data on a conceptual level, says MacRae.
“In other industries, data sharing is de facto,” he said. “When you use Uber or Google, you are contributing data to the model and helping those companies refine their algorithms and improve the ability for you and all other consumers to use the model going forward.”
It might be unsettling for consumers when they actually think about how their data is being commoditized, but on the whole, it doesn’t actually change their behavior or even stop them from using a specific company’s services, he pointed out.
“The world did not fall apart on the day that the news broke about Facebook and Cambridge Analytica,” he said. “Everyone was mad. Everyone took a second to think about what was happening. But they didn’t stop using Facebook on a massive scale. They recalibrated a little, but they’re still sharing their personal information.”
That isn’t to say that healthcare should be as cavalier towards data privacy and security as some of the world’s tech giants, MacRae made clear.
But there is room for improvement in terms of freeing up data that consumers are already generating.
“I think healthcare is at least as important as connecting with your old college roommates. But the reality is that there is this disproportionate sense of indignation or anger when we think about a healthcare organization using our data for research,” he continued. “And we haven’t really done anything to explain why that should be different.”
Convincing patients that they can see improvements to their health if they share data more freely is an important place to start, he said.
“I think healthcare is at least as important as connecting with your old college roommates.”
“There is a sense that people hold onto their health data because they don’t get anything in return. If we could tell patients that we are using their clinical data and their TV viewing data to predict the risk of dementia, would that not be worth allowing us to get access, in a secure environment, to both of those things? But nobody is making those arguments. That’s a huge missed opportunity.”
MacRae was not the only presenter at the World Medical Innovation Forum to express frustration with the inability to access data on the scale required to train AI robustly.
Nearly every session, no matter what its focus, featured some mention of the need to educate patients about contributing their data to research and the challenge of unlocking data siloes created by competing business incentives, legacy technologies, and a reluctance to expose organizations to any level of risk.
“What we need is the equivalent of a USB port for data,” said Erskine. “Right now, there is no way for me to go into a healthcare organization, plug my algorithm into a standardized interface, and be sure that it will work in the same way no matter where I go.”
“Part of that is because there is no universal model for data in healthcare; part of it is because of the transactional health IT systems we have. But if we could build those standards and lock down the security of the data so that the risk is lower than it is right now, we could make progress in leaps and bounds.”
MacRae doesn’t believe that the problem can be completely attributed to cumbersome technologies or outdated regulations, however. The attitudes of the healthcare industry itself are also to blame.
“It actually requires an active effort to stop the technologies, approaches, and ideas from outside of healthcare,” he said. “I think we’ve done that partially because we’re conservative, partly because we’re afraid, and partly because the risks are legitimately high.”
“But we have to break out of that mentality soon, otherwise we’re going to keep having an 18th century healthcare system in a 23rd century world.”
“I believe that one of the core professional mandates for physicians is to lead us there in a way that is responsible and truly beneficial to the patient. If we don’t do that, we are compromising our long-term vision for how healthcare should evolve.”
MEASURING SUCCESS IN AN AI-DRIVEN WORLD
Artificial intelligence may be uncannily adept at identifying cancers or flagging high-risk patients, but it can’t necessarily determine how humans define success for an AI implementation.
Financial return on investment may be part of it; improvement on clinical quality measures might be another.
“We have to break out of that mentality soon, otherwise we’re going to keep having an 18th century healthcare system in a 23rd century world.”
But the real metric for providers is whether or not artificial intelligence helps them inform patients better about their personal risks, catch developing conditions earlier, and empower consumers to make the right choices for their own individual care.
“I’m looking forward to having the ‘easy button’ for evidence-based medicine,” said Erskine. “When a patient comes into the hospital, I want to be able to push a button and find every other patient in the history of my data repository that is just like them.”
“As a clinician, sometimes I think that I make decisions based on my heart or my gut: things that don’t have evidence that I can point to.”
“But the truth is that the evidence does exist, and I’m not making these decisions based on nothing. It’s just that the evidence is buried somewhere so deep in the data that I can’t find it within the time I have to treat that person. Artificial intelligence can do that for me.”
Lehman’s definition of success is similar. Lack of education and awareness among breast cancer patients are some of her biggest foes, but artificial intelligence is helping her win the battle.
“No matter what clinics I work in, we see women who are so surprised when we tell them they have breast cancer,” she said. “It happens in women who have no history of it in their family, who eat right and exercise. And it happens in women who have an incredibly high risk who come in with advanced cancers, some of which have already metastasized. The most common response I get is, ‘I just didn’t know.’”
“This is the year we are going to change that. This is the year we can start to use artificial intelligence to inform women of their risk with a level of accuracy that we have never had before. We are going to get rid of that feeling of ‘I didn’t know; I never heard.’ And as a result, we are going to save lives.”
“That is the promise of AI in healthcare, and that is achievable right now with the technology and the data we have today.”