The specialty symposia and forums got underway on Monday, April 17, during HIMSS23, the annual conference of the Chicago-based Healthcare Information & Management Systems Society (HIMSS). And this year, the HIMSS Conference is being held at McCormick Place Convention Center in Chicago, the first time in several years that it has been held in Chicago.
The Machine Learning and AI Forum involved a number of presentations and panels all focused on artificial intelligence- and machine learning-related topics. A particularly important panel took place in the morning entitled “Building and Evaluating Business Cases for ML/AI in Health.” It was moderated by Adair Chesley of the HIMSS Analytics North America Advisory Board. Chesley was joined by Line Helen Linstad, a senior advisor in health analytics at the Norwegian Center for E-Health Research; Tatyana Fedotova, director of global data, platforms and partnerships at Johnson & Johnson; and Chris Gill, senior manager in customer engineering at SambaNova Systems.
Adair asked Fedotova, “Tatyana, what issues have not been resolved around business value?”
“A lot of you have faced this question,” Fedotova said. “Let me just highlight for you some steps that I see as crucial” in AI development processes. “The first is acquiring the right data sets, having the right data quality. Developing those AI algorithms and testing and validating them. And from the pharma perspective, the key is, how do we integrate those AI algorithms into systems in real time, and measure them in real time? That’s a key challenge. For us, patient identification early on in the disease journey is so important, to identify them, and put them into the right treatments and pathways. We’ve used AI algorithms to accelerate those processes. Also, a point around partnership. The way we’re looking at these last-mile challenges is by utilizing partnerships. So we’re looking at the right AI partnerships, and the right vendor partnerships. I also wanted to highlight that we’ve observed that the most successful partnerships involve understanding what the end-to-end processes look like. We have to understand what the physician workflows look like. Just throwing AI algorithms out there isn’t going to work.”
What’s more, she said, “The second point is around adoption. AI algorithms, the only way they can provide value able insights, is if they’re adopted widely by HC professionals. AI algorithms won’t be widely used by physicians if they’re not part of the clinical workflow.”
“Line, can you give an example of leaders overcoming resistance to using AI in healthcare in Norway?” Chesley asked Linstad.
“In the Norwegian context, what we’re doing, though small in the American context, is quite big for us. Together with my colleagues, we have a mandate from the Norwegian Health Ministry to follow implementation of AI in a hospital trust in southern Norway”—the Wester Wiken Hospital Trust in the Oslo metropolitan area. “It involves a community of 500,000, which is quite a big in the Norwegian context,” where the total population of the country is just 4.5 million. “This hospital is not a university hospital, but they’re very forward-leaning, and wanted to implement AI. And they worked with clinicians in radiology, asking what major areas could leverage AI to alleviate pressures and challenges in their daily lives. Conventional X-rays, and lung and thorax x-rays and MS controls. The director said, OK, let’s go for a procurement process. The studied the procurement process during COVID, and in fact, they had to do all the procurement work on Teams. To do this procurement—we have a national actor helping to purchase everything. They met with them and they helped them out and said we have to follow an innovative way of procurement” for diagnostic imaging-related algorithm purchase, and we needed a market dialogue. So we went to the market, saying we would like to buy two or three algorithms. So we did that first experiment. And in the dialogue with the market, the vendors aid, we’re not selling single algorithms, we’re selling platforms. So they agreed to buy a platform. And as a political scientist, I see that this small approach becomes bigger. And so you need a national strategy, and a regional strategy, of platformization,” she said. “And now, we’re at the stage where they’re trying to implement one of the apps in the platform, related to conventional x-ray for broken legs and arms. They’ve acknowledged that the vendors aren’t coming into the radiology department but another part of the hospital. And they’ve appointed three AI ambassadors in the hospital. They will be the ones working to get this project going, and the testing, and if they get a little success here trying out these small algorithms, that’s a start” for a broader initiative.”
“Chris, what are the first steps that executives should take, to identify and prioritize AI, when there are so many problems involved?” Chesley asked Gill.
“Yes, there are so many problems involved” in algorithm development,” Gill said, “and we need to start with open minds. You’re never going to be one step ahead of where everything is going. So you have to find what speaks to you and get going. Rely on the AI champions, adopt projects nimbly and quickly. Healthcare is a highly regulated industry, but also one of the biggest research industries as well. Work with partners, vendors, whatever the case might be. So really take that approach of exploration, guided by governance. You’ll be able to go through your list of opportunities. And really setting up those horizons for yourselves, to approach opportunities as they come at you. And realize that not everything’s going to work right away. The big ideas, those tend to be six or twelve months out to accomplish. But solving paperwork challenges, those can come right away.”
“On the pharma side, we are really looking into data,” Fedotova said. “It’s important to understand that the data foundation is essential. Poor data, poor analysis, poor governance, lead to poor outcomes. So a three-step process. First step is the right data: clean data, good data, well-governed data. Second, historically, some of the medical data: the models need to incorporate the correct populations accurately tl eliminate data bias. Second, we’re focusing on data at the right time. We’ve seen … A couple of years ago, we were looking at the data form a batch-receiving standpoint. Now, we’re moving into real-time data. And from the AI perspective, the algorithms are going to be developed in real time. And what are the risks associated with that? And the third part at the enterprise level is around data governance, which is absolutely critical. We need to make sure the right people have the right access to the right data sets. For us, data is a backbone. High quality of the data is absolutely important. Reliable, free from errors, accurate, algorithms.”
“The use cases are manifest right now,” Gill noted. “For example,” he said, “I’ve been working with a university program in Asia. They’re teaching future doctors. They’ve always had a problem generating test data. How do you prove these students know what they’re doing? Because of the ability to create images from a training set, this university has generated testable data, meaning here’s a student looking at an x-ray generated by an algorithm. That’s really allowed that university program to explore how well they’re able to upload diagnostic processes. Prior to that, they had to go through all sorts of regulatory issues, and anonymizing data, etc. Now, these cases are being generated so that data is being created that can be used to test students.”
“What’s the timeline for generating that kind of data?” Chesley asked.
“The reality is that your first model is going to take you a lot longer than you expect, and your second one is going to take a lot less time,” Gill said. “Creating the base building blocks is key. This university took about six months to generate their first data set, based on open CT scans. They used that as a base training set. And now they’re planning to publish their model out to the world so that anyone can build on that set and evolve it further. So, six to twelve months to get your first project up and running, that’s not unreasonable. You can move fast. And the second project that this time will undertake will be up and running in a few weeks. These are platforms, and they’re pretty standard platforms now. The vast majority of use cases are now coming into a well-defined process of model building or model iteration. So these processes can move pretty fast.”
“Just a few years ago, you had to be a well-endowed university or academic medical center” to do serious AI work, Chesley said. “So what are some of the risks that you’ve experienced along the way, with your clients?”
“I’m talking to a lot of executives about data leakage,” Gill said. “People are worried that their data might be sucked into these models. Solving that can take several different paths. There is an ability to run these things internally; there’s also the ability to run these things externally and avert data leakage, including through segregation of models, not just segregation of data. You need to stay away from some of the big public models.”
“In Norway and Europe, it’s coming from the research area,” Linstad said, referring to where the funding and sponsorship are coming from. “My colleagues are all AI and machine learning analysts. And their biggest challenge is actually to get access to the data. And most of the activities today, at least in the Norwegian HC system, involves in-house production and testing.”
“What are the key takeaways you’d all like to leave with our audience?” Chesley asked.
“The lines between providers and vendors will blur” over time, Gill said. “And the pace of change and the ability for AI/ML to change, is shifting. My wife’s mentor says, “Shift happens.” The iterative approach is happening right now, so you’ll have to become very comfortable trying things out, and find ways to sandbox these experiments and letting them grow. Democratizing this technology—we have to find a way to do that in a powerful but sandboxed way. Protect this work but democratize it.”
“I would leave with three short points,” Fedotova said. “I think that for start-ups, think about the last-mile challenge, and how to overcome it. And it will be a process. And we need strong partnerships with you for successful implementations. Data quality is an absolute foundation. And the third thing, I really wanted to highlight to you that pharma companies are interested in developing algorithms and successful partnerships with vendor startups.”
And Linstad said that “I think that the main point would be, bring people together, bring all the different stakeholders together, and be flexible. The benefit you think of today will not be the benefit you see at the end. You have to be respectful of others and look at it as a collective good that we want to realize together.”
