CIO Survey: AI Strategies Still a Work in Progress

In a recent survey of CIOs and other health IT leaders, two-thirds of respondents said they are still developing their AI strategy, while 20 percent conceded it is "limited or fragmented. Only 2% of IT leaders deemed their electronic health record’s AI offerings as “mature.” 

Mudit Garg, CEO of Qventus, the AI-driven care operations company that conducted the survey, and Joseph Sanford, M.D., chief clinical informatics officer and Director of the Institute for Digital Health & Innovation (IDHI) at the University of Arkansas for Medical Sciences (UAMS), recently sat down with Healthcare Innovation to dig deeper into the survey’s results. 

Here are a few more highlights of the survey: 

• When asked about their primary method for evaluating the ROI of AI investments, improved margins were ranked at the top (26 percent), followed by cost reductions (24 percent) and staff productivity and clinician satisfaction (each 16 percent).

• 54% identified enhancing operational efficiency and reducing costs as the strategic objective that will have the biggest impact in care operations.

• More than half of the survey respondents indicated that they have a formal AI governance committee that collectively oversees and manages the development and deployment of AI tools.

Healthcare Innovation: Dr. Sanford, do the responses in this survey about AI readiness line up with your own experience at UAMS?

Sanford: I think the responses are pretty typical of the gamut of healthcare as a whole. The first bifurcation is academic vs. non-academic. Where do you have the the extra resources and and non-commercial interest in the state of the art? Within that, you have academics that have connected undergraduate colleges of engineering, where you have a real startup entrepreneurial mentality, and you’ve got a pipeline of development so that you are eager and interested in doing pilots like this. UAMS is a graduate organization. We have undergraduate programs, but they're very specifically limited to nursing and other allied health professionals. We mostly train grad students, so we have partnerships with the University of Arkansas at Fayetteville, probably most notably in their engineering school. But we don't have that same pipeline. That's something we've been working to build. 

At the University of Arkansas, we have the full spectrum of passionate physicians who are early adopters and technologists at heart. We also have the more traditional clinician who is patient-focused at all times, and doesn't necessarily have that interest, and is a little more hesitant to be one of those bleeding-edge doctors. And then we have those who don't want anything to do with it. It’s not that they're uninterested; they just have other responsibilities. And then within the commercial space and the non-academic health systems, you have a whole different set of personnel. 

HCI: So if you are a CIO or CMIO at a small or or medium-sized community health system, do you have a lot less in the way of resources and might that limit your choices around AI strategy?

Garg: To some extent, I feel like all of healthcare right now feels like we have limited resources and not as many choices. There's continuously declining reimbursement and labor shortages, but you’re right, the extent of it probably stronger in smaller places. I think one of the unifying factors across most CIO conversations has been not AI for AI’s sake, but understanding it is a really great tool in our toolbox. It can solve a lot, but really focusing on the problem that we're solving. It needs to be the right clinical thing to do, but it also needs to have impact on the staff. It also needs to have impact in terms of financial margins and quality. In a world of limited resources, you just need to be a lot more thoughtful around that problem. We are seeing interest, not just within health systems, but physician groups as well.

HCI: In the survey, only 14% of respondents told Qventus that their current AI strategy was comprehensive and well-defined, while two-thirds said that it was still in the development phase and 20% said it was limited or fragmented. That 14% number seems pretty low, but I guess people haven't been working on it that long yet and trying to figure out a system-wide strategy.

Garg: Yes, first of all, I would say even if you build a strategy, things are changing fast enough that it feels like it's never fully done. You know, in three to six months, the learnings probably change a fair amount. But I think it is fair to say that the degree of change possible implies that people are still grappling with what it means. For many folks, the first step involves understanding the operational pieces, the financial pieces, the patient pieces, as well as understanding the tech and then bridging that gap. I think that's the phase a lot of institutions are in. 

HCI: The survey found only 11% self report that they fully implemented responsible AI capabilities, including governance and risk management tools. Dr Sanford, could you talk about that at UAMS? Has the governance approach evolved or shifted over time as as you've gotten more sophisticated?

Sanford: Yes, it has. We're moving away from deterministic models and tools that can be vetted from a reproducibility standpoint, very specifically, into a probabilistic space. When you do that, you really can't make individual one-off determinations, because it's just not amenable to that. So you have to set up a set of guiding philosophies and some guard rails that then you can do your unit testing to make sure that it fails safely in the traditional sense of that concept and that it abides by your organization's overriding principles. 

In our circumstance, those overriding principles are our patients’ autonomy, our data privacy, transparency and trust. We are more than willing to engage the patient using the variety of tools available to us and try new things, recognizing that we're going to learn in doing, but that we go back to our fail-safes, where we have an absolute sense of data privacy and we have an absolute understanding of data reuse and where the data flows and where it lives. 

We are transparent about how we're using the data, and we're transparent about when our patients are talking to an AI assistant vs. a flesh-and-blood human. Those are new concepts in our governance. 

HCI: Mudit, as you listen to Dr. Sanford talk about the governance approach there at UAMS, does it seem pretty similar to other health systems you work with? Or do you see lots of different approaches to governance?

Garg: In general, what he is describing in terms of the shift in governance, I think everyone is grappling with that, and everyone is in different phases of that. People agree on the core principles around data use and and upfront declaration of who you're talking to. But I will say that there's a big spectrum of the degree to which is people have adjusted to the new reality. 

HCI: One of the things I think the survey didn't ask, but I was curious about is organizations creating the role the chief health AI officer, and whether you're seeing that a lot — and where they fit into the executive team and reporting structure? 

Garg: In some cases CIOs or CMIOS are filling this role, and in some cases, there are chief AI officers or chief transformation officer roles being created to do it. I think there's very much a shift, where you're seeing more operational involvement, people getting closer to the clinical care and the workflows and the technology and bringing all those pieces together. And as I said, in some cases, those are new roles, but in almost all cases, it is someone changing what was expected of them before to bringing all these things together much more tightly. 

HCI: Does the survey address the question of how people are thinking about working with their EHR vendor on AI innovations, and how much they feel like they can rely on the the EHR vendor, and how that impacts decisions about working with startups or third-party AI companies?

Sanford: It depends, first and foremost, on who your EMR partner is. There’s a lot of variability in that space, so their interest and who they focus on, what market segment, is going to drive their adoption rate. I think that there are always advantages to doing things entirely native within the EMR. There are disadvantages in terms of expertise and focus. When you have a comprehensive EMR, no organization can focus everywhere all at once, right? It's just not possible. So I think a healthy ecosystem allows organizations to make enlightened decisions about whether they want to try and build something themselves or partner. Every organization should have the opportunity and capacity to choose among those options. I think that's just a general good for healthcare as a whole.

Garg: Folks are seeing that there are options where things are very well integrated in the EHR, and there are options to go deep into specific areas and drive benefit in those areas. Survey respondents said they are figuring out which are the problems that really matter to the organization, in terms of margin, in terms of ROI, in terms of patient benefit. Where you want to make sure you are the best and you can't afford to wait, those are the places that tend to look for the best of breed.

HCI: We’re also seeing more examples of health systems, especially ones that have venture arms attached to them, both deploying and developing along with an AI vendor, but also making an investment in them.

Garg: Absolutely, I think the true success of AI is in understanding the job to be done very deeply, and therefore working very closely with customers is critical to the success of the work. So being very close to the customer is the most important thing that we can have in the development of these new tools. 

HCI: You’ve mentioned how rapidly all of this is changing. Are you going to redo this survey next year?

Garg: That is the intention. I am sure there will be new challenges, problems and learning opportunities that the next year will present.