Upskilling staff to work with large language models will be a hurdle healthcare organizations will face when trying to implement the technology at their institutions, Karandeep Singh, MD, inaugural chief health artificial intelligence officer of UC San Diego Health, told Becker’s. 

“It’s really nice in theory to say we want to use large language models to improve or to solve this problem that we’re having. But it’s a whole other thing to educate folks on the differences between something like zero-shot learning and few-shot learning, and how to actually put that into practice,” he said. 

According to Dr. Singh, educating teams on these fundamentals will be essential, as it can accelerate the initiation of pilot projects and facilitate a quicker understanding of the additional work required beyond zero-shot learning, the process in which models are instructed to answer questions without prior training.

Another challenge with working with large language models and AI in healthcare is how organizations make informed decisions about model selection and adapt to the ever-evolving advancements in open-source alternatives, Dr. Singh said.

“Now that the open models are catching up, there’s a really big question around how we make decisions about which model to use,” he said. “Do we potentially swap out the models every two or three weeks when a better open source model comes out? You can’t view this as a fixed thing, you have to make sure you understand how you are going to adapt the latest thing when it becomes available.”

Open-source models such as Mixtral that are freely available have now emerged as formidable contenders, even outperforming their proprietary counterparts in certain cases, according to Dr. Singh. 

Previously, preferences leaned heavily toward proprietary models due to their perceived superiority. However, with the rise of open-source models catching up, Dr. Singh said decision-makers are grappling with the dilemma of choosing the most suitable model for their specific needs.

As organizations navigate this evolving landscape, considerations about infrastructure and upskilling are integral to ensuring the successful integration of large language models into healthcare organizations’ workflows. The ability to adapt to the latest technological advancements and make informed decisions will be key in harnessing the transformative power of these models.