Talk title: Designing and Benchmarking Healthcare Large Language Models

Talk abstract: It is perhaps no exaggeration that large language models (LLMs) have taken the world by storm due to their widespread access coupled with their excellent generative and discriminative abilities. However, the major strengths of such LLMs such as their creative outputs and stochasticity, are perhaps their largest liabilities for using and adapting LLMs in healthcare. Moreover, unlike natural language processing (NLP) tasks where a discordance between standard NLP metrics and human preferences may be acceptable, such a discordance is not acceptable for clinical preferences and decision making. To help bridge this gap, in this talk, I will present some of our work on adapting open-source and closed-source LLMs for clinical text summarization tasks alongside a framework for robust technical and clinical evaluation. I will end by briefly describing how unimodal LLMs can be extended to multi-modal settings for operating on medical images.

Bio: Dr. Chaudhari is an Assistant Professor in the Integrative Biomedical Imaging Informatics at Stanford (IBIIS) section in the Department of Radiology and (by courtesy) in the Department of Biomedical Data Science. He leads the Machine Intelligence in Medical Imaging research group at Stanford and has a primary research interest that lies at the intersection of artificial intelligence and medical imaging. His group develops new techniques for accelerated MRI acquisition and downstream image analysis, extracting prognostic insights from already-acquired CT imaging, and developing new multi-modal deep learning algorithms for healthcare that leverage computer vision, natural language, and medical records. Dr. Chaudhari has won the W.S. Moore Young Investigator Award and the Junior Fellow Award from the International Society for Magnetic Resonance in Medicine. Dr. Chaudhari has also been inducted into the Academy of Radiology’s Council of Early Career Investigators in Imaging program. He also serves as the Associate Director of Research and Education at the Stanford AIMI Center and is an advisory board member of the Precision Health and Integrated Diagnostics Center.