AMIA 2024 Annual Symposium Tutorial onDevelopment and Evaluation of Large Language Models in Healthcare Applications |
||
Location: Franciscan A - Hilton San Francisco Union Square, San Francisco, CA, USA Time: 1-4:30pm PST, November 9, 2024 |
||
Language models are being increasingly used in natural language processing (NLP) applications, which require neither the development of a task-specific architecture nor customized training on large datasets. In particular, large language models (LLMs), such as the GPT, PaLM, and Llama-2, have demonstrated significant advances in NLP tasks. On the other hand, concerns have also been raised about the impact of these tools in health care, education, research, and beyond. One notable concern is the potential for LLMs to reinforce disparities in healthcare, as these models are typically trained on data that is historically biased against certain disadvantaged groups. Another concern is the potential for LLMs to be applied for malicious purposes. Although it is widely accepted that LLMs should be used with integrity, transparency, and honesty, how to appropriately do so and, if needed, regulate the development, and use of this technology needs further discussion.
This course provides students with an understanding of LLMs, using ChatGPT, Llama-2, and other models as examples, and their applications in health. Students will acquire knowledge of natural language processing, large language models, chain-of-though, Retrieval-Augmented Generation (RAG), and the range of prompting methods available for processing clinical text. Hands-on experience and a toolkit will provide useful skills for managing text data to solve a variety of problems in the health domain.
We believe that the proposed tutorial is timely and urgently needed for AMIA stakeholders, including informaticists from a broad array of disciplines, clinicians, software developers, and IT professionals, to learn how to develop and use these models to ensure that their potential benefits are realized while any potential risks and negative consequences are minimized. This tutorial will also likely be one of many conversations at AMIA 2024 about this issue as we learn more about LLMs, their capacity, and their potential impact on healthcare.
1-2pm. Topic 1: An introduction to LLMs and their development in the medical domain (Hua Xu)
2-3pm. Topic 2: Integration, application, and evaluation of LLMs in healthcare (Yanshan Wang)
3-4pm. Topic 3: Multimodal LLMs and their applications (Yifan Peng)
4-4:30pm. Group discussion
Hua Xu, Ph.D., FACMI, is Robert T. McCluskey Professor and Vice Chair for Research and Development at the Section of Biomedical Informatics and Data Science of Yale School of Medicine. He also serves as Assistant Dean for Biomedical Informatics at Yale School of Medicine. He has worked on different clinical NLP topics and has built multiple clinical NLP systems. Dr. Xu served as the Chair of the AMIA NLP working group between 2014-2015 and currently leads the OHDSI NLP working group. He taught NLP tutorials at various conferences such as AMIA, Medinfo, AIME, etc. Recently, Dr. Xu has worked on building foundation medical LLMs including the recently released Me LLaMA models based on the open LLaMA2 model. He will provide a generation introduction to LLMs and hands-on experience in developing medical LLMs and their applications in clinical NLP tasks such as information extraction.
Yanshan Wang, Ph.D., FAMIA, is vice chair of Research and assistant professor within the Department of Health Information Management at the University of Pittsburgh. His research interests focus on artificial intelligence (AI), natural language processing (NLP), and machine/deep learning methodologies and applications in health care. Dr. Wang has led several NIH-funded projects, which aimed to develop NLP and AI algorithms to automatically extract information from free-text electronic health records (EHRs). He has over 60 peer-reviewed publications. Dr. Wang has been actively serving the informatics and NLP communities. He has served on a Student Paper Competition Committee for the AMIA Annual Symposium and was an associate editor for MedInfo conference. He is also a regular reviewer for a dozen of prestigious journals, such as Nature Communications, JAMIA, and JBI. Wang also organized several shared tasks, including the first BioCreative/OHNLP challenge in 2018 and the second n2c2/OHNLP challenge in 2019, to encourage the informatics and NLP communities to tackle NLP problems in the clinical domain. He is also a steering committee member for the HealthNLP workshop. In 2020, he was inducted into the Fellows of AMIA (FAMIA). Dr. Wang serves as the Chair of the AMIA NLP working group between 2023-2024.
Yifan Peng, Ph.D., FAMIA, is an Assistant Professor in the Department of Population Health Sciences at Weill Cornell Medicine. His main research interests include BioNLP and medical image analysis. He has published in major AI and healthcare informatics venues, including ACL, NAACL, CVPR, and MICCAI, as well as medical venues, including Nature Medicine, Nucleic Acids Research, npj Digital Medicine, and JAMIA. His research has been funded by federal agencies, including NIH and NSF and industries such as Amazon and Google. He received the AMIA New Investigator Award in 2023.
Please contact Yifan Peng if you have question. The webpage template is by the courtesy of awesome Georgia.
