Weil Institute Data Science Team showcased at Symposium on Artificial Intelligence in Learning Health Systems (SAIL)
Above: Negar Farzaneh, PhD, with her team’s abstract at SAIL
Above: Sardar Ansari, PhD, with two of the Data Science Team’s abstracts at SAIL
Among the only forty-eight invited projects, every member of the Data Science Team had an abstract accepted for display.
Contact:
Katelyn Murphy
Marketing Communications Specialist, Weil Institute
mukately@med.umich.edu
---
The Data Science Team at the University of Michigan Max Harry Weil Institute for Critical Care Research and Innovation was invited to showcase its expertise on a global stage at the 2024 Symposium on Artificial Intelligence in Learning Health Systems (SAIL). Launched in 2020 to explore the integration of artificial intelligence (AI) techniques into clinical medicine, SAIL unites leading groups across healthcare and AI spaces with the goal of fostering collaboration between methodologists and key decision makers, and better integrating the clinical informatics and machine learning communities.
"It was really encouraging to see our critical care models and solutions received with such enthusiasm by people who are leaders in this space and are familiar with the next frontiers in healthcare AI."
Sardar Ansari, PhD
Research Assistant Professor, Emergency Medicine
Director, Weil Institute Data Science Team
University of Michigan
Representing the Weil Institute at this prestigious meeting were Drs. Sardar Ansari and Negar Farzaneh, Data Science Team Director and Senior Data Scientist, respectively. Between the two of them, they presented a range of posters representing the breadth of their team’s work – from developing new ways of monitoring predictive models after they have been deployed, to exploring how physicians and AI models could complement each other to improve disease diagnosis.
“It was really encouraging to see our critical care models and solutions received with such enthusiasm by people who are leaders in this space and are familiar with the next frontiers in healthcare AI,” said Dr. Ansari. “We are honored to have had this opportunity to share our work.”
Notably, every abstract submitted by the Data Science Team was chosen for display at the symposium, meaning that four out of the only forty-eight original research projects selected for the showcase were from the Weil Institute. Further adding to the accolades, Dr. Ansari was also chosen to present one of the abstracts as an official SAIL Spotlight Talk—an honor reserved for only five projects total.
“It was a real honor to see Weil's work recognized at SAIL,” said Dr. Andrew Admon, Assistant Professor of Epidemiology and Internal Medicine and a co-author on two of the featured abstracts. “We've long felt that this work was transformative for patient care. Weil's strong showing at SAIL shows that it's just as transformative for the field of healthcare AI.”
“To have such representation at this prestigious meeting is a testament to the Weil Institute’s dedication to innovation using deep integration,” said Dr. Kevin Ward, Executive Director of the Weil Institute and Professor of Emergency Medicine and Biomedical Engineering. “Being able to combine rich expertise in artificial intelligence with extensive clinical experience is resulting in these transformative predictive analytics that will save lives.”
Abstracts Presented
2024 Spotlight Talk) Monitoring Dataset Shift in Clinical AI/ML Models during the Post-Deployment Phase
Authors: Connor J. O’Brien, Andrew J. Admon, Brandon C. Cummings, Joseph M. Blackmer, Kevin R. Ward, Sardar Ansari
When models are integrated into clinical workflows, there is often a discrepancy between the data the model was trained on and the data that the model is applied to in real-time. This phenomenon, called dataset shift or drift, may cause a decline in the model’s predictive power and lead to flawed clinical decisions. In this study, the team developed a framework to identify, quantify, and diagnose data drift in clinical AI/ML models.Strategies for Deploying Artificial Intelligence to Complement Physician Diagnoses: An Application to Acute Respiratory Distress Syndrome Diagnosis
Authors: Negar Farzaneh, Elizabeth Lee, Kevin R. Ward, Sardar Ansari, Michael W. Sjoding
Advances in AI have led to models achieving human-level diagnostic performance across many health conditions using clinical images, including radiographic images. However, a growing gap exists between studies describing AI’s diagnostic capabilities using deep learning versus efforts to investigate when and how to integrate AI systems into a real-world clinical practice to support physicians and improve diagnosis. To address this gap, the team examined the strengths and weaknesses of physicians and a previously published AI model in interpreting chest X-rays. Additionally, the team investigated potential strategies for AI model deployment and physician collaboration to determine their potential impact on diagnostic accuracy.Evaluating the Performance of Predictive Clinical AI/ML Tools After Deployment
Authors: Brittany Baur, Andrew Admon, Brandon Cummings, Connor J. O’Brien, Joseph Blackmer, Kevin R. Ward, Sardar Ansari
Evaluating the performance of an early warning system after it has been deployed is a challenging task. If the model correctly predicts a deterioration event, the clinician may take actions to prevent the predicted deterioration. If those actions are successful, the deterioration is averted, and the label that the model is being tested against will become negative despite the fact that the model’s positive prediction was correct. Without the context of the clinician intervention, the model will be evaluated as if it was wrong--a concept known as confounding medical interventions (CMI). In this study, the team proposed and then simulated a novel method of estimating the true performance of a model in the presence of CMI.External Validation and Comparison of a General Ward Deterioration Index Between Diversely Different Health Systems
Authors: Brandon C. Cummings, Connor J. O’Brien, Joseph M. Blackmer, Negar Farzaneh, James D. Glassbrook, Michael D. Roebuck, Michael W. Sjoding, Kevin R. Ward, Sardar Ansari
Dataset shift is a major last-mile challenge when implementing a predictive model outside its original clinical environment. In this study, the team applied the Weil Institute’s PICTURE analytic, which predicts deterioration in general ward patients, to a previously unseen dataset from a second institution with a significantly different patient population and benchmarked the model’s performance against that of early warning systems currently in use.
About the Weil Institute
The team at the Max Harry Weil Institute for Critical Care Research and Innovation is dedicated to pushing the leading edge of research to develop new technologies and novel therapies for the most critically ill and injured patients. Through a unique formula of innovation, integration and entrepreneurship that was first imagined by Weil, their multi-disciplinary teams of health providers, basic scientists, engineers, data scientists, commercialization coaches, donors and industry partners are taking a boundless approach to re-imagining every aspect of critical care medicine. For more information, visit weilinstitute.med.umich.edu.