國立虎尾科技大學 |

Development of a Framework and Checklist to Guide the Translation of AI Systems for Clinical Care /

Record Type:	Language materials, printed : Monograph/item
Title/Author:	Development of a Framework and Checklist to Guide the Translation of AI Systems for Clinical Care // Ayomide Owoyemi.
Author:	Owoyemi, Ayomide,
Description:	1 electronic resource (218 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-04, Section: B.
Contained By:	Dissertations Abstracts International86-04B.
Subject:	Health sciences. -
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31693757
ISBN:	9798384461074

Development of a Framework and Checklist to Guide the Translation of AI Systems for Clinical Care /
Owoyemi, Ayomide,

Development of a Framework and Checklist to Guide the Translation of AI Systems for Clinical Care /Ayomide Owoyemi. - 1 electronic resource (218 pages)

Source: Dissertations Abstracts International, Volume: 86-04, Section: B.

Artificial intelligence (AI) technologies have historically fallen short of their translational aims because the technological perspective has dominated how the systems fit and function in the actual world. AI tools have generally shown promise in their development. Few, meanwhile, have been able to apply them to patient management in actual environments. For instance, a management decision tool for diabetes management was developed and implemented in a Utah hospital, which faced the challenge of not providing all the information that physicians and patients wanted to make decisions about managing Type 2 Diabetes. Any technology application in the real world, particularly in a clinical context, necessitates careful consideration of its deployment's technical and social aspects. Understanding "the technical, cognitive, social, and political factors in play and incentives impacting integration of AI into health care workflows" was emphasized in the National Academy of Medicine paper. It is crucial to know the environment in which the technology will be used, how it will seamlessly integrate with current processes, and how users will react to it. Examples of clinical applications that have brought attention to these issues and their current limitations include IBM's Watson and the Epic Sepsis Model. Despite the numerous proof-of-concept publications in this field, the lack of robust frameworks for supporting the development and management of these tools is one of the main barriers to their adoption in healthcare. There is a paucity of specific guidance and rigorous best practices for people designing and developing AI solutions targeted at clinical settings and use cases. Findings from a review conducted by Gama et al. highlighted the need to develop an AI-specific implementation framework because there is an unrealized opportunity to draw insights from implementation science using theoretical and practical insights to accelerate and improve the implementation of AI in clinical settings.There have been frameworks and guidelines proposed recently. Salwei and Carayon built a sociotechnical systems framework for AI, building on the work of Salwei et al. This framework recognizes the social and technical components of work related to AI's successful design and implementation. Their model shows how an AI can be integrated into clinical processes if appropriate for the work system or context in which it is used. Two other models are examples of models that are limited in their application and focused on trials, performance, and comparison. They are only helpful in one stage of the AI life cycle. These are the Consolidated Standards of Reporting Trials-AI extension and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis.Most existing frameworks and studies have focused on the technological, clinical, and ethical aspects of AI tool development and usage and less on how the tool will eventually integrate and perform in the clinical setting. In other cases, they do not address the AI system's design, development, and pre-implementation stages. This study aimed to develop a determinant framework that will guide the design, development, and subsequent implementation of AI systems with a focus on the human, cognitive, and governance factors in clinical settings relevant to the success of AI tools. In addition, this study was also designed to produce a checklist based on the developed framework to help operationalize the framework and make it more accessible and easier to use. The research had three objectives. Objective one revolved around assessing the factors that influenced the implementation and use of the Epic Sepsis tool. This first part of the research involved a primary study that used a multi-method approach of focus group discussion and in-depth interviews to investigate the factors influencing the implementation and effectiveness of the Epic Sepsis System at UI Health and clinicians' experiences with the same tool. These were based on three frameworks: The non-adoption, abandonment, scaleup, spread, and sustainability, Salwei and Carayon sociotechnical framework for AI system implementation, and Sittig & Singh's 8-dimensional model. This part of the study was conducted at the University of Illinois Hospital and Health Sciences Systems (UI-Health). Four Epic Sepsis management team members at UI-health were recruited for the focus group discussion, and ten clinicians were recruited for the in-depth interview. The interviews were recorded and transcribed. The data from both the focus group discussion and in-depth interviews were analyzed using a deductive and inductive content analysis. Objective two was to develop a framework and checklist to guide the predictive evaluation of AI tools in clinical settings. The results of a literature review were merged with the results of the primary study conducted to create the Clinical AI Sociotechnical Framework (CASoF). Based on this framework, the checklist's first draft was developed and shared to a team of panelists for evaluation and review using a Delphi approach. The initial checklist had four overall categories corresponding to the four stages in the development and deployment process, with 15 sub-categories corresponding to the domains of CASoF that were important in each of the stages. The checklist was refined over two rounds of the Delphi study to produce a final checklist with three overall categories: planning, design, and development. It also proposed implementation with 34 questions across its sub-categories to make it less cumbersome and focused. The third objective was to evaluate the checklist for completeness and usefulness while developing the Discovery Partners Institute (DPI) diabetes AI tool. This assessment was conducted with three purposively selected members of the DPI-diabetes tool, which is presently being developed at UI Health. Interviews were conducted with each participant. The interviews were recorded and transcribed. The data from interviews were analyzed using an inductive content analysis approach. All qualitative analyses were done using Dedoose, a qualitative and mixed methods data analysis web-based software that organizes, codes, annotates, and visualizes data. Findings from this study show that the successful implementation and integration of AI in a clinical setting depends on a balanced focus on the clinical settings' technical and social dynamics. Also identified were critical gaps in existing frameworks primarily focused on technical specifications or ethics, neglecting the comprehensive sociotechnical dynamics essential for developing and implementing AI systems in clinical environments. CASoF addresses this gap by providing a framework and a structured checklist that guides the planning, design, development, and implementation stages of AI systems in clinical settings. The checklist emphasizes the importance of considering the value proposition, data integrity, human-AI interaction, technical architecture, organizational culture, and ongoing support and monitoring, ensuring that AI tools are technologically sound, practically viable, and socially adaptable within clinical settings. CASoF represents a step forward in bridging this divide, offering a holistic approach to AI deployment that is mindful of the complexities of healthcare systems. CASoF is the first checklist that addresses sociotechnical factors across the phases of the AI cycle with a general approach that is not limited to any specific condition or use case in clinical care. The checklist aims to help ensure that AI solutions for clinical use cases are better built for impact, adoption, and success. The usefulness of the checklist was validated through the interviews conducted with the DPI-Diabetes team members, who found the checklist helpful in thinking through and evaluating the ongoing tool development process. They also found it helpful to think beyond the technical aspects of the tool to the attendant sociotechnical dynamics that are important to its eventual adoption and success. Future research will focus on applying the checklist to different AI development and implementation projects to see how well it helps identify gaps and likely challenges. This will also help to iterate on and improve the checklist.

English

ISBN: 9798384461074Subjects--Topical Terms:

1179212
Health sciences.
Subjects--Index Terms:

Sociotechnical framework

Development of a Framework and Checklist to Guide the Translation of AI Systems for Clinical Care /
LDR:10188nam a22004693i 4500 001 1157885
005 20250603111435.5
006 m o d
007 cr|nu||||||||
008 250804s2024 miu||||||m |||||||eng d
020 $a 9798384461074
035 $a (MiAaPQD)AAI31693757
035 $a (MiAaPQD)0799vireo532Owoyemi
035 $a AAI31693757
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Owoyemi, Ayomide, $e author. $3 1484170
245 1 0 $a Development of a Framework and Checklist to Guide the Translation of AI Systems for Clinical Care / $c Ayomide Owoyemi.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 1 electronic resource (218 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-04, Section: B.
500 $a Advisors: Boyd, Andrew Committee members: Layden, Brian; Di Eugenio, Barbara; Salwei, Meghan; Harwani, Sagar.
502 $b Ph.D. $c University of Illinois at Chicago $d 2024.
520 $a Artificial intelligence (AI) technologies have historically fallen short of their translational aims because the technological perspective has dominated how the systems fit and function in the actual world. AI tools have generally shown promise in their development. Few, meanwhile, have been able to apply them to patient management in actual environments. For instance, a management decision tool for diabetes management was developed and implemented in a Utah hospital, which faced the challenge of not providing all the information that physicians and patients wanted to make decisions about managing Type 2 Diabetes. Any technology application in the real world, particularly in a clinical context, necessitates careful consideration of its deployment's technical and social aspects. Understanding "the technical, cognitive, social, and political factors in play and incentives impacting integration of AI into health care workflows" was emphasized in the National Academy of Medicine paper. It is crucial to know the environment in which the technology will be used, how it will seamlessly integrate with current processes, and how users will react to it. Examples of clinical applications that have brought attention to these issues and their current limitations include IBM's Watson and the Epic Sepsis Model. Despite the numerous proof-of-concept publications in this field, the lack of robust frameworks for supporting the development and management of these tools is one of the main barriers to their adoption in healthcare. There is a paucity of specific guidance and rigorous best practices for people designing and developing AI solutions targeted at clinical settings and use cases. Findings from a review conducted by Gama et al. highlighted the need to develop an AI-specific implementation framework because there is an unrealized opportunity to draw insights from implementation science using theoretical and practical insights to accelerate and improve the implementation of AI in clinical settings.There have been frameworks and guidelines proposed recently. Salwei and Carayon built a sociotechnical systems framework for AI, building on the work of Salwei et al. This framework recognizes the social and technical components of work related to AI's successful design and implementation. Their model shows how an AI can be integrated into clinical processes if appropriate for the work system or context in which it is used. Two other models are examples of models that are limited in their application and focused on trials, performance, and comparison. They are only helpful in one stage of the AI life cycle. These are the Consolidated Standards of Reporting Trials-AI extension and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis.Most existing frameworks and studies have focused on the technological, clinical, and ethical aspects of AI tool development and usage and less on how the tool will eventually integrate and perform in the clinical setting. In other cases, they do not address the AI system's design, development, and pre-implementation stages. This study aimed to develop a determinant framework that will guide the design, development, and subsequent implementation of AI systems with a focus on the human, cognitive, and governance factors in clinical settings relevant to the success of AI tools. In addition, this study was also designed to produce a checklist based on the developed framework to help operationalize the framework and make it more accessible and easier to use. The research had three objectives. Objective one revolved around assessing the factors that influenced the implementation and use of the Epic Sepsis tool. This first part of the research involved a primary study that used a multi-method approach of focus group discussion and in-depth interviews to investigate the factors influencing the implementation and effectiveness of the Epic Sepsis System at UI Health and clinicians' experiences with the same tool. These were based on three frameworks: The non-adoption, abandonment, scaleup, spread, and sustainability, Salwei and Carayon sociotechnical framework for AI system implementation, and Sittig & Singh's 8-dimensional model. This part of the study was conducted at the University of Illinois Hospital and Health Sciences Systems (UI-Health). Four Epic Sepsis management team members at UI-health were recruited for the focus group discussion, and ten clinicians were recruited for the in-depth interview. The interviews were recorded and transcribed. The data from both the focus group discussion and in-depth interviews were analyzed using a deductive and inductive content analysis. Objective two was to develop a framework and checklist to guide the predictive evaluation of AI tools in clinical settings. The results of a literature review were merged with the results of the primary study conducted to create the Clinical AI Sociotechnical Framework (CASoF). Based on this framework, the checklist's first draft was developed and shared to a team of panelists for evaluation and review using a Delphi approach. The initial checklist had four overall categories corresponding to the four stages in the development and deployment process, with 15 sub-categories corresponding to the domains of CASoF that were important in each of the stages. The checklist was refined over two rounds of the Delphi study to produce a final checklist with three overall categories: planning, design, and development. It also proposed implementation with 34 questions across its sub-categories to make it less cumbersome and focused. The third objective was to evaluate the checklist for completeness and usefulness while developing the Discovery Partners Institute (DPI) diabetes AI tool. This assessment was conducted with three purposively selected members of the DPI-diabetes tool, which is presently being developed at UI Health. Interviews were conducted with each participant. The interviews were recorded and transcribed. The data from interviews were analyzed using an inductive content analysis approach. All qualitative analyses were done using Dedoose, a qualitative and mixed methods data analysis web-based software that organizes, codes, annotates, and visualizes data. Findings from this study show that the successful implementation and integration of AI in a clinical setting depends on a balanced focus on the clinical settings' technical and social dynamics. Also identified were critical gaps in existing frameworks primarily focused on technical specifications or ethics, neglecting the comprehensive sociotechnical dynamics essential for developing and implementing AI systems in clinical environments. CASoF addresses this gap by providing a framework and a structured checklist that guides the planning, design, development, and implementation stages of AI systems in clinical settings. The checklist emphasizes the importance of considering the value proposition, data integrity, human-AI interaction, technical architecture, organizational culture, and ongoing support and monitoring, ensuring that AI tools are technologically sound, practically viable, and socially adaptable within clinical settings. CASoF represents a step forward in bridging this divide, offering a holistic approach to AI deployment that is mindful of the complexities of healthcare systems. CASoF is the first checklist that addresses sociotechnical factors across the phases of the AI cycle with a general approach that is not limited to any specific condition or use case in clinical care. The checklist aims to help ensure that AI solutions for clinical use cases are better built for impact, adoption, and success. The usefulness of the checklist was validated through the interviews conducted with the DPI-Diabetes team members, who found the checklist helpful in thinking through and evaluating the ongoing tool development process. They also found it helpful to think beyond the technical aspects of the tool to the attendant sociotechnical dynamics that are important to its eventual adoption and success. Future research will focus on applying the checklist to different AI development and implementation projects to see how well it helps identify gaps and likely challenges. This will also help to iterate on and improve the checklist.
520 $a This will also involve developing comprehensive feedback and guidance information to help users of the checklist better understand how to respond to the gaps and challenges identified and how to react to and fix them.
546 $a English
590 $a School code: 0799
650 4 $a Health sciences. $3 1179212
650 4 $a Information science. $3 561178
650 4 $a Bioinformatics. $3 583857
653 $a Sociotechnical framework
653 $a Clinical settings
653 $a Framework
653 $a Checklist
653 $a Implementation
690 $a 0800
690 $a 0566
690 $a 0723
690 $a 0715
710 2 $a University of Illinois at Chicago. $b Biomedical and Health Informatics. $e degree granting institution. $3 1484171
720 1 $a Boyd, Andrew $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 86-04B.
790 $a 0799
791 $a Ph.D.
792 $a 2024
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31693757