https://orcid.org/0000-0002-8323-662X
Abstract
Amidst rapid advancements in artificial intelligence and machine learning-enabled medical devices (AI/ML-MD), this article investigates the regulatory challenges highlighted in the current academic literature. Using a PRISMA-guided scoping review, 18 studies were selected for in-depth analysis to highlight the multifaceted issues in regulating AI/ML-MD. The study's findings are organized into key themes: adaptive AI/ML, usability and stakeholder engagement, data diversity and use, health disparities, synthetic data use, regulatory considerations, medicolegal issues, and cybersecurity threats. The scoping review reveals numerous challenges associated with the regulation of AI/ML-based medical devices, reflecting various sustainability pillars. The study advocates for integrating sustainability principles into the materiovigilance ecosystem of AI/ML-MD and proposes a novel sustainable ecosystem for AI/ML-MD materiovigilance. This proposed ecosystem incorporates social, economic, and environmental sustainability principles to create a comprehensive and balanced regulatory approach. By presenting a thorough analysis of regulatory challenges, the study provides policymakers with a nuanced understanding of the complex landscape surrounding these technologies. This insight enables the development of informed strategies and solutions to address regulatory gaps and ensure the safe and effective integration of AI/ML-MD into healthcare systems.
References
1.
Rejikumar
, G.
,
Sreedharan
, V. R.
,
Arunprasad
, P.
,
Persis
, J.
, and
Sreeraj
, K. M.
, 2019
, “Industry 4.0: Key Findings and Analysis From the Literature Arena
,” Benchmarking
, 26
(8
), pp. 2514
–2542
.10.1108/BIJ-09-2018-02812.
Xu
, X.
,
Lu
, Y.
,
Vogel-Heuser
, B.
, and
Wang
, L.
, 2021
, “Industry 4.0 and Industry 5.0—Inception, Conception and Perception
,” J. Manuf. Syst.
, 61
, pp. 530
–535
.10.1016/j.jmsy.2021.10.0063.
Basulo-Ribeiro
, J.
, and Teixeira
, L.
, 2024
, “The Future of Healthcare With Industry 5.0: Preliminary Interview-Based Qualitative Analysis,” Future Internet
, 16
(3
), p. 68
.10.3390/fi160300684.
Tortorella
, G. L.
,
Saurin
, T. A.
,
Fogliatto
, F. S.
,
Rosa
, V. M.
,
Tonetto
, L. M.
, and
Magrabi
, F.
, 2021
, “Impacts of Healthcare 4.0 Digital Technologies on the Resilience of Hospitals
,” Technol. Forecast. Soc. Change
, 166
, p. 120666
.10.1016/j.techfore.2021.1206665.
Gomathi
, L.
,
Mishra
, A. K.
, and
Tyagi
, A. K.
, 2023
, “Industry 5.0 for Healthcare 5.0: Opportunities, Challenges and Future Research Possibilities
,” 2023 Seventh International Conference on Trends in Electronics and Informatics (ICOEI
), Tirunelveli, India, Apr. 11–13, pp. 204
–213
.10.1109/ICOEI56765.2023.101256606.
Maddikunta
, P. K. R.
,
Pham
, Q.-V.
,
Prabadevi
, B.
,
Deepa
, N.
,
Dev
, K.
,
Gadekallu
, T. R.
,
Ruby
, R.
, and
Liyanage
, M.
, 2022
, “Industry 5.0: A Survey on Enabling Technologies and Potential Applications
,” J. Ind. Inf. Integr.
, 26
, p. 100257
.10.1016/j.jii.2021.1002577.
Saraswat
, D.
,
Bhattacharya
, P.
,
Verma
, A.
,
Prasad
, V. K.
,
Tanwar
, S.
,
Sharma
, G.
,
Bokoro
, P. N.
, and
Sharma
, R.
, 2022
, “Explainable AI for Healthcare 5.0: Opportunities and Challenges
,” IEEE Access
, 10
, pp. 84486
–84517
.10.1109/ACCESS.2022.31976718.
Pradhan
, B.
,
Bhattacharyya
, S.
, and
Pal
, K.
, 2021
, “IoT-Based Applications in Healthcare Devices
,” J. Healthcare Eng.
, 2021
, p. 6632599.10.1155/2021/66325999.
World Health Organization
, 2020
, Guidance for Post-Market Surveillance and Market Surveillance of Medical Devices, Including In Vitro Diagnostics
, World Health Organization
, Geneva, Switzerland.10.
Muehlematter
, U. J.
,
Daniore
, P.
, and
Vokinger
, K. N.
, 2021
, “Approval of Artificial Intelligence and Machine Learning-Based Medical Devices in the USA and Europe (2015–20): A Comparative Analysis
,” Lancet Digital Health
, 3
(3
), pp. e195
–e203
.10.1016/S2589-7500(20)30292-211.
Curchoe
, C. L.
, 2023
, “Unlock the Algorithms: Regulation of Adaptive Algorithms in Reproduction
,” Fertil. Steril.
, 120
(1
), pp. 38
–43
.10.1016/j.fertnstert.2023.05.15212.
Ahmed
, A.
,
Aziz
, S.
,
Abd-Alrazaq
, A.
,
Farooq
, F.
,
Househ
, M.
, and
Sheikh
, J.
, 2023
, “The Effectiveness of Wearable Devices Using Artificial Intelligence for Blood Glucose Level Forecasting or Prediction: Systematic Review
,” J. Med. Internet Res.
, 25
, p. e40259
.10.2196/4025913.
Benjamens
, S.
,
Dhunnoo
, P.
, and
Meskó
, B.
, 2020
, “The State of Artificial Intelligence-Based FDA-Approved Medical Devices and Algorithms: An Online Database
,” NPJ Digital Med.
, 3
(1
), p. 118
.10.1038/s41746-020-00324-014.
Zhu
, S.
,
Gilbert
, M.
,
Chetty
, I.
, and
Siddiqui
, F.
, 2022
, “The 2021 Landscape of FDA-Approved Artificial Intelligence/Machine Learning-Enabled Medical Devices: An Analysis of the Characteristics and Intended Use
,” Int. J. Med. Inf.
, 165
, p. 104828
.10.1016/j.ijmedinf.2022.10482815.
Food and Drug Administration
, 2019
, Developing a Software Precertification Program, Food and Drug Administration, Silver Spring, MD.16.
Food and Drug Administration
, 2021
, Good Machine Learning Practice for Medical Device Development: Guiding Principles, Food and Drug Administration, Silver Spring, MD.17.
Food and Drug Administration
, 2024
, Transparency for Machine Learning-Enabled Medical Devices: Guiding Principles, Food and Drug Administration, Silver Spring, MD.18.
Thomas
, A.
,
Lubarsky
, S.
,
Durning
, S. J.
, and
Young
, M. E.
, 2017
, “Knowledge Syntheses in Medical Education: Demystifying Scoping Reviews
,” Acad. Med.
, 92
(2
), pp. 161
–166
.10.1097/ACM.000000000000145219.
Hunger
, T.
,
Arnold
, M.
, and
Ulber
, M.
, 2024
, “Circular Value Chain Blind Spot—A Scoping Review of the 9R Framework in Consumption
,” J. Cleaner Prod.
, 440
, p. 140853
.10.1016/j.jclepro.2024.14085320.
Tricco
, A. C.
,
Lillie
, E.
,
Zarin
, W.
,
O'Brien
, K. K.
,
Colquhoun
, H.
,
Levac
, D.
, and
Moher
, D.
, et al., 2018
, “PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
,” Ann. Intern. Med.
, 169
(7
), pp. 467
–473
.10.7326/M18-085021.
Paul
, J.
,
Lim
, W. M.
,
O'Cass
, A.
,
Hao
, A. W.
, and
Bresciani
, S.
, 2021
, “Scientific Procedures and Rationales for Systematic Literature Reviews (SPAR-4-SLR)
,” Int. J. Consum. Stud.
, 45
(4
), pp. O1
–O16
.10.1111/ijcs.1269522.
Halder
, D.
,
Pradhan
, D.
, and
Roy Chaudhuri
, H.
, 2021
, “Forty-Five Years of Celebrity Credibility and Endorsement Literature: Review and Learnings
,” J. Bus. Res.
, 125
, pp. 397
–415
.10.1016/j.jbusres.2020.12.03123.
Page
, M. J.
,
McKenzie
, J. E.
,
Bossuyt
, P. M.
,
Boutron
, I.
,
Hoffmann
, T. C.
,
Mulrow
, C. D.
,
Shamseer
, L.
,
Tetzlaff
, J. M.
,
Akl
, E. A.
,
Brennan
, S. E.
,
Chou
, R.
,
Glanville
, J.
,
Grimshaw
, J. M.
,
Hróbjartsson
, A.
,
Lalu
, M. M.
,
Li
, T.
,
Loder
, E. W.
,
Mayo-Wilson
, E.
,
McDonald
, S.
,
McGuinness
, L. A.
,
Stewart
, L. A.
,
Thomas
, J.
,
Tricco
, A. C.
,
Welch
, V. A.
,
Whiting
, P.
, and
Moher
, D.
, 2021
, “The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
,” BMJ (Clinical Research Ed.)
, 372
, p. n71
.10.1136/bmj.n7124.
Nees
, J.
,
Van
, E.
, and
Waltma
, L.
, 2018
, VOSviewer Manual
, Univeristeit Leiden, Leiden, The Netherlands
.25.
Ota
, N.
,
Tachibana
, K.
,
Kusakabe
, T.
,
Sanada
, S.
, and
Kondoh
, M.
, 2020
, “A Concept for a Japanese Regulatory Framework for Emerging Medical Devices With Frequently Modified Behavior
,” Clin. Transl. Sci.
, 13
(5
), pp. 877
–879
.10.1111/cts.1278426.
Gilbert
, S.
,
Fenech
, M.
,
Hirsch
, M.
,
Upadhyay
, S.
,
Biasiucci
, A.
, and
Starlinger
, J.
, 2021
, “Algorithm Change Protocols in the Regulation of Adaptive Machine Learning-Based Medical Devices
,” J. Med. Internet Res.
, 23
(10
), p. e30545
.10.2196/3054527.
Patil
, R. S.
,
Kulkarni
, S. B.
, and
Gaikwad
, V. L.
, 2023
, “Artificial Intelligence in Pharmaceutical Regulatory Affairs
,” Drug Discovery Today
, 28
(9
), p. 103700
.10.1016/j.drudis.2023.10370028.
Farah
, L.
,
Davaze-Schneider
, J.
,
Martin
, T.
,
Nguyen
, P.
,
Borget
, I.
, and
Martelli
, N.
, 2023
, “Are Current Clinical Studies on Artificial Intelligence-Based Medical Devices Comprehensive Enough to Support a Full Health Technology Assessment? A Systematic Review
,” Artif. Intell. Med.
, 140
, p. 102547
.10.1016/j.artmed.2023.10254729.
Welzel
, C.
,
Cotte
, F.
,
Wekenborg
, M.
,
Vasey
, B.
,
McCulloch
, P.
, and
Gilbert
, S.
, 2023
, “Holistic Human-Serving Digitization of Health Care Needs Integrated Automated System-Level Assessment Tools
,” J. Med. Internet Res.
, 25
, p. e50158
.10.2196/5015830.
Alami
, H.
,
Lehoux
, P.
,
Auclair
, Y.
,
de Guise
, M.
,
Gagnon
, M. P.
,
Shaw
, J.
,
Roy
, D.
,
Fleet
, R.
,
Ahmed
, M. A. A.
, and
Fortin
, J. P.
, 2020
, “Artificial Intelligence and Health Technology Assessment: Anticipating a New Level of Complexity
,” J. Med. Internet Res.
, 22
(7
), p. e17707
.10.2196/1770731.
Panagiotou
, O. A.
,
Högg
, L. H.
,
Hricak
, H.
,
Khleif
, S. N.
,
Levy
, M. A.
,
Magnus
, D.
, and
Murphy
, M. J.
, et al., 2020
, “Clinical Application of Computational Methods in Precision Oncology: A Review
,” JAMA Oncol.
, 6
(8
), pp. 1282
–1286
.10.1001/jamaoncol.2020.124732.
Tarakji
, K. G.
,
Silva
, J.
,
Chen
, L. Y.
,
Turakhia
, M. P.
,
Perez
, M.
,
Attia
, Z. I.
, and
Passman
, R.
, et al., 2020
, “Digital Health and the Care of the Patient With Arrhythmia: What Every Electrophysiologist Needs to Know
,” Circ.: Arrhythmia Electrophysiol.
, 13
(11
), p. E007953
.10.1161/CIRCEP.120.00795333.
Ferryman
, K.
, 2020
, “Addressing Health Disparities in the Food and Drug Administration's Artificial Intelligence and Machine Learning Regulatory Framework
,” J. Am. Med. Inf. Assoc.
, 27
(12
), pp. 2016
–2019
.10.1093/jamia/ocaa13334.
Alloza
, C.
,
Knox
, B.
,
Raad
, H.
,
Aguilà
, M.
,
Coakley
, C.
,
Mohrova
, Z.
, and
Boin
, É.
, et al., 2023
, “A Case for Synthetic Data in Regulatory Decision-Making in Europe
,” Clin. Pharmacol. Ther.
, 114
(4
), pp. 795
–801
.10.1002/cpt.300135.
Moingeon
, P.
,
Chenel
, M.
,
Rousseau
, C.
,
Voisin
, E.
, and
Guedj
, M.
, 2023
, “Virtual Patients, Digital Twins and Causal Disease Models: Paving the Ground for In Silico Clinical Trials
,” Drug Discovery Today
, 28
(7
), p. 103605
.10.1016/j.drudis.2023.10360536.
Kelly
, B. S.
,
Quinn
, C.
,
Belton
, N.
,
Lawlor
, A.
,
Killeen
, R. P.
, and
Burrell
, J.
, 2023
, “Cybersecurity Considerations for Radiology Departments Involved With Artificial Intelligence
,” Eur. Radiol.
, 33
(12
), pp. 8833
–8841
.10.1007/s00330-023-09860-137.
Näher
, A. F.
,
Vorisek
, C. N.
,
Klopfenstein
, S. A. I.
,
Lehne
, M.
,
Thun
, S.
,
Alsalamah
, S.
, and
Pujari
, S.
, et al., 2023
, “Secondary Data for Global Health Digitalisation
,” Lancet Digital Health
, 5
(2
), pp. e93
–e101
.10.1016/S2589-7500(22)00195-938.
Oliva
, A.
,
Grassi
, S.
,
Vetrugno
, G.
,
Rossi
, R.
,
Della Morte
, G.
,
Pinchi
, V.
, and
Caputo
, M.
, 2022
, “Management of Medico-Legal Risks in Digital Health Era: A Scoping Review
,” Front. Med. (Lausanne)
, 8
, p. 821756
.39.
Miller
, B. J.
,
Blanks
, W.
, and
Yagi
, B.
, 2023
, “The 510(k) Third Party Review Program: Promise and Potential
,” J. Med. Syst.
, 47
(1
), p. 93
.10.1007/s10916-023-01986-540.
Hernandez-Boussard
, T.
,
Lundgren
, M. P.
, and
Shah
, N.
, 2021
, “Conflicting Information From the Food and Drug Administration: Missed Opportunity to Lead Standards for Safe and Effective Medical Artificial Intelligence Solutions
,” J. Am. Med. Inf. Assoc.
, 28
(6
), pp. 1353
–1355
.10.1093/jamia/ocab03541.
Meher
, B. R.
, 2018
, “Materiovigilance: An Indian Perspective
,” Perspect. Clin. Res.
, 9
(4
), p. 175
.10.4103/picr.PICR_26_1842.
Joshi
, D.
,
Sharma
, I.
,
Gupta
, S.
,
Singh
, T. G.
,
Dhiman
, S.
,
Prashar
, A.
, and
Gulati
, M.
, et al., 2021
, “A Global Comparison of Implementation and Effectiveness of Materiovigilance Program: Overview of Regulations
,” Environ. Sci. Pollut. Res.
, 28
(42
), pp. 59608
–59629
.10.1007/s11356-021-16345-543.
Kaladharan
, S.
,
Dhanya
, M.
,
Rejikumar
, G.
, and
Puthanpurayil
, J.
, 2023
, “Sustainability Triangle Framework for Digital Health Systems: A Conceptual Model for Sustainable Health Care
,” Smart Trends in Computing and Communications, SmartCom 2023, Lecture Notes in Networks and Systems
, Vol. 650, Springer, Singapore, pp. 533
–541
.10.1007/978-981-99-0838-7_4644.
Adadi
, A.
, and
Berrada
, M.
, 2020
, “Explainable AI for Healthcare: From Black Box to Interpretable Models
,” Adv. Intell. Syst. Comput.
, 1076
, pp. 327
–337
.10.1007/978-981-15-0947-645.
Barredo Arrieta
, A.
,
Díaz-Rodríguez
, N.
,
Del Ser
, J.
,
Bennetot
, A.
,
Tabik
, S.
,
Barbado
, A.
,
Garcia
, S.
, et al., 2020
, “Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges Toward Responsible AI
,” Inf. Fusion
, 58
, pp. 82
–115
.10.1016/j.inffus.2019.12.01246.
van Wynsberghe
, A.
, 2021
, “Sustainable AI: AI for Sustainability and the Sustainability of AI
,” AI Ethics
, 1
(3
), pp. 213
–218
.10.1007/s43681-021-00043-6
