A 2022 report from the International Air Transport Association revealed that the odds of dying in a plane crash are extremely low, with an industry fatality risk of 0.11. This implies that a person would need to fly daily for over 25,000 years to have a 100% chance of experiencing a fatal accident, underscoring why aviation is considered one of the safest modes of transportation.
This safety record and the highly-regulated nature of aviation has prompted researchers at the Massachusetts Institute of Technology (MIT) to propose it as a model for regulating artificial intelligence (AI) in healthcare.
Assitant Professors Marzyeh Ghassemi and Julie Shah of MIT’s Department of Electrical Engineering and Computer Science (EECS), teamed up with a multi-disciplinary group of researchers, attorneys, and policy analysts from across MIT, Stanford University, Emory University, Microsoft, and others to launch a research project on this topic, with their findings recently accepted to the Equity and Access in Algorithms, Mechanisms and Optimization Conference.
The team compares the current state of AI in healthcare to the aviation industry of the 1920s – a period known as the “Golden Age of Aviation” despite a high number of fatal accidents. The National Transportation Safety Board (NTSB) cites 1929 as the record year for most aviation fatalities, with 51 reported accidents – the equivalent of 7,000 accidents per year, or 20 per day.
This crisis prompted President Calvin Coolidge to pass the Air Commerce Act in 1926, a landmark legislation regulating air travel via the Department of Commerce. The researchers suggest that this historical response could serve as a blueprint for the regulation of health AI.
The team is also drawing parallels between the aviation industry’s journey into automation and the current trajectory of AI, particularly around AI explicability and the contentious “black box” problem — that is, the debate over how much an AI model must “explain” its result to the user without biasing them to follow the model’s guidance uncritically.
These parallels led to the paper’s proposal that rigorous pilot training could serve as a model for training medical doctors to use AI tools in clinical settings. The authors also advocate for a reporting system that encourages clinicians to report safety concerns about health AI tools without fear of punishment; a “limited immunity” concept adapted from the Federal Aviation Agency’s (FAA) approach with pilots.
Medical errors, or “adverse events,” affect one in every 10 hospital patients in high-income countries on average, according to a 2023 World Health Organization report. In the existing healthcare system, however, clinicians and healthcare workers often fear reprisal for reporting these errors, including losing their medical licenses.
In response, the researchers are advocating for reform to the system that would be less focused on punishing individuals for errors and more on making systemic changes to eye-risk elements.
The paper also calls for the establishment of an independent auditing authority similar to the NTSB that would perform safety audits on malfunctioning health AI systems. The authors believe this process will help create better technological governance in the AI sector.
The team acknowledges that establishing a comprehensive regulatory system for health AI is not going to be a speedy process, especially given that the current NTSB and FAA system, with investigations and enforcement in separate bodies, took decades to establish. Despite this, the researchers are hopeful that their work will contribute to the discussions around AI regulation, and encourage further developments in safer health AI tools.
The researchers’ work was funded by an MIT CSAIL METEOR Fellowship, Quanta Computing, the Volkswagen Foundation, the National Institutes of Health, the Herman L. F. von Helmholtz Career Development Professorship, and a CIFAR Azrieli Global Scholar award.
