Skip to content Skip to footer

Investigating the Effect of Data Drift on Artificial Intelligence

Data drift is a phenomena that impacts any AI model in current operation. It is essentially a change in the features distribution an AI model receives while it’s in production, thereby leading to a decline in the model’s performance. A visible impact in imaging AI, for instance, could be an algorithm becoming less reliable at flagging certain pathologies in specific images. The reasons leading to data drift may vary, but the result often leads to corrupted data, interruptions in processes and a plethora of other challenges for today’s data architectures.

The prominence of data drift can particularly be felt within clinical AI algorithms. The causative factors include new protocols being introduced, older machines being replaced, or changing practices for image acquisition. Data drift contributes to the fallibility of AI algorithms. Due to the changes in protocols and metadata, AI may overlook vital findings leading to dissatisfaction amongst users.

The first wave of AI integration options like Point Solutions, Marketplaces and Platforms respond differently when confronted with data drift. To illustrate this, the example of Intracranial Hemorrhage (ICH) protocols at one location over a span of 2.5 years is used. A retrospective analysis was done where an advanced AI orchestration method (AIO) was compared with a traditional rule-based metadata orchestration (MBO). It was found that AIO identified 66,581 ICH scans, whereas MBO recognized only 61,902 scans, indicating a 7% decrease for ICH. The average weekly decrease was 0.1% with a variation margin of 4.2%. The highest weekly decrease observed was 17.5% for ICH.

These statistics highlight the challenges posed by data drift in AI integration strategies, which are discussed in more detail in the whitepaper “The Clinical AI Scorecard: How Different Integration Approaches Handle Deployment Challenges”. A more comprehensive study detailing the benefits of using an AI platform for implementation and deployment can be found in this downloadable whitepaper.

Leave a comment

0.0/5