JTS Grassroots Grant: ‘How to adapt to the dynamic nature of AI’

With the support of the JTS Grassroots Grant, a round-table session was recently organized on the use of AI systems in healthcare and mobility. The meeting at the Law Faculty of the University of Groningen on March 27 led to new insights and opportunities for future research.

Artificial intelligence (AI) offers many opportunities, especially in healthcare and mobility. AI can alleviate the administrative burden of healthcare professionals, assist in surgery, and diagnose patients. AI in mobility can make roads safer through vehicle automation, reducing road accidents. However, the use of AI in these sectors also poses many challenges, not least for the legal framework. The JTS Grassroot grant facilitated an expert roundtable session, bringing together professionals from healthcare, vehicle safety and academia to address this challenge and explore options for new approaches to vehicle and medical device safety assessment.

A specific concern relates to the EU legal framework of static pre-market approval and safety assessment of AI systems in healthcare and mobility: this approval needs to take place before these systems enter the market. This has to guarantee the safety of the vehicles and medical devices within the EU. These approval regimes, however, are not equipped to deal with ever-evolving systems. As AI systems ‘learn on the job’ through their self-learning capabilities, the AI system originally approved no longer exists in its original form when used in healthcare or in automated vehicles. As AI systems develop, new risks can arise that should require a new assessment of the safety of the AI systems.

As the round-table discussion unfolded, it became clear that a pre-market safety assessment is important, but not sufficient for ensuring safety throughout the lifetime of an AI system. Therefore, the focus should shift to post-market monitoring. Monitoring AI systems when they are used in hospitals and in vehicles, learning as they go, can facilitate the identification of new risks. These risks can be immediately addressed and mitigated as part of the post-market monitoring system.

This roundtable, initiated by JTS theme coordinator Nynke Vellinga, therefore has provided new and useful insights into how AI systems are deployed in healthcare and mobility, and has led to the identification of a new pathway for further research: post-market monitoring as a continuous eye on the AI system’s development, mitigating risks as they emerge.