AI-generated from publicly available materials.A new council in healthcare is shifting focus from pre-deployment AI governance to ongoing monitoring after deployment, emphasizing the importance of continuous oversight.
The Council for Healthcare AI Responsibility and Safety (CHAIRS), co-founded by Waleed Mohsen, aims to address the critical issue of how AI systems perform in real-world patient interactions after they have been implemented. While traditional governance processes ensure AI tools meet certain criteria before deployment, Mohsen highlights a significant oversight: the lack of ongoing evaluation once these systems are in use. This gap could lead to unintended consequences as AI operates at a scale beyond human capacity, with errors potentially becoming systematic rather than isolated incidents.
CHAIRS' inaugural report, titled "The Adherence Gap," explores the discrepancies between intended clinical protocols and actual patient care delivered by AI. It underscores the importance of timely detection of these gaps, as manual oversight is often limited and costly. The report reveals that many organizations do not conduct thorough quality assurance, relying instead on clinician notes that may not accurately reflect the actual interactions, creating potential blind spots in monitoring AI behavior.
Mohsen advocates for independent continuous monitoring, akin to practices in cybersecurity, to ensure that AI systems are held accountable post-deployment. He believes that establishing robust oversight mechanisms will not hinder AI adoption but will instead enhance organizational credibility and readiness to address inquiries from stakeholders regarding AI performance. As healthcare continues to integrate AI, the emphasis on accountability and transparency will be crucial for building trust and ensuring patient safety.