Simulation, Explainability, and Safety at Scale
We can't deploy AI in clinical settings until we can guarantee it's safe. We must identify exactly how and when algorithms fail before exposing patients to those risks. However, rigorous evaluation is currently stifled by the limitations of real-world medical data - it's often noisy, legally restricted, and lacks the volume of rare "edge cases" required for proper stress testing.
To solve this, I develop systems that anchor virtual patients and diagnostic logic in hardcoded medical rules. This approach creates a fully explainable environment designed to probe where model's fail. By prioritising computationally lightweight approaches, we can generate infinite, privacy-free datasets to evaluate AI at scales that are impossible with traditional methods.
This system offers:
- Breadth: Instant access to varied clinical scenarios, from common presentations to rare conditions that would take years to find in hospital records.
- Depth: Granular feedback that pinpoints exactly why a diagnosis was correct or missed.
This aims to bridge the current large gap between experimental code and clinical tools through better:
- Validation: Developers can stress-test models against thousands of dangerous 'red flag' scenarios, allowing algorithms to either ‘fail safely’ in a simulation or demonstrate their robustness.
- Training: It provides the structured "reasoning traces" necessary to teach the next generation of AI models not just what the diagnosis is, but how to reach it.
I've developed early proof-of-principle in Headache and Multiple Sclerosis which have identified dangerous safety failures in frontier AI models - blind spots that remain invisible to standard evaluation but become clear only when tested at this level of depth and scale. With this baseline now in place, the platform's designed to be easily expanded to other clinical diagnoses and domains.
Recent papers on this:
- Headache conversational transcripts: Pre-print here
- Multiple sclerosis cases: Pre-print here
- Earlier work informing some of the methods: Pre-print here