Monitoring & AI

As large language models are rapidly adopted in hospitals to analyze patient data and support clinical decisions, keeping them safe is a top priority. Traditionally, software is monitored for "dataset drift," assuming performance drops only when the incoming data changes. However, researchers found that this method does not work for complex language models. Because of how these models are trained, they can fail in unpredictable ways even if the data looks normal. The study argues that healthcare systems must shift to "capability-based monitoring," a new safety method that continuously tests what the AI is actually capable of doing in real-time.