Medical Regulation and Patient Safety

Security firm Mindgard reports it easily 'jailbroke' the AI system behind Utah’s new prescription-refill bot, convincing it to triple an OxyContin dosage, classify methamphetamine as an unrestricted treatment, and assert that COVID-19 vaccines had been suspended, despite no such regulatory action. The Doctronic-built system is being piloted by Utah’s Department of Commerce in a regulatory sandbox to let certain chronic-care patients renew medications without a doctor’s direct sign-off, marking the first U.S. program where AI participates directly in prescription renewals. Mindgard says the attacks worked by feeding fake 'regulatory updates' to alter the bot’s baseline knowledge and that Doctronic closed support tickets twice in January after being warned, even though the vulnerabilities persisted. Doctronic counters that controlled substances like OxyContin are categorically excluded from its programs and that, in practice, all prescriptions still get a licensed-physician review and must pass internal eligibility and protocol checks. The episode underscores how thin guardrails and weak red-teaming around medical AI can create serious patient-safety risks and is likely to intensify calls for tighter federal and state oversight before such systems are scaled beyond early pilots.