Utah AI prescription pilot concerns medical board

Utah has become the first state in the nation to allow an AI chatbot to renew prescriptions without direct oversight from a licensed physician, a move that has sparked a heated debate over safety and regulation. The program, operated by the company Doctronic, quietly launched in January and is now drawing sharp criticism from medical professionals and licensing authorities.

Through the chatbot, residents can bypass a traditional doctor’s visit entirely. The AI asks about their medication history, cross-references a national pharmacy database, and either authorizes a refill or escalates the request to a human physician. This setup was made possible by a regulatory sandbox in Utah, which permits state officials to waive certain licensing laws for promising artificial intelligence applications. Normally, both state and federal rules strictly limit prescribing authority to licensed medical professionals.

“We have crossed a threshold in terms of giving something that is not human a medical license, whether or not we want to call it that,” said Dr. Eric Bressman of the University of Pennsylvania. He and other critics emphasize they are not inherently opposed to AI-driven prescribing, but insist it must meet the same rigorous standards applied to human doctors.

The state’s medical licensing board was caught off guard by the launch. In an April letter, 11 board members called for the pilot to be suspended, citing serious concerns about auto-renewing medications that could cause harmful side effects or dangerous drug interactions. Dr. Alan Smith, a family physician who chairs the board, said they were essentially told, “Yes this is going on. And no, you don’t have a say in it.” The state declined to halt the program, noting that human doctors still review every refill during this initial phase.

Oversight of the program currently rests with a five-member board composed entirely of AI specialists, none of whom are medical doctors. Doctronic expects to transition to fully automated refills in the near future. Smith warns that the risks are not hypothetical, pointing out that Doctronic’s list of roughly 190 refillable medications includes blood thinners, which can become life-threatening if a patient develops internal bleeding. The American Medical Association has echoed these concerns, stating that “prescription renewals aren’t routine checkboxes.”

The situation highlights a regulatory vacuum between federal and state authorities. Medical technology is regulated at the federal level, while the oversight of medical professionals falls to states. Doctronic frames its AI as part of state-regulated medical practice, though some experts argue the company has crossed into territory that should be governed by the FDA. The company has not disclosed whether it has sought FDA approval. The agency told the Associated Press it has authorized no AI chatbots but wants to encourage innovation, reflecting a broader hands-off posture toward AI health tools.

Critics see parallels with the haphazard medicine of the early 20th century, before licensing boards and safety benchmarks existed. The Cicero Institute, a pro-AI think tank founded by Palantir co-founder Joe Lonsdale, has provided a template for other states considering similar licensing frameworks for AI medical services.

The stakes are far from theoretical. Safety researchers have warned that medical chatbots can sound authoritative while dispensing dangerous advice. Others caution that removing humans from the care loop can undermine the very outcomes the technology promises. Competitors are also racing to identify potential failure modes. Meta, for example, went as far as posing as teenagers to test how rival chatbots handle sensitive topics.

Doctronic plans to publish peer-reviewed studies later this year, but its only published paper so far was written by its own scientists and has not undergone independent review. As one Utah law professor noted, companies risk letting the technology race ahead of the evidence, potentially betraying public trust in the process.