Healthcare Uses Specialized Language. It Needs Specialized AI, Too.

Healthcare has always struggled with translation. Not between English and other languages but between the dialects spoken by different corners of the industry. Radiologists write in precise measurements. Psychiatrists craft narrative assessments. Oncologists think in probabilities and protocols. Even the word “stable” means different things to different specialists—sometimes “unchanged,” sometimes “improving,” occasionally “precarious but holding.” Now we’re asking a type of generative AI known as generalist large language models (LLMs) to bridge these gaps, and the early results reveal a troubling pattern: AI excels at speed but stumbles on nuance. It’s an especially worrisome dynamic, given how many generalist LLMs are doubling down on their healthcare offerings. These generalist systems—which include ChatGPT, Claude, and Gemini—excel at many enterprise applications because of their ability to synthesize information broadly across domains, make unexpected connections, and handle the full spectrum of business communications. In contrast, specialized generative AI models understand not just what information to retrieve but also how that information operates within a specific domain’s decision-making framework. As researchers at Stanford’s Human-Centered AI Institute have noted, generalist large language models show promise in clinical settings but remain vulnerable to contextual errors and overgeneralization when deployed beyond controlled demonstrations. Another Nature Medicine study demonstrated that both generalist and “medical-domain” large language models are not ready for autonomous clinical decision-making because they often fail to follow workflow instructions and are sensitive to input variations, underscoring the importance of structured oversight. As leaders who have confronted this issue repeatedly ourselves in recent years in one distinct area of healthcare—we have built generative AI systems for prior authorization—we’ve evolved some useful methods to address it that we believe others in the industry would do well to adopt and evolve. But the first step, as in many realms of change management, starts with awareness. The Promise Is Real The potential applications of LLMs are, of course, deeply exciting. They can generate patient education materials or clinical notes in seconds, flag incomplete documentation to keep care on track, or even suggest differential diagnoses. The simple act of summarizing a very sick patient’s electronic health record—which can swell to 20,000 pages of information—can be game changing for clinicians. But healthcare’s most critical communications often happen at the industry’s busiest intersections, where providers and insurers discuss treatment decisions, where specialists interpret each other’s findings, when patients try to decipher their physician’s notes, or where clinicians and administrators form operational strategies. And in each case, parties are communicating not only from vastly different perspectives; they’re also using language that is distinct to their experience and particular professional discipline. This is where we’ve watched generalist language models fail in ways that matter. Where the Models Break Down Our own examples are telling. While testing LLMs processes in the realm of prior authorization—where insurers review whether proposed treatments are medically necessary—we assigned generalist models to review hospital charts and arrive at a recommended decision to approve the treatment or not. The results were often unsettling. A patient with knee pain was denied an MRI. Why? The model noted that policy required six weeks of physical therapy first, but since it had found no documentation of those six weeks, the denial seemed logical. The problem: The chart documented prior ACL repair surgery—which, according to the patient’s insurance policy, automatically qualifies for immediate imaging regardless of the six-week rule. The model read every word, but it couldn’t understand what those words meant together. A patient presented with neck pain and hand weakness, and the model quite incorrectly recommended conservative treatment. Why? The neurological exam documented a “markedly positive Hoffman’s sign”—a reflex signaling spinal cord compression that, as clinicians would know, demands urgent imaging. The problem: The LLM failed to correctly interpret the Hoffman’s-related finding and therefore missed a red flag. An asthma patient’s chart documented three courses of oral prednisone over 12 months, and the model incorrectly recommended against treatment escalation. Why? The clinician had not characterized the patient’s asthma as “severe,” and the model had been trained to only escalate treatment for severe cases. The problem: Any pulmonologist would know that this patient was steroid dependent and therefore had severe disease, which warranted biologic therapy. It counted the prescriptions, in other words, but couldn’t interpret what they signified. These aren’t edge cases. They represent a fundamental challenge: Generalist LLMs don’t understand that healthcare communication is actually dozens of specialized dialects, each with its own logic, assumptions, and critical details that go unstated because everyone in that specialty already knows them. The problem is further complicated when generalist LLM’s struggle to form a temporal sequence of past events from often incomplete data—all the while under deadline pressure. In a randomized clinical trial published in JAMA Network Open, physicians using a generalist LLM did not significantly improve diagnostic reasoning compared with conventional resources, highlighting that such LLMs require further refinement and integration strategies to be safely effective. The Organizational Blind Spot Here’s the uncomfortable parallel: Healthcare organizations deploying generalist LLMs often don’t know what they don’t know about these systems—just as the LLMs themselves don’t know what they don’t know about clinical dialects. Executives see impressive demonstrations where AI summarizes charts accurately, and they assume the technology generalizes. They don’t realize that a model performing well on cardiology notes may catastrophically misread oncology protocols, or that accuracy on routine cases predicts nothing about performance on complex patients. The models can also miss the complex relationships between data sets—each hindered by distinct limitations of its own—that occur over long time spans and across multiple visits, tests, specialists, and hospitalizations. As leaders at Stanford Medicine have observed, real-world clinical AI deployments are far more complex than demonstrations suggest, often revealing operational and interpretive challenges only after they have been integrated into practice. Those failures are often invisible until they reach patients or create costly disputes, because clinicians and administrators may not be aware of the model’s limitations and its ability to fail in the first place. They assume that the model performs well (just as they assume that the chemistry lab is accurately measuring blood sugar levels) until they encounter situations where it is clear that the model failed and might even be dangerous. A situation such as the asthma model failing to recognize that three prednisone courses signal that the patient’s asthma is severe or poorly controlled. The (often appropriate) result is that clinical staff will resist adoption. Physicians and nurses have watched decades of IT implementations ignore workflow and clinical realities. They’re not being Luddites when they distrust another “efficiency” tool—they’re being appropriately skeptical of systems built by people who don’t understand the clinical nuances that matter. The solution we’ve evolved—and that we recommend for others in healthcare—requires adding an additional training step that, effectively, turns a generalist LLM into a specialist LLM. It’s a step that organizations typically skip: systematic dialect mapping with domain experts before deployment. What implicit knowledge does each specialty rely on? Which clinical findings carry urgency that isn’t explicitly documented? Where do specialties use identical terms to mean different things? And where should the model—and the organization—slow down to ensure accuracy? Implicit in this strategy is transparency about limitations. Harvard experts have similarly warned that without clear governance structures and clinical oversight, the rapid deployment of AI in health care risks amplifying bias and undermining patient trust. Training programs that show clinicians exactly what the AI sees—and crucially, what it misses—transform skeptics into collaborators. When staff understand that the system will flag “Hoffman’s sign” as a data point but not recognize its urgency, they know where to watch for errors and how to provide corrective feedback. They become partners in improvement rather than obstacles to adoption. Building Accountable Translation Our approach treats language models the way hospitals treat medical residents: capable but requiring structured supervision. We call it the Clinical Alignment and Accountability Framework, and we’ve used it thus far to build specialized LLMs for health plans to address both technical and organizational challenges posed by generalist LLMs. Require the LLM to explicitly link every output to source evidence. It should be able to cite specific clinician notes in the electronic health record, clearly communicating the time the note was written, while also citing the algorithm’s confidence in the accuracy of its conclusions. Polished summaries are dangerous; we need visible reasoning chains that clinicians can verify. Institute active clinical oversight of specialty-specific adaptation. Rather than one model trying to master all of medicine, healthcare organizations should consider using one of the leading generalist LLMs as a foundation but extend its capabilities to encompass the unique data characteristics of specific medical specialties. How? Via structured review, where clinicians correct the model’s interpretations until it learns how that field actually reasons. It adds a step akin to clinical internship, yes, but that builds critical process integrity. Calibrate speed with risk. Technology and clinical teams can permit the LLM to run routine tasks—e.g., checking the completeness of documentation, matching codes—automatically. But they must add a human review step, even at the risk of slowing down results, for situations involving complex cases with multiple conditions or conflicting findings. Maintain transparent reasoning logs of how the LLM arrived at its conclusion. The logs should include the data used by the model, the sequence of analytical steps, and the rationale for each step. This way, when errors occur, technology teams can understand why and update the system accordingly. While initially training the model, ensure that specialty clinicians and data scientists work together. The purpose is to identify medical situations where an LLM might easily make errors. By training the model to identify such situations, it effectively builds a specialized knowledge repository on which it can rely in the future. Ensure a continuous feedback loop. Establish technical protocols to capture and codify every instance where a clinician corrects the model, thus enabling model developers to close off blind spots. This framework creates visible disagreement that humans can resolve. When a hospital’s LLM and an insurer’s model interpret a case differently, both should flag the discrepancy, show their reasoning, and escalate for clinical review, rather than silently generating conflicting conclusions that will cost precious days or weeks to adjudicate— likely at the cost of the patient’s health. The Broader Imperative Prior authorization represents just one communication pathway in a healthcare system that overflows with them. The same translation problems emerge when specialists consult across disciplines, when hospitals report quality metrics to regulators, when researchers extract findings from electronic health records, and when patients try to understand their own medical information. As healthcare organizations rush to deploy language models across these areas, we face a choice. We could, of course, treat LLMs as black-box efficiency tools—faster than humans at reading charts, cheaper than hiring more staff—and accept the errors as inevitable friction. But since healthcare’s foundational tenet is “do no harm”—and not “be more efficient”—we believe we must design LLMs to preserve clinical nuance, maintain accountability, and grow more reliable through structured feedback. The technology is here. The question is whether we’ll deploy it with the rigor that healthcare demands—and the humility to recognize what we don’t yet understand—or simply move our errors faster than before.

Healthcare Uses Specialized Language. It Needs Specialized AI, Too.

Summary.

Partner Center