Diagnostic Capabilities of LLMs vs Physicians: A Comparative Analysis

Introduction

Large language models (LLMs) have rapidly advanced into the medical domain, raising the question of how their diagnostic abilities compare with those of human physicians. Recent studies have put LLMs through medical licensing exams, clinical case benchmarks, and specialty-specific challenges to gauge their performance. In parallel, surveys and user studies have examined whether patients are willing to trust and engage with AI-driven tools in healthcare. This report provides an in-depth analysis of LLM diagnostic performance relative to physicians across various metrics and specialties, discusses patient and clinician attitudes toward these technologies, and explores the strengths, limitations, and ethical implications of integrating LLMs into clinical decision-making.

Benchmark Performance on Medical Exams and Vignettes

LLMs on Standardized Medical Exams: Modern LLMs have demonstrated impressive results on medical knowledge benchmarks. For example, Google’s Med-PaLM (an LLM fine-tuned for medicine) was the first to exceed the passing score on United States Medical Licensing Examination (USMLE)-style questions nature.com. The successor model, Med-PaLM 2, achieved about 86.5% accuracy on a USMLE question dataset (MedQA), outperforming the original Med-PaLM by 19% nature.com. Notably, Med-PaLM 2 also showed dramatic improvements on other benchmarks like MedMCQA, PubMedQA, and the medical portions of MMLU nature.com. In controlled head-to-head evaluations, physicians actually preferred Med-PaLM 2’s answers over answers written by other physicians on 8 of 9 clinical quality metrics nature.com. In a pilot with real clinical questions, specialists still favored human-authored answers overall, but they chose Med-PaLM 2’s answers over general physician answers 65% of the time, and rated the AI’s answers as equally safe as physician answers nature.com. These findings suggest that on structured Q&A tasks, state-of-the-art LLMs can approach expert-level performance, sometimes providing answers on par with or even preferred over human clinicians’ responses.

Performance on Clinical Case Vignettes: Beyond multiple-choice exams, researchers have tested LLMs on open-ended clinical scenarios resembling NEJM-style clinical vignettes. In one study, GPT-4 (via ChatGPT) was presented with complex diagnostic cases and achieved a high accuracy in formulating correct diagnoses, often matching or exceeding the performance of physicians who attempted the same cases nature.com nature.com. For instance, Eriksen et al. found GPT-4’s diagnostic accuracy surpassed most human physicians in a challenge involving 38 difficult online case vignettes nature.com. Another evaluation by Shea et al. gave GPT-4 six real-world diagnostic cases; GPT-4 successfully suggested important diagnoses that clinicians had initially missed, highlighting the AI’s potential to broaden differentials nature.com. However, these small-sample studies also noted limitations – e.g. cases drawn from literature raise the risk that the AI had “seen” similar scenarios in training data (information leakage) nature.com nature.com. Overall, on written case simulations, advanced LLMs have shown they can mimic clinical reasoning and often list the correct diagnosis in challenging cases. This suggests a growing potential for LLMs to assist in diagnostic decision support, at least for text-based cases.

Human vs AI in Diagnostic Accuracy: A 2025 systematic review pooled results from 30 studies (covering 19 different LLMs and ~4,762 total cases) to compare diagnostic accuracy of LLMs against clinicians pmc.ncbi.nlm.nih.gov. It found LLMs’ primary diagnosis accuracy ranged widely (25% up to 97.8%, depending on the model and task), and remained generally below physician accuracy in most scenarios pmc.ncbi.nlm.nih.gov. Nonetheless, the review concluded that LLMs have demonstrated “considerable diagnostic capabilities” across varied cases and could become valuable healthcare assistants if used cautiously pmc.ncbi.nlm.nih.gov. In fact, the best-performing models showed triage accuracy (i.e. correctly determining urgency/level of care) as high as ~98% in some studies pmc.ncbi.nlm.nih.gov. These benchmarks indicate that while LLMs aren’t yet consistently outperforming doctors on diagnosis, they are no longer trivial systems – their competence now approaches the level of medical trainees or even experts on certain tasks.

Head-to-Head Trials: Direct comparisons have also been done via controlled trials. In a randomized trial, internal medicine physicians were given access to ChatGPT (GPT-4) as a diagnostic aid for case vignettes, and their performance was measured against physicians using conventional resources (like UpToDate or Google). Interestingly, doctors with LLM assistance did not significantly outperform those without it pmc.ncbi.nlm.nih.gov. The chatbot’s suggestions sometimes helped but also introduced distractions, resulting in no net improvement in diagnostic reasoning scores pmc.ncbi.nlm.nih.gov. However, the study noted that the LLM alone (when it answered cases by itself) actually outscored both groups of physicians in accuracy pmc.ncbi.nlm.nih.gov. This implies that the integration of AI into physician workflow is non-trivial – without proper training or interface, simply adding an AI assistant may not immediately boost a clinician’s performance. It highlights that realizing the potential of human–AI collaboration will require workflow changes and clinician training to effectively interpret and use AI recommendations pmc.ncbi.nlm.nih.gov.

Performance Across Medical Specialties

The diagnostic abilities of LLMs relative to specialists can vary by specialty, depending on the nature of data (text vs images) and the complexity of domain-specific knowledge:

General Medicine and Emergency Cases: LLMs have shown proficiency in internal medicine cases that rely on history and lab data. For example, one study found GPT-4 matched the differential diagnosis accuracy of attending physicians on a set of complex internal and emergency medicine cases pmc.ncbi.nlm.nih.gov. Another cross-sectional study of 40 emergency department cases reported GPT-4 achieved diagnostic accuracy comparable to ER doctors pmc.ncbi.nlm.nih.gov. These models can rapidly synthesize patient histories, presenting symptoms, and test results to suggest diagnoses. They tend to enumerate a broader differential diagnosis list than humans – one survey noted an LLM proposed on average 4–5 possible diagnoses per case versus ~1–2 by physicians nature.com. This breadth can be a double-edged sword: it means the AI is less likely to miss a rare diagnosis, but it may also include more false positives. Still, in challenging diagnostic dilemmas, this expansive knowledge is valuable. A recent study focusing on difficult cases with gastrointestinal symptoms (drawn from New England Journal of Medicine case records) illustrates this well. In that evaluation of 67 tough cases, the best-performing LLM (Anthropic’s Claude 3.5) had the correct diagnosis somewhere in its suggestions 76.1% of the time – significantly higher than the success rate of 22 experienced gastroenterologists, who on average covered the correct diagnosis only 45.5% of the time nature.com nature.com. The LLMs effectively cast a wider net, offering instructive diagnostic possibilities that many specialists didn’t initially consider, which suggests LLMs could augment physician diagnostic thinking especially for atypical cases nature.com.
Radiology: Diagnostic radiology poses a unique challenge for LLMs because it is heavily image-based. Current “general” LLMs like GPT-4 are primarily text-based, but newer multimodal versions (GPT-4Vision and others) can process images. Studies in 2024-2025 have compared these models to radiologists. In musculoskeletal radiology, for instance, Horiuchi et al. gave ChatGPT the text of patients’ history and imaging findings (without actual images) and separately tested GPT-4V on the raw images; they also had a radiology resident and an attending radiologist diagnose the same 106 cases link.springer.com link.springer.com. The result: GPT-4 (text-only) correctly diagnosed 43% of the cases, nearly identical to the radiology resident’s performance (41%), though still below the board-certified radiologist (53%) link.springer.com link.springer.com. GPT-4’s text-based accuracy was statistically indistinguishable from the resident’s, highlighting how far LLMs have come in a complex field like radiology link.springer.com link.springer.com. By contrast, GPT-4V (which analyzed the medical images directly) was correct in only 8% of cases link.springer.com – a dramatically poor showing, significantly worse than both human radiologists link.springer.com. This underscores that current vision-capable LLMs struggle with interpreting specialized medical images. In fact, multiple studies have now observed that feeding raw images to a general LLM can degrade performance relative to providing a careful text description of the image pmc.ncbi.nlm.nih.gov link.springer.com. In the musculoskeletal study, GPT-4V missed many findings that GPT-4 (given textual imaging descriptions) caught link.springer.com. The lesson: at present, LLMs excel at synthesizing textual reports of imaging, but pure image recognition likely requires domain-specific computer vision models or fine-tuning. Another radiology example comes from thoracic imaging: Güneş et al. compared GPT-4 to thoracic radiologists on chest CT cases of lung disease. They found GPT-4’s diagnostic accuracy around 81%, versus ~91% for the radiologists pmc.ncbi.nlm.nih.gov. Similarly, in ophthalmology, LLMs have been applied to analyze retinal fundus photographs; early research showed an LLM-based system could identify subtle lesions for glaucoma and macular degeneration from fundus images pmc.ncbi.nlm.nih.gov. However, these systems often combine vision models with LLMs. Overall, in image-heavy specialties, LLMs are not yet surpassing human experts – a board-certified radiologist still outperforms ChatGPT in accuracy link.springer.com – but LLMs can achieve resident-level competence on text-based image interpretation. This is promising for supporting tasks like generating radiology reports or suggesting differential diagnoses from imaging findings, especially when the LLM is provided structured descriptions rather than raw pixels link.springer.com.

S00330 024 10902 5 - link.springer.com

Figure: Summary of a radiology study comparing GPT-4 (text input) vs. GPT-4V (image input) vs. human radiologists on musculoskeletal cases. GPT-4 (text-only) achieved ~43% accuracy, comparable to a radiology resident (41%) and below an attending radiologist (53%), whereas GPT-4V (image) performed very poorly (~8% accuracy). This highlights the current gap in image understanding link.springer.com link.springer.com.

Oncology and Other Specialties: In oncology, LLMs have mostly been tested on knowledge recall and treatment recommendations rather than raw diagnosis. For example, studies have examined ChatGPT’s ability to answer patient questions about cancer therapy. One report found ChatGPT’s answers to common questions about radiotherapy and chemotherapy were on par with, or even more comprehensive than, answers from oncologists, with experts rating many chatbot responses as high-quality forbes.com. However, when it comes to making an actual cancer diagnosis, LLMs would rely on clinical data inputs (imaging, pathology, genomics) that often require other AI tools. An interesting niche example is in pathology: multi-modal LLMs have been experimented with for interpreting biopsy reports and images, but accuracy remains behind specialized image analysis algorithms. In fields like primary care and emergency medicine, LLMs are being explored for triage – e.g. determining if symptoms suggest a self-care issue vs. need ER – and as virtual “second opinions.” A systematic review noted that departments focusing on diagnosis (like radiology, internal medicine, ophthalmology, dermatology) have seen the most LLM integration so far, whereas surgical specialties lag behind due to the procedural, hands-on nature of their work pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. Notably, even in fields with data beyond text, clinicians have found ways to leverage LLMs. For instance, in dermatology, an LLM can be fed a description of a skin lesion (or even the output of an image classifier) to get a differential diagnosis. Ophthalmologists have begun using LLMs to combine imaging findings with patient data to enhance diagnostic predictions pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. These cross-domain applications are still experimental, but they point toward LLMs serving as integrative “brain-like” assistants that compile data from multiple sources (images, labs, histories) into a unified diagnostic suggestion.

In summary, across specialties, the trend is that LLMs already perform at least at the level of a junior physician on many diagnostic tasks, and in specific challenging cases they can outshine individual human experts by virtue of broader knowledge. At the same time, human specialists retain an edge in deeply domain-specific or visual diagnostic tasks, and the best results might come from combining strengths (e.g. a doctor verifying and filtering an AI’s suggestions).

Patient Preferences, Trust, and Engagement with LLMs

The adoption of AI in healthcare will ultimately depend not just on raw performance, but on whether patients and providers trust these tools. Recent surveys and behavioral studies reveal a complex picture of patient attitudes:

Public Skepticism and Conditional Trust: A late-2022 Pew Research Center survey of over 11,000 U.S. adults found significant public hesitation about AI in medicine. 60% of Americans said they would be uncomfortable if their own physician relied on AI for their diagnosis or treatment, whereas only 39% would feel comfortable pewresearch.org. More than half of respondents doubted that AI use would improve health outcomes – only 38% believed outcomes would get better with AI, while the rest felt it would make no difference or even worsen care pewresearch.org. The biggest concerns were around the human touch: 57% expected the patient–provider relationship to deteriorate if AI became common in care, fearing a loss of personal connection pewresearch.org. Privacy was another worry, with 37% thinking AI would make health data security worse (only 22% thought it would improve) pewresearch.org. Interestingly, when asked about specific applications, people’s comfort varied – for example, a strong majority (65%) said they would want AI to be used in reading images for skin cancer screening, where they perceived a tangible accuracy benefit pewresearch.org. But in more subjective areas like pain management or mental health counseling, large majorities did not want AI involvement pewresearch.org pewresearch.org. Overall, the public appears cautiously optimistic about AI catching technical problems (like skin lesions) but fears AI might erode interpersonal aspects of care or make mistakes without accountability. Indeed, 75% of Americans said their greater concern is that healthcare providers will adopt AI too fast before risks are understood, rather than too slowly pewresearch.org. This caution underscores the need for demonstrating safety and maintaining transparency with patients when introducing LLM-based tools.

60 Of Americans Would Be Uncomfortable With Provider Relying On Ai In Their Own Health Care - pewresearch.org

Figure: A Pew Research Center survey indicates 60% of U.S. adults would be uncomfortable if their health provider relied on AI for diagnosis/treatment (green bars), versus 39% comfortable (blue). Women and older adults are especially uneasy, whereas men and younger adults show relatively more openness. Higher education and familiarity with AI also correlate with slightly greater comfort【43†source.

Patient Willingness and the “Human in the Loop” Effect: Notably, patient trust in AI can improve under certain conditions. A University of Arizona study (with ~2,472 participants across diverse demographics) tested how people choose between an AI doctor vs. a human doctor under different framings. Overall, 52% of participants preferred the human physician for diagnosis and treatment, but about 47% were willing to choose an AI system in at least some scenarios healthsciences.arizona.edu. Crucially, the researchers found that when a human physician endorsed the AI as a helpful tool, patients’ acceptance of the AI diagnosis increased healthsciences.arizona.edu. In other words, if their own doctor explicitly said, “This AI can help and I will oversee it,” many patients became more comfortable with it. This suggests a complementary model – patients prefer AI to augment, not replace, their physicians. The study also found that disease severity did not significantly change preferences (participants were just as split about AI for a serious illness like leukemia as for a milder condition like sleep apnea) healthsciences.arizona.edu. However, demographic factors did play a role: older patients and those identifying as Black or politically conservative were less likely to trust an AI doctor, whereas Native American participants and those with higher tech familiarity were more open to AI involvement healthsciences.arizona.edu. These insights highlight that AI tools in healthcare might need different introduction strategies for different patient populations. Overall, the message is that many patients are potentially open to AI assistance if it’s framed as being monitored and guided by human clinicians (a form of “human-AI team” approach) healthsciences.arizona.edu healthsciences.arizona.edu. Trust is not binary – it can be nurtured by showing the AI’s accuracy, having physicians champion its use, and ensuring the patient still feels heard and in control.
Perception of Quality and Empathy: Interestingly, when patients (or other observers) have compared AI vs physician-provided advice, the AI can sometimes come out ahead in perceived quality. In a notable 2023 experiment, researchers took actual patient questions posted on an online health forum and compared the answers given by verified physicians to answers generated by ChatGPT (without telling evaluators which was which). A panel of licensed healthcare professionals rated the responses for quality and empathy. They preferred the chatbot’s answers 79% of the time, and rated ChatGPT’s responses as significantly higher in both quality and empathy than the physicians’ answers pmc.ncbi.nlm.nih.gov. The doctors’ answers tended to be more terse, whereas the AI provided more detailed explanations and a warmer tone, which likely influenced the ratings pmc.ncbi.nlm.nih.gov. Similarly, a survey of people with cancer found that AI chatbot responses to questions were seen as more empathetic than physician responses nature.com. These findings do not imply that AI is “more caring” in any true sense – but they do suggest well-designed LLMs can mimic empathy and bedside manner effectively through language. For patient-facing applications like triage chatbots or informational Q&A assistants, this ability to deliver a comforting tone along with accurate information is a strength. Patients might appreciate the thoroughness and calm demeanor of an AI’s answer in situations where human providers are rushed or use too much jargon. Of course, important caveats apply: the questions in such studies are usually general (“What should I do about symptom X?”) and not urgent life-and-death decisions. Nonetheless, these results indicate patients are willing to engage with LLM-based tools for education and advice, especially if those tools provide easy-to-understand, considerate responses. The positive reception to chatbot answers also hints that AI might have a role in relieving physician workloads for routine counseling, as long as oversight is in place (for factual accuracy).

In summary, patient attitudes are mixed – there is both excitement about AI’s potential (particularly for technical improvements in diagnosis) and significant hesitation rooted in trust and the need for human connection. Many patients appear to want the best of both worlds, accepting AI help as long as their human providers remain in the loop to personalize care and double-check the machine’s conclusions healthsciences.arizona.edu healthsciences.arizona.edu. Going forward, building trust will require clearly demonstrating where LLMs excel, being transparent about their use, and ensuring that adopting LLM-based tools actually enhances the patient experience rather than making healthcare feel impersonal.

Strengths of LLMs in Clinical Decision-Making

LLMs offer several notable strengths that could augment clinical decision-making:

Broad and Up-to-Date Medical Knowledge: LLMs like GPT-4 and Med-PaLM are trained on vast corpora including medical textbooks, research papers, clinical guidelines, and case reports. This gives them a wider knowledge base than any individual physician. They can recall rare diseases, obscure drug interactions, or atypical presentations that a human doctor might not have encountered. For example, LLMs have a broad understanding across multiple medical domains simultaneously nature.com. A gastroenterologist might not consider a neurological disorder in their differential, but an LLM might – simply because it has read about both. This versatility means LLMs can function as a “generalist expert,” ensuring that less common diagnoses are not overlooked due to specialty tunnel vision nature.com nature.com. In diagnostics, this manifests as suggesting “zebra” diagnoses or cross-specialty insights that could prompt additional tests or consults that lead to the correct answer.
Analytical Reasoning and Differential Diagnosis Generation: With appropriate prompting, LLMs can imitate the clinical reasoning process. Researchers have developed prompting techniques where the LLM is asked to summarize a case, list pertinent findings, propose a differential, and give supporting evidence for and against each possibility pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. Using this structured approach, LLMs have shown an ability to logically work through cases much like a clinician writing an assessment and plan. In fact, one trial introduced an “AI structured reflection” grid and found that ChatGPT could fill it out in a way that was scored as highly as physicians’ entries in terms of identifying key supporting and opposing findings for diagnoses pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. This suggests LLMs are not just spitting out answers; they can also articulate why a given diagnosis is likely or unlikely (drawing from their training data of medical explanations). That capability can make them useful as a teaching or cognitive aid – for instance, helping medical trainees see the rationale behind certain clinical conclusions. Moreover, LLMs excel at enumerating a thorough differential diagnosis, as noted earlier. This thoroughness can reduce premature closure in diagnostic reasoning by always providing a list of alternative diagnoses to consider. In complex cases or diagnostic dilemmas, having an AI-generated differential can counteract human cognitive biases by ensuring that possibilities aren’t dismissed too soon.
Speed and Accessibility: LLMs can analyze large amounts of text (e.g., lengthy patient histories or electronic health record notes) in seconds and provide an answer almost instantaneously. This speed could be lifesaving in scenarios where quick triage or decision support is needed (for example, in emergency settings or when consulting on multiple cases rapidly). Additionally, an LLM “never sleeps” – it is available 24/7, which means patients could potentially get preliminary advice after hours, or clinicians could use it on off shifts to double-check a case. Unlike human experts who may require formal consult requests or be limited in number, an LLM can be deployed widely, including in under-resourced areas. This offers the promise of improved access to expertise, as even clinics without a specialist on-site could input patient data into an LLM-based tool to get specialist-level insights in real time.
Consistency and Lack of Fatigue: Human performance is subject to fatigue, distractions, or varying levels of experience – an overtired junior doctor might miss something at 4 AM that a rested colleague would catch. LLMs, on the other hand, deliver a consistent performance at any hour. They won’t skip steps out of exhaustion, and they remain uniformly attentive to details provided in the input. For tasks like cross-checking medication lists for interactions, reviewing guideline criteria, or calculating risk scores, an LLM can be more reliable than a harried clinician juggling multiple tasks. Consistency is also valuable in reducing variability in care. For example, if integrated into clinical decision support, an LLM could ensure standardized adherence to guidelines (e.g., reminding a physician of the latest diagnostic criteria or to consider a certain test) nature.com nature.com. A well-tuned LLM could function as a failsafe that always checks the boxes a human might forget when busy.
Patient Communication and Education: As noted, LLMs can excel in providing empathetic, easy-to-understand explanations. This can be harnessed to improve patient understanding and satisfaction. An LLM-driven tool could, for instance, explain a new diagnosis to a patient in plain language, tailor the explanation to their level of health literacy, and answer follow-up questions patiently. This “virtual health educator” role could free up clinicians’ time while empowering patients with accurate information. Early studies have shown that patients appreciate the clarity of AI-provided information and often can’t distinguish it from doctor-written content in terms of empathy pmc.ncbi.nlm.nih.gov. Thus, LLMs might improve the communication aspect of clinical decision-making – translating medical jargon into layperson terms or composing post-visit summaries that reinforce what was discussed.
Augmenting Clinical Decision Support: Already, many Electronic Health Records have rule-based alerts (for drug interactions, etc.). LLMs could take this further by providing context-aware advice. For example, if a physician is formulating a plan for a complex patient, an LLM could proactively surface relevant considerations: “Given the patient’s history of peptic ulcer, consider avoiding NSAIDs and use acetaminophen for pain.” In diagnostic contexts, an LLM can integrate data (symptoms, labs, imaging impressions) and compare against millions of documented cases to spot patterns. This kind of probabilistic, pattern-based insight might catch subtler diagnostic clues (like a constellation of mild signs that together point to an autoimmune disease). Notably, LLMs can also incorporate latest medical literature if connected to updated databases, meaning they can suggest cutting-edge diagnostics or therapies that a busy clinician might be unaware of (for instance, pointing out an available clinical trial or a newly FDA-approved test that fits the case). In one report, an LLM (fine-tuned on oncology knowledge) was able to match patients to suitable cancer clinical trials by parsing both patient data and trial criteria, a task that normally is very time-consuming for humans mdpi.com mdpi.com. Such applications hint at LLMs becoming a real-time assistant that continuously scans the horizon of medical knowledge to supplement physician decision-making.

In summary, the strengths of LLMs lie in their knowledge breadth, reasoning ability, consistency, and communication skills. They can function as tireless medical librarians and consultants, providing second opinions, differential diagnoses, and patient education on demand. Used appropriately, LLMs could improve diagnostic thoroughness (catching what humans miss), efficiency (speeding up workups), and perhaps even bedside manner (via clear explanations). These strengths complement human clinicians, who excel in hands-on examination, intuitive judgment honed by experience, and empathy grounded in real human connection.

Limitations of LLMs in Clinical Decision-Making

Despite their potential, LLMs have critical limitations and risks when applied to clinical settings:

Hallucinations and Fabricated Information: LLMs are notorious for “hallucinating” – i.e. generating plausible-sounding but incorrect or completely fabricated information. In a medical context, hallucinations can be especially dangerous. An LLM might invent a symptom the patient never had, or cite a non-existent clinical study to justify a diagnosis. One analysis defined medical hallucinations as any fabricated detail in the AI’s diagnostic output (for example, adding a symptom or lab result that was not in the patient’s records) nature.com. These often occur because the LLM tries to make its diagnostic reasoning sound more convincing, effectively “filling in” details from its training memory. The GI case study found that hallucination frequency varied widely between models – one model (Claude 3.5) hallucinated in about 21% of its responses, while another (Gemini) did so in 63% nature.com nature.com. They also observed a trend that models with higher hallucination rates tended to have lower diagnostic accuracy (a moderate negative correlation between hallucination count and accuracy was noted) nature.com. Even when hallucinations don’t directly lead to a wrong diagnosis, they can erode trust and muddy the diagnostic process with irrelevant “red herrings.” For example, if an AI suggests a diagnosis of tuberculosis and hallucinatingly adds “patient has night sweats” when that wasn’t reported, a clinician might be led down an erroneous path. Crucially, current LLMs have no built-in mechanism to distinguish fact from their own fiction. They lack true understanding, so a fabricated output can be delivered as confidently as a verified fact. This means any LLM-generated diagnostic or recommendation must be scrutinized for accuracy. In practice, this limits using LLMs autonomously – they require human oversight to catch mistakes or hallucinations.
Errors and Gaps in Medical Reasoning: While LLMs can mimic reasoning, they do not have genuine comprehension of physiology or causality. They operate on statistical correlations, which can lead to subtle errors in logic. For instance, an LLM might suggest treating a viral infection with antibiotics because it has seen those two mentioned together in text, even though that’s incorrect medical management. In complex cases, LLMs might also struggle to weigh the relative importance of conflicting data the way an experienced clinician would. There are documented cases where LLMs have recommended clearly unsafe actions. One study reported that in a set of infectious disease scenarios, GPT-4 suggested harmful or substandard treatment plans in 16% of cases nature.com. For example, it might pick an inappropriate antibiotic or fail to adjust a medication dose in kidney failure, whereas a trained physician would know to do so nature.com. These errors arise because the AI, lacking real-world experience, might not prioritize safety the same way – it only “learns” what it saw in text, which may include outdated or biased practices. Moreover, LLMs are not good at recognizing when they don’t know something. A human doctor, when unsure, might say “I’m not certain – we need more tests or a specialist consult.” An LLM, on the other hand, is more likely to always produce an answer – even if that answer is a wild guess. This overconfidence is dangerous in clinical decision-making, where acknowledging uncertainty is often important.
Lack of Clinical Context and Sensory Input: By design, today’s LLMs only process textual (and sometimes image) input. They have no direct access to the patient. They cannot perform a physical exam, observe patient behavior, or gauge subtle cues like patient anxiety, pain severity, or nuance in how a symptom is described. Much of diagnosis is sensory and contextual – e.g. the warmth of a joint, the sound of a heart murmur, the patient’s body language – factors entirely outside the text realm. As a result, an LLM’s diagnostic suggestions might be incomplete or off-target if the input data omits something a clinician would pick up in person. Even with structured data, LLMs don’t handle continuous monitoring or time-sequenced data well (like trends in vital signs) unless those are explicitly described to them. This limitation means LLMs work best as adjuncts that synthesize provided information, but they cannot replace the holistic assessment a physician or multi-disciplinary team provides. Furthermore, LLMs lack situational awareness – they don’t know the clinical setting or the patient’s personal history beyond what is given. A recommendation appropriate for a tertiary hospital might be unsafe in a rural clinic setting (e.g., suggesting an expensive test or a drug that isn’t available). A human doctor implicitly factors in such context, whereas an LLM would need to be explicitly told.
Biases and Health Disparities: The output of LLMs can reflect biases present in their training data. If the medical literature or case databases the LLM learned from have under-representation of certain groups, the AI’s diagnostic accuracy may be lower for those groups. For example, if data on diseases in Black patients are sparse or not well-represented, an LLM might be more likely to miss diagnoses or make errors for Black patients. This could inadvertently amplify healthcare disparities. Bias can also appear in how the AI communicates – for instance, one study found differences in how often an AI would recommend pain medication based on patient demographics, mirroring known biases in pain management pewresearch.org pewresearch.org. Ethically, deploying an LLM without addressing these biases could worsen outcomes for marginalized populations. Researchers have noted that LLMs need thorough evaluation for bias and fairness before clinical use nature.com nature.com. Importantly, unlike humans who can be trained in cultural competence, an AI might lack context to interpret cultural or linguistic nuances in patient symptoms, leading to misdiagnosis. Ongoing work on fine-tuning models with diverse datasets and adding bias mitigation strategies is attempting to tackle this, but it remains a limitation to be very mindful of.
Transparency and Explainability: LLMs operate as “black boxes” – they do not provide an easily understandable rationale for their outputs. While they can produce a reasoning-sounding explanation, this is essentially generated text, not a reliable peek into the true internal process. In medicine, this is problematic because clinicians and patients need to understand why a particular diagnosis or treatment is suggested. Trusting an inscrutable algorithm is difficult, especially when stakes are high. If an AI suggests a rare diagnosis, a doctor might be rightfully skeptical without a clear explanation or supporting evidence. This limitation is a barrier to adoption: many clinicians will simply not use a tool they don’t trust or can’t justify to themselves or the patient. Research is now focusing on “explainable AI” techniques, but for LLMs, genuine interpretability remains challenging. The lack of a guarantee for veracity in the AI’s explanations further complicates this; an LLM might generate a very convincing rationale that is actually incorrect or based on false assumptions, making it even more dangerous than a simple black-box output.
Regulatory and Knowledge Update Lag: LLMs like GPT-4 have a fixed cutoff in their training data (e.g., September 2021 for the original GPT-4 release). They do not automatically know about medical developments after that point unless specifically updated. This means an LLM might not be aware of the latest clinical trial evidence, newly emerged diseases, updated guidelines, or recently recalled unsafe medications. Physicians, on the other hand, continue to learn and adapt with new information (through Continuing Medical Education, etc.). An LLM could potentially give outdated medical advice if its knowledge is not refreshed. For example, an LLM might recommend an older therapy as first-line when guidelines have since changed to a newer drug. Mitigating this requires continuous fine-tuning on new data or connecting the LLM to current medical databases (which some systems are beginning to do). But until that is seamless, static knowledge is a limitation. There is also the regulatory aspect: in many jurisdictions, using AI for clinical decisions might require it to be treated as a medical device, which means demonstrating that it can be kept updated and safe. Ensuring an LLM’s knowledge doesn’t fall behind medicine’s rapid advancements is both a technical and regulatory hurdle.

In essence, LLMs in their current form cannot be trusted as autonomous decision-makers. They excel as intelligent text processors, but they lack true understanding, can produce incorrect information without warning, and don’t incorporate the full richness of clinical context. These limitations mean that any use of LLMs in diagnosis must be as assistive tools, with robust checks and balances. Rigorous validation on real patient cases, continuous monitoring for errors, and clear guidelines on how clinicians should use (and not use) the AI are necessary to avoid harm. Many of these issues also feed into the ethical and legal challenges discussed next, since who is responsible for an AI’s mistake, or how to maintain patient trust, are directly tied to these limitations.

Ethical, Legal, and Safety Implications

Integrating LLMs into patient care raises important ethical and legal questions that must be addressed to ensure safe and equitable use:

Patient Safety and Accountability: Foremost is the question: Who is responsible if an AI’s diagnostic suggestion leads to harm? Currently, there is no clear-cut answer. Under existing malpractice law, the consensus is that if a physician relies on an AI and a misdiagnosis occurs, the physician would most likely be held liable, not the AI or its manufacturer ncbi.nlm.nih.gov. This is because the physician has the duty of care and is expected to meet the standard of a reasonably competent practitioner ncbi.nlm.nih.gov. Deviating from standard practices by following a flawed AI recommendation could be seen as negligence on the physician’s part ncbi.nlm.nih.gov. This scenario puts clinicians in a cautious position: unless an AI is highly trustworthy and widely adopted as part of standard care, using it might increase legal risk. On the flip side, as AI tools become more common, there’s an emerging question – could not using an available AI tool one day be considered substandard care? (For example, if AIs are proven to catch things humans often miss.) The legal landscape is evolving, but one thing is clear: to avoid a chilling effect or unsafe use, roles and responsibilities need to be defined. Some have argued for shared liability models or safe harbor provisions when clinicians follow validated AI guidance. Until laws adapt, though, doctors will appropriately act as the final gatekeepers, and AI output must be treated as advice that the human validates. Ethically, this maintains that a patient’s well-being is firmly in the hands of accountable professionals rather than unaccountable algorithms.
Informed Consent and Transparency: Ethically, patients have a right to know if AI is being used in their care. If an LLM is aiding in diagnosis or making treatment recommendations, should the patient be informed and consent to this? Many ethicists say yes – transparency is key to trust. Patients might feel differently about a diagnosis if they know it came partly from a machine. In the UArizona survey, patient acceptance increased when physicians explained AI’s use and vouched for it healthsciences.arizona.edu healthsciences.arizona.edu. That suggests explaining AI involvement can be beneficial. However, disclosing AI use also requires explaining its status (for example: “This is a tool that helps me consider all possibilities, but it doesn’t replace my judgment”). Medical societies and regulators are beginning to craft guidelines on how to introduce AI to patients. In trials of AI-assisted care, patients are usually informed and often have the option to decline AI involvement. In routine practice, making this a norm could be challenging (imagine a busy ER where an AI triage system is running in the background – obtaining explicit consent might be impractical). One compromise is general consent: healthcare systems might include consent for AI-assisted services in their intake forms, with the ability for patients to opt-out if they have concerns. Overall, honesty about AI use respects patient autonomy and can preempt misunderstandings – e.g., a patient feeling deceived if they later learn a “robot” helped diagnose them.
Data Privacy and Security: LLMs integrated into clinical workflows will inevitably handle sensitive patient information. If using a cloud-based LLM (like the public ChatGPT), there are serious HIPAA (Health Insurance Portability and Accountability Act) concerns. Patient data might be transmitted to external servers and even inadvertently used to further train the AI (if proper data handling agreements aren’t in place). Early on, some doctors experimented with ChatGPT by inputting parts of patient charts – an obvious privacy risk if done without safeguards. Healthcare providers must ensure any LLM meets strict privacy standards: data encryption, on-premises processing or robust business associate agreements, and no unauthorized secondary use of patient data. The FDA and other regulators have also emphasized that AI systems should have audit trails – logs of what data went in and what decision came out – which ties into both safety and privacy (for forensic analysis if something goes wrong). Additionally, there is the risk of security: if an AI system interfaces with medical devices or EHR systems, could a cyber attacker manipulate it or extract information? The more connected and powerful these systems become, the more they might become targets for hacking. Ensuring cybersecurity for AI in healthcare is an emerging priority, requiring collaboration between tech companies and health IT departments.
Bias and Fairness: As mentioned, bias in AI decisions can lead to ethical issues, essentially automating inequality. Ethically, we must ensure LLMs do not propagate or worsen disparities. This means during development, using diverse training data and testing the AI on various subpopulations. If an AI is found to perform worse for, say, women or minorities, developers have a responsibility to address that, either by improving the model or at least warning users of the limitation. Some experts have called for an “AI Hippocratic Oath” where developers pledge to do no harm and to evaluate bias and safety rigorously pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. From a regulatory standpoint, there is increasing talk of requiring bias audits for clinical AI algorithms pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. For instance, an oversight committee or ethics board in a hospital could periodically review AI recommendations for signs of bias (e.g., does the AI tend to under-diagnose certain groups?). The JMIR Medical Informatics review emphasized implementing such audit and oversight mechanisms – including possibly an ethics committee specifically to monitor AI use and outcomes pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. Ensuring fairness is not just ethical but also important for patient trust: if communities perceive that an AI is biased against them, they will justifiably reject it.
Regulatory Compliance and Validation: The introduction of LLMs into healthcare blurs the line between a general-purpose tool and a medical device. Regulators like the U.S. FDA have started treating certain AI algorithms as medical devices, which means they require proof of safety and efficacy through clinical trials or studies before approval. If an LLM is marketed for diagnostic purposes, it might need regulatory clearance. This is challenging because LLMs are often constantly evolving (through updates) and they’re not deterministic. How do you validate something that can change its answer with a slight rephrase of the question? The FDA has issued guidance on “adaptive” AI algorithms, suggesting that significant changes would need new approval. For LLMs, one approach might be locking down a specific version for clinical use that’s been tested. Some companies (e.g., Google with Med-PaLM) are likely pursuing this: a fine-tuned model that undergoes a trial in a healthcare setting. Regulators will also look at the risk category of an AI: giving diagnostic advice is high risk (if wrong, a patient could die), so the bar for approval will be high. Legal frameworks in the EU (like the EU Medical Device Regulation and the upcoming AI Act) are similarly focusing on high-risk AI in medicine. In short, to ethically deploy LLMs in practice, they must undergo extensive validation – ideally peer-reviewed studies in clinical environments – and obtain regulatory clearance for the intended use.
Professional and Legal Ethics for Clinicians: Clinicians using AI must also navigate their own ethical duties. For example, if an AI provides a recommendation that the clinician disagrees with, there could be pressure (from administrators or protocol) to follow the AI for efficiency’s sake. But ethically, a clinician should use their judgment first. Guidelines are starting to appear from professional bodies (like the AMA or specialty societies) on how doctors should incorporate AI. Many emphasize that AI is a tool not a colleague – it does not have credentials or accountability, so the physician bears ultimate responsibility for decisions. This perspective is echoed in early position statements: the physician must verify AI outputs and should not use them outside their validated scope. There’s also an ethical obligation for education: tomorrow’s doctors need training in AI literacy – understanding its pitfalls, knowing how to question its output, and being aware of how it was developed. Educating clinicians is indeed highlighted as essential for adoption pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. An untrained physician might over-trust the AI or misuse it, whereas with proper training, they can harness it appropriately. We are likely to see formal inclusion of AI in medical curricula soon.
Safeguards and Monitoring: To responsibly integrate LLMs, healthcare systems will need new safety processes. This could include “shadow mode” testing, where the AI makes diagnoses in parallel with physicians for a period, to compare and ensure it’s not missing anything major. It also includes continuous monitoring post-deployment: tracking outcomes of AI-influenced decisions, reporting AI errors (similar to how medication errors are reported), and having a clear process to turn off or override the AI when it’s suspected to be wrong. Ethically, having a feedback loop is important – if an AI causes a near-miss or an adverse event, that information should be fed back to developers to improve the model (just like drug side effects are reported). There’s also discussion of implementing “restriction” on generative AIs in healthcare so they do not provide advice outside of certain bounds – for instance, programming it to say “seek immediate medical attention” rather than giving any advice in scenarios that sound life-threatening, to avoid patients relying on it at home in lieu of calling 911.

In summary, the ethical, legal, and safety landscape for LLMs in medicine is still taking shape, but it’s clear that careful governance is needed. We must ensure patient welfare, privacy, and trust remain at the center. That includes clarifying liability (so clinicians aren’t unfairly exposed, but patients have recourse if harmed), requiring transparency and patient consent for AI use, vigorously addressing bias, and holding these tools to high standards of validation. Many experts advocate a phased introduction: use LLMs in advisory roles with human oversight, study their impact, and gradually increase responsibility as confidence in their safety grows pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov. By proactively tackling these issues, we can hopefully harness LLMs’ benefits while minimizing risks.

Conclusion and Future Outlook

Large language models have made remarkable strides in emulating diagnostic reasoning and medical knowledge, to the point that they can pass exams and assist with complex cases. In controlled evaluations, they’ve shown near-human or even human-level performance on certain diagnostic tasks, and they offer clear advantages in breadth of knowledge and consistency. Patients and physicians are intrigued by these capabilities – envisioning faster diagnoses, more comprehensive differentials, and enhanced patient education – yet they are understandably cautious. Current evidence suggests that LLMs work best not as replacements for physicians, but as powerful adjuncts: an ever-ready second opinion or assistant that can support clinical decision-making.

To integrate LLMs successfully into healthcare, a few key steps will be critical in the coming years:

Robust Validation: Before LLMs are widely trusted, more large-scale studies in real clinical settings are needed. These should involve diverse patient populations and prospective comparisons of outcomes with and without AI assistance. Benchmark performance is encouraging, but real-world performance (with all the messy complexity of true cases) is the gold standard.
Improving the Technology: We can expect rapid improvements in LLM capabilities. Efforts like fine-tuning models on medical data (e.g., Med-PaLM, BioGPT) will continue to improve accuracy on clinical questions nature.com nature.com. Integration of multimodal inputs – combining text with imaging, waveform, or lab data – is an area of intense research, aiming to overcome the limitations we see in fields like radiology. Techniques to reduce hallucinations (such as retrieval-augmented generation, where the LLM consults a database of verified medical sources) are being developed to make outputs more factual nature.com. If successful, tomorrow’s medical LLMs might always provide cited sources or probabilities with their answers, increasing reliability. Companies are also exploring smaller, specialty-specific models that could be embedded in devices or EHR systems, increasing privacy and speed.
Workflow Integration and Training: Hospitals and clinics will need to figure out how to fit AI into their processes in a way that truly helps. This might involve new user interfaces – for instance, an AI assistant directly within the electronic medical record that doctors can consult with a click, rather than having to copy-paste into a separate app. It will also involve educating clinicians on when to trust vs. double-check the AI. Early adopters (like radiology departments using AI for scan analysis, or primary care clinics using symptom-checker bots before appointments) will provide lessons on the dos and don’ts. Over time, best practices will emerge, potentially even standard-of-care guidelines (e.g., “For difficult differential diagnoses, consider obtaining an AI-assisted diagnostic review, provided it has been validated for that context.”).
Regulation and Oversight: We will likely see clearer regulatory pathways for AI in the next couple of years. Regulatory agencies may certify certain models for specific uses – for example an “AI radiology assistant” cleared to read chest X-rays for screening, or a “triage LLM” cleared to advise patients on level of care. Professional organizations might also create accreditation or audit programs to ensure AI tools in use meet safety and efficacy benchmarks. Within healthcare institutions, AI oversight committees (as recommended by some scholars pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov) might become common, actively monitoring the performance of AI in practice and addressing any ethical issues that arise.
Patient and Public Engagement: Finally, an open dialogue with the public will be necessary to set appropriate expectations. Both over-hype and fear need to be managed. Patients should know that these systems, as helpful as they are, are not infallible “doctor bots” – but they also should hear about success stories where AI has improved care. Public trust can be built by involving patient advocacy groups in AI evaluation, being transparent about both successes and failures, and above all, keeping human caregivers at the center of care. As one survey indicated, people respond well to AI in healthcare when it’s paired with human support healthsciences.arizona.edu healthsciences.arizona.edu. Thus, the narrative should be about augmentation of doctors, not replacement.

In conclusion, the comparison of LLMs vs physicians in diagnosis is no longer science fiction – we have tangible data now. LLMs already rival average physicians on many test metrics and can even outperform in certain niche tasks or voluminous knowledge recall nature.com nature.com. Physicians, for their part, bring contextual understanding, real-world experience, and accountability that AI lacks. The foreseeable future of diagnosis will likely involve a synergy between the two: clinicians empowered by AI algorithms that provide rapid insights and recommendations, with the clinicians providing oversight, empathy, and the final judgment call. When properly combined, this could reduce diagnostic errors, improve efficiency, and personalize patient care in ways that neither alone could achieve. However, careful implementation is key – balancing innovation with caution to uphold the timeless medical principles of do no harm and patient-centered care. With continued research, collaboration between technologists and healthcare professionals, and thoughtful governance, LLMs could become one of the best intelligent assistants in healthcare, fulfilling their promise to enhance human expertise rather than replace it pmc.ncbi.nlm.nih.gov. The coming years will test our ability to integrate these tools wisely, but if successful, the outcome stands to benefit both clinicians (by reducing cognitive burden) and patients (by improving diagnostic accuracy and access to knowledge) in profound ways.

Sources:

Shan, G. et al. Comparing Diagnostic Accuracy of Clinical Professionals and Large Language Models: Systematic Review and Meta-Analysis. JMIR Med. Inform. (2025) – LLM diagnostic accuracy vs physicians pmc.ncbi.nlm.nih.gov pmc.ncbi.nlm.nih.gov.
Singhal, K. et al. Toward expert-level medical question answering with LLMs. Nature Med. 31, 943-950 (2025) – Med-PaLM 2 performance on MedQA, physician evaluations nature.com nature.com.
Yang, X. et al. Multiple LLMs vs Physicians in Diagnosing Challenging GI Cases. npj Digital Med. 8, 85 (2025) – Claude vs gastroenterologists, LLM broad coverage advantage nature.com nature.com.
Horiuchi, D. et al. ChatGPT’s diagnostic performance vs radiologists in musculoskeletal radiology. Eur. Radiol. 35(1):506-516 (2025) – GPT-4 vs GPT-4V vs radiologists results link.springer.com link.springer.com.
Pew Research Center – “60% of Americans Would Be Uncomfortable With Provider Relying on AI in Their Health Care.” (Feb 2023) – Public survey on AI in healthcare, trust and concerns pewresearch.org pewresearch.org.
Slepian, M. et al. “Diverse Patients’ Attitudes Towards AI in Diagnosis.” PLOS Digital Health (May 2023) – Survey experiment on patient preference for AI vs doctor healthsciences.arizona.edu healthsciences.arizona.edu.
Ayers, J. et al. “Comparing Physician and Chatbot Responses to Patient Questions.” JAMA Intern Med. 183(7):589-596 (2023) – ChatGPT answers rated higher in quality and empathy than physician answers pmc.ncbi.nlm.nih.gov.
Zhu, H. et al. “Evaluating ChatGPT’s performance across radiology subspecialties: a meta-analysis.” Acad. Radiol. 32(9):e369-e376 (2025) – ChatGPT ~61% average on radiology board Qs, variability by subspecialty pubmed.ncbi.nlm.nih.gov.
Rao, A. et al. “Assessing the performance of ChatGPT on the USMLE.” medRxiv preprint (2023) – ChatGPT (GPT-3.5) achieved passing score on USMLE Step questions, demonstrating potential in medical knowledge nature.com.
Liu, X. et al. “Large Language Model Influence on Diagnostic Reasoning: a Randomized Trial.” JAMA Network Open 6(7):e2327234 (2023) – Physicians with GPT-4 aid vs without, and GPT-4 alone diagnostic performance pmc.ncbi.nlm.nih.gov.
Jin, X. et al. “Patients’ trust in AI for diagnosis: interview and survey study.” PLOS Digital Health 2(5): e0000237 (2023) – Finding that physician endorsement increases patient acceptance of AI healthsciences.arizona.edu healthsciences.arizona.edu.
Nori, H. et al. “Capabilities of GPT-4 on Medical Challenge Problems.” arXiv:2303.13375 (2023) – GPT-4 chain-of-thought prompting improves medical problem-solving; notes on limitations like reasoning errors. nature.com nature.com.
Wang, F. et al. “Clinical Reasoning of a Generative AI Model versus Physicians.” Radiology 307(2):e230373 (2023) – GPT-4V and Gemini models on radiology “Diagnosis Please” cases vs radiologists (found vision-mode underperformed text mode) pmc.ncbi.nlm.nih.gov.
Gunes, B. et al. “AI vs Radiologists in Thoracic Imaging: Comparative Study.” J. Thorac. Imaging (2024) – Reported diagnostic accuracy: GPT-4 ~85% vs radiologists ~93% on thoracic cases pmc.ncbi.nlm.nih.gov.
Stanford HAI – “Understanding Liability Risk from Healthcare AI.” (2023) – Legal analysis: likely physicians hold liability for AI errors under current law ncbi.nlm.nih.gov.