Webinar – Validating AI-Assisted Dermatopathology

Watch on demand
×
Log In Get Started
Log In Get Started

The Hard Part of AI Starts After the Model Has an Answer

Friday October 30, 2026

Dr. Rajendra Singh photo

by Rajendra Singh MD
Professor of Pathology, University of Pennsylvania
Co-Founder, PathPresenter

The Hard Part of Pathology AI Starts After the Model Has an Answer

Most of the conversation about AI in pathology is still about the models. Which algorithm detects cancer best, which foundation model was trained on the most slides, and which biomarker or mutation can now be predicted from an H&E. These are remarkable advances, and I follow them as closely as anyone. But after working in digital pathology for many years, I am increasingly convinced that the question that will determine whether AI really changes our practice is a different one. What happens after the model has an answer?

Imagine a prostate biopsy with twelve cores scanned early in the morning. An AI model may have analyzed every core before I even open the case. That sounds like exactly what we have been working toward. But where is the result? If it sits in another application behind another login, if I have to manually transfer a measurement into the report, or if my disagreement with the AI is never captured, then we have solved a very difficult computational problem without finishing the clinical workflow. This gap between the algorithm and the pathologist is where AI middleware becomes important.

Middleware sounds like an IT term, but much of what it does is actually clinical. It is the layer connecting our LIS, image management system, viewer and report with the AI models we want to use. The laboratory already knows that a case is a prostate biopsy, which slides are H&E and which scanner produced the image. That information should determine which AI runs, whether the slide is appropriate for that model and where the result should appear. In pathology, the identity chain runs from patient to accession to specimen, block, slide, image and now AI result. A correct answer connected to the wrong slide is not a small error. It is a clinical failure.

The same applies to what comes back from the model. One system may return a heatmap, another polygons, another a measurement. The pathologist should not have to understand how every vendor packages its output. Middleware should put each result where it belongs. A heatmap belongs on the slide. A validated tumor measurement or Breslow thickness may belong in a synoptic report, but it should remain preliminary until I review it. And when a model cannot analyze a slide, “no result” has to be just as obvious as a positive result. “No cancer detected” and “could not analyze” can never be allowed to mean the same thing. We would not accept that ambiguity from an immunostain or a molecular assay, and we should not accept it from software.

Perhaps the concepts pathologists most need to understand are what happens before and after an AI model becomes clinical. I find it useful to separate three stages: validation, verification and continuous surveillance. The developer validates the model for a defined intended use. When we bring it into our laboratory, we verify that it works in our environment, with our scanners, stains, specimens, interfaces and workflow. After it goes live, somebody has to keep watching it. Scanners change, staining changes, software changes and models themselves change. A model that performed well at validation and local verification is not automatically safe forever.

There is a similar progression toward clinical use. In the sandbox, the model is still research and nothing it produces should influence patient care. The next step should often be a silent trial, where the model is locked and run prospectively on live cases while its output remains hidden from the pathologist. That is where we learn whether the right cases are selected, whether the right images reach the model, how often inputs fail and whether performance holds up on our own patients. Only after that should we move into clinical use, where the result may influence diagnosis or reporting. Once that happens, we are no longer simply studying a model; we are operating a clinical system that needs continuous oversight.

Commercial AI and internally developed AI may start very differently. One may arrive as a regulated product with a defined intended use, while the other may begin as research code that changes frequently. The validation, regulatory and maintenance responsibilities are different, but once either begins influencing patient care, I believe they should enter through the same clinical infrastructure, with the same identity checks, provenance, audit trail and surveillance. Otherwise we end up with two systems, two audit trails and two ways to be wrong.

The return path is just as important. Most AI today is one-way: the model produces an answer and the pathologist receives it. But if AI measures a tumor at 4.2 mm and I change it to 3.8 mm, that is valuable information. If I repeatedly reject the same type of AI finding, somebody monitoring that model should know. The pathologist should not simply be the endpoint of AI; we should be part of the feedback loop. That is how continuous surveillance becomes more than checking whether the software is still running.

Foundation models and AI agents will raise the stakes on all of this, and they deserve their own discussion in Part 2. For now, pathologists do not need to become software engineers, but we do need to understand these concepts well enough to own the clinical decisions inside them. Accuracy should be the beginning of the discussion, not the end. We need to know how a model was validated, how we will verify it locally, what happens when it cannot give an answer, where its output will appear, how we will know when it changes and how we will monitor it after deployment.

Digital pathology turned glass into data. AI middleware has to turn algorithms into reliable clinical practice. Every arrow should run both ways. Build the return path first.

About the Author

Dr. Rajendra Singh is a Professor of Pathology at the University of Pennsylvania and co-founder of PathPresenter. He serves as a member of the Digital and Computational Pathology Committee of the CAP, Editorial Board of the WHO for Classification of tumors, 5th Edition and the Board of Digital Pathology Association.