Webinar – Validating AI-Assisted Dermatopathology

Watch on demand
×
Log In Get Started
Log In Get Started

The Hard Part of AI Starts When it Begins to Act (Part 2)

Wednesday November 4, 2026

Dr. Rajendra Singh photo

by Rajendra Singh MD
Professor of Pathology, University of Pennsylvania
Co-Founder, PathPresenter

In Part 1, I argued that the hard part of pathology AI starts after the model has an answer. A good model is not enough. We need the infrastructure to get the right case to the right model, bring the result back into the pathologist’s workflow, know exactly what happened and continue monitoring the system after it goes live. Foundation models and AI agents take that discussion a step further because they begin to change not just how we deploy AI, but what we expect AI to do.

The first distinction I think pathologists need to understand is that a foundation model is not a validated clinical application. Foundation models learn broad representations from very large collections of slides and increasingly from reports and other clinical information. Those representations can support many downstream uses, but the fact that a model has seen millions of slides does not mean that everything built on top of it is ready for patient care. Detecting cancer, predicting a biomarker and retrieving similar cases are different clinical uses with different risks and validation requirements. Even the embeddings generated by these models need to be versioned because changing the underlying encoder may change the representation of the slide. Foundation models may give us a reusable layer of intelligence, but every clinical application still has to earn its way into practice.

Agents go further still. Most AI we use today gives us an answer to a defined question. An agent can potentially decide what information it needs, retrieve it and choose which tool to use next. It might review the clinical history before I open a case, retrieve prior pathology, identify relevant slides, call an image-analysis model, check whether molecular studies are pending and draft part of the report for me to review. The important change is that AI is no longer simply producing an answer. It is beginning to coordinate a sequence of actions.

That raises a much more important question than whether an agent can technically do all of this: what should we allow it to do? Reading from the LIS is different from writing to it. Suggesting an immunostain is different from ordering it. Drafting a diagnosis is very different from signing the report. Once AI can use tools, those distinctions cannot simply live in a prompt. They need to become real permissions in the clinical infrastructure, with appropriate approval points and a record of every action.

This is why middleware becomes even more important in an agentic world. Consider a pigmented lesion from a patient who had a melanoma three years earlier. An agent limited to that accession might review the requisition and clinical note, retrieve the prior report and slides, and run only those image-analysis models the laboratory has validated for that specimen type. It could draft a clinical summary and selected parts of the report, all marked preliminary. It might suggest PRAME, but it cannot order it without my approval, alter the prior report or sign the current one. Every action, model version and output is recorded alongside what I ultimately accepted, changed or rejected. None of these individual components is particularly exotic. What most laboratories do not yet have is the infrastructure that brings them together safely.

Our approach to validation also has to evolve. With a conventional algorithm, we can ask whether the model produced the correct answer. With an agent, we have to evaluate the entire chain. Did it retrieve the correct patient information, choose the right model, use the correct version, interpret the output appropriately and stay within its permissions? Most importantly, did it know when to stop and ask the pathologist? The chain also includes us. A well-drafted report makes agreement easy, and agreement is not the same as review. I would therefore watch how often AI drafts are accepted unchanged and whether that pattern changes by agent version or pathologist. An unusually low override rate should make us ask whether we are carefully reviewing the AI or simply accepting it.

For agents, the progression I described in Part 1 matters more, not less: sandbox, silent trial, clinical use and continuous surveillance. In the sandbox, the system can be developed without affecting patient care. It should then run prospectively in a silent trial so we can see how the entire chain behaves in our own environment before anyone relies on it. Only then should it move into clinical use. Once it does, continuous surveillance becomes essential because its behavior now depends not just on one model, but on the data, tools, permissions and sequence of actions around it.

Pathology is particularly well positioned for this next phase because integrating information is what we have always done. We bring together morphology, clinical history, prior pathology, immunostains, molecular findings and increasingly radiology and other data into one interpretation. Foundation models may give us a reusable layer of intelligence across that information, while agents may help find, organize and coordinate it. I can see real value in that, but none of it should remove clinical judgment from the pathologist. If anything, these systems make it more important that we define where our judgment enters the workflow, which decisions remain ours and how those decisions are recorded.

The next phase of pathology AI will not be defined simply by bigger models. It will be defined by systems in which models, agents and pathologists can work together safely. Foundation models can provide reusable intelligence. Agents can use that intelligence to coordinate tasks and interact with clinical tools. Middleware defines the boundaries, permissions and audit trail around those actions. The pathologist still determines what ultimately enters the medical record.

About the Author

Dr. Rajendra Singh is a Professor of Pathology at the University of Pennsylvania and co-founder of PathPresenter. He serves as a member of the Digital and Computational Pathology Committee of the CAP, Editorial Board of the WHO for Classification of tumors, 5th Edition and the Board of Digital Pathology Association.