A growing number of AIs cleared for clinical use is finally available: The AI-assisted Pathologist

The past four years have finally brought the first Computational Pathology, AI-based (Artificial Intelligence) products to the market that are cleared for usage in a routine diagnostic setting. This article is likely to age fast, but at the time of writing nearly 40 products have a CE mark, either according to the old European in vitro diagnostic device directive No. 98/79/EC (CE-IVD) or the new regulation No. 2017/746 (CE-IVDR), and a first AI was approved by the American Food and Drug Administration (FDA). This is compared to over 500 FDA-cleared AIs listed in the “radiology” category. While the usefulness of AI for histopathology in different scenarios has long been demonstrated in the literature, the transfer into the field has remained an obstacle. This is changing now.

In this article, we will inventory the various AI products available today, and describe and categorize them. We will not, however, comment on their quality or make recommendations.

A prerequisite for AI-assisted pathology, sometimes referred to as Computational Pathology, is the digitization of the pathology laboratory, known as Digital Pathology. All of the image analysis AIs addressed here, operate on digitised whole-slide-images (WSIs). A wide range of scanners is now available and many institutes own at least one scanner. In contrast, the fraction of labs that are fully digital is still small, but growing, and varies from country to country, with the Nordic countries, the United Kingdom and the Netherlands well ahead. For a pathologist, the ability to employ AI is just one of the advantages of going digital, the other main drivers being the possibility to work remotely – “home office” – and to inquire a second opinion more easily and rapidly.

While FDA-approved AI products have long been listed in a curated online registry [1], this has not been the case for CE-IVD marked products. AIs certified according to the new CE-IVDR regulations are now also publicly listed in the European Database on Medical Devices “EUDAMED” [2]. A more complete list of both CE-IVD and CE-IVDR-certified as well as research-use-only (RUO) pathology AIs is available for download from the Fraunhofer IIS’s research blog [3].

Spoiler ahead on the issue of replacingpathologist by AI: A first question that may arise pertains to the impending replacement of human pathologists by AI systems. It is important to clarify that such a scenario is unlikely to happen in the foreseeable future. Rather than framing the discourse as a dichotomy between AI and pathologists, it is more apt to delineate it as a comparison between AI-assisted pathologists and conventional, or "100 % manual" pathologists. AIs for histopathology, and especially those approved for diagnostic use, typically perform a very narrow task. This could be the automated calculation of a particular score for a particular stain, a particular organ and usually even limited to scans created with a particular scanner model. By doing this task, they are supposed to help the pathologist by increasing the speed and reliability of such defined measurements. Given the broad bandwidth of tasks that pathologists cover in their day-to-day routine, a large number of different AIs would be required to assist in every task. Yet, among the few clinical AIs available so far, multiple products from different vendors address the same use case. Not surprisingly, analyses conducted very frequently in the clinical routine and suited well for automation were targeted first by the various vendors. The following sections list the most common examples, although other AI products are available for use cases incl. colorectal cancer grading, melanoma detection, mitotic figure counting, and more.

Breast cancer IHC scoring

A good example that underlines the point that many vendors have come to the same conclusion which use cases should be automated first is the widespread availability of breast cancer AI assistance systems for various Immunohistochemistry (IHC) assays. Breast cancer is the most diagnosed cancer in the EU and therefore diagnosis is on high demand in any pathology lab. The list of vendors that have released clinical AIs that automatically conduct a Ki67, ER, PR and/or HER2 scoring for breast cancer includes (in alphabetical order) aetherAi, Aiforia, Indica Labs, Mindpeak, Visiopharm, VMscope and WSK Medical. A distinguishing feature among these solutions is whether the pathologist manually selects the field of view or region of interest, where cells get analyzed, or whether the system automatically analyzes the entire slide without any user interaction. The latter is a requirement for a digital workflow, where scans are automatically analyzed after scanning, so that when the pathologist opens a case, all analysis results are already and immediately available.

Prostate cancer

A second good example for a use case where already multiple vendors offer solutions is Gleason grading for prostate cancer. Similar to breast cancer in women, prostate cancer has the highest incidence among men. CE-marked solutions are available by (in alphabetical order) Aiforia, Deep Bio, IBEX, Indica Labs, Paige.AI. Furthermore, Optra-Scan, Qritive and Virasoft offer RUO solutions. In particular, Paige.AI’s cancer detection AI is to date the only FDA-approved image analysis AI for histopathology. Their CE-approved variant is also capable of grading while the FDA-approved variant is limited to cancer detection. An additional CE-marked component checks for the presence of tumor cells in proximity of nerve fibers, indicating risk of perineural invasion.

Metastasis detection in lymph nodes

Another very common and tedious diagnostic task is to screen lymph nodes for the presence of metastases. Depending on the location of the primary tumor a number of lymph nodes are surgically removed. All nodes, typically one section per node, must be examined by the pathologist to determine the number of infiltrated nodes. In several specific scenarios such as melanoma or cervical cancer the lymph nodes are additionally extensively screened for micro metastases. An AI can pre-analyze the scans to make it easier to sort the worklist by descending likelihood of metastasis presence. When a slide is opened, typically a tumor probability heatmap is overlaid that directs the pathologist's eye to the region that was found to likely contain tumor cells. If the pathologist agrees, they can stop screening this section and save time. Alternatively, the AI can be utilized as a safety net. The pathologist could first screen the scan without enabling the overlay. If they found no traces of tumor cells, they can then enable the heatmap. In case the AI did find a metastatic region and the pathologist confirms this, a false negative assessment was prevented and the patient benefits from knowing that the metastasis was not overlooked.

Vendors that offer metastasis detection include (in alphabetical order) aetherAI (gastric), Mindpeak (breast), Primaa (Breast), Visiopharm (breast and CRC), and WSK Medical (entity not stated).

Non-small cell lung cancer

PD-L1 is a well-known predictive biomarker. Patients with a high PD-L1 score are well-suited for therapy with immune response modulators. However, there are different antibodies available and there is a plethora of scoring systems. In addition to the laborious process of generating such score values, the pathologist also has to constantly learn and apply the various indications, scores, etc. In summary, this is again a good use case for AI-based evaluation tools.

On one hand, these are highly laborious approaches, and on the other hand, there is a high clinical need (with approximately 1 in 5 cancer-related deaths caused by lung cancer, making it the #1 cause in the EU). It is good news that AI assistance systems are now available for this condition as well. Aiforia, Mindpeak, Indica Labs, and Visiopharm (in alphabetical order) have all released automated PD-L1 scoring solutions for non-small cell lung cancer (NSCLC).

Automation and 2^nd look

What the aforementioned solutions for breast, prostate and lung cancer as well as metastasis detection have in common is that they essentially automate a task that is currently conducted manually. This is already very valuable for several reasons. For one, most countries have a growing shortage of pathologists due to a shrinking workforce and a growing workload. The workload grows, among other factors, due to an aging population – as age brings with it a higher risk of developing cancer – and also due to medical advances. Personalized medicine, after all, delivers improved treatment outcomes for patients, but requires more precise diagnostic testing to select the appropriate therapy among many options. Digitization and utilization of AI will hopefully mitigate this growing clash between supply and demand and increase the available pathology workforce’s efficiency. Another promise of automated scoring is to guarantee a constant and high level of quality. When presented with the same image, an AI – and we exclusively talk about “frozen” AI that do not continue to learn once deployed – will come to the same conclusion, regardless of whether it is Monday morning of Friday afternoon, before or after the coffee break, at the beginning of their career or close to retirement. From a marketing perspective, this category of AI products has another advantage: it will likely have a higher user acceptance rate as pathologists can easily confirm the AI predictions or recognize errors. Even if the internal reasoning process itself may not be explainable, the predicted result is. This is particularly psychologically relevant in relation to the above-mentioned fear of being replaced by AI.

AI 2.0

By now it is clear that another category of histopathology AI models are those that do not only mimic established scores, but that actually shed new light. Let us call them AI 2.0. It is clear that today's scores have to be simplified to a point that a pathologist can reliably execute the algorithm according to the guidelines, in a limited amount of time, under pressure, and with a number of interrupting phone or Zoom calls. There is no point, however, in limiting an AI-computed score. The utilization of global assessments instead of hotspot-based scoring may increase the prognostic accuracy. Going one step further, why only train an AI to compute an already established score? After all, today's scores are essentially proxies for an actual endpoint. These scores or features are a biomarker for e.g. therapy recommendation or risk prognosis. Instead, AI models can be trained to predict these endpoints directly without a detour over defining histological features and scores. For example, the above-mentioned example of PD-L1 with its various competing scores shows that this score-based approach has its limitations. In fact, multiple vendors have already published AI products that fall into this category as the next two paragraphs will show.

Risk prediction

Karolinska spin-off Stratipath has developed an AI for breast cancer risk profiling that refines the approximately 50% of cases found to be in Nottingham Histological Grade “intermediate” into two new low- and high-risk subgroups. This is done merely by image-analysis of a H&E scan.

French biotech AI company Owkin has developed its RlapsRisk BC AI that predicts for ER+/HER2- breast cancer patients the risk of relapse. To give a third example, Norwegian DoMore Diagnostics offers their CE-marked AI Histotype Px that predicts, based on image analysis of a H&E-stained scan of a colorectal cancer resection, a good prognosis vs. poor prognosis.

Morphological pre-screening

Instead of predicting risk or prognosis as in the previous section’s examples, it is also now possible to predict protein expression levels or genetic alterations directly from an H&E scan. Owkin’s MSIntuit CRC AI is a CE-marked pre-screening tool for microsatellite instability (MSI). Notably, it was not tuned to predict MSIness, but instead it can identify microsatellite stable (MSS) cases with high confidence. The time otherwise spent on waiting for the result of the MSI test, that is predicted to likely come back negative, can be saved and instead alternative treatment options can be explored right away. US/Israel based AI company Imagene has developed a similar AI for NSCLC, currently for research use only, that predicts multiple genetic alterations directly from H&E, including EGFR, ALK, BRAF, KRAS and others. Paige.AI‘s CE-marked HER2Complete AI is able to predict HER2 expression from H&E-stained breast cancer sections. This AI is reported to detect HER2 expressions even for IHC HER2-negative or low cases. Similarly, UK-based Panakeia’s PANProfiler Breast predicts ER, PR, and HER2 expression from H&E and is also CE-marked.

Outlook: Foundation Models decrease time-to-model

While the pace at which new CE-marked AI products are published has steadily grown since the first such product was released ca. four years ago, this trend is expected to increase even more. Multiple vendors have recently published scientific articles about their so-called foundation models. These models were trained in a self-supervised fashion, i.e. without training annotations, on a very large and very broad pan-cancer dataset, comprising multiple thousands of scans. As a result, the numerical features extracted by these models from small image patches are highly descriptive of the contained morphology and broadly applicable. The quality of foundation models is assessed by performing and evaluating a number of different downstream prediction tasks such as detecting cancer and other tissues, identifying genetic alterations, predicting risk, etc. Each such task requires training of a prediction head - an additional AI model - that operates on top of the foundation model. Since the extracted features are so well formed, training these prediction heads is easier and faster compared to developing and training an AI from scratch. As a result, vendors with access to a good foundation model will shorten their “time-to-model”.

Integrating AIs into the Digital Pathology Workflow

In conclusion, it appears that a critical number of AI approved for clinical use is now available for routine diagnostics. And it can be deduced that this is number will continue to grow further.

How can this plethora of AI products be integrated into a single pathology workflow? A pathologist can hardly be expected to switch between programs and user interfaces for every AI they want to use. They should be able to select the WSI viewer independently from the AI models. What will be required is that vendors of image management systems (IMS) integrate third party AI models seamlessly as plugins into their systems, enabling pathologists to select and purchase AIs in an app store not unlike downloading an app for their cell phone.