Successful deployment of an Artificial Intelligence solution for primary diagnosis of prostate biopsies in clinical practice
DOI: https://doi.org/10.47184/tp.2023.01.03In order to evaluate the feasibility of artificial intelligence within the setting of prostate histopathology, 3975 slides from 860 patients were digitally scanned and supplied to the IBEX-Artificial Intelligence (AI) system before evaluation by histopathology consultants with recommendations from the AI. Data comparing reporting with and without AI assistance were analysed along with accuracy of diagnosis for the AI. Request rates for additional immunohistochemistry from consultants in cases of diagnostic uncertainty dropped from 8.7 % to 4.5 %. Qualitative reporting confidence increased with AI assistance and valued the highlighting of the most suspicious areas within a biopsy. Positive and negative predictive values for the AI were 0.994 and 0.995 when using the consultants’ diagnosis as the true value. AI shows significant potential as an assistant for histopathologists in the field of cancer diagnosis.
Keywords: Artificial intelligence, IBEX, AI, Prostate Biopsy, assistance
Introduction
The screening of prostate biopsies in UK histopathology departments commonly involves examining up to 10–12 cores per case [1]. Biopsy rates have increased compared to numbers of qualified pathologists; overburdened services may have an increased potential for human error identifying small foci of cancers [2, 3]. A financially viable and dependable aid for pathologists was sought to rebalance this supply-demand mismatch. The study primarily aimed to validate the use of artificial intelligence (AI) as an assistance tool for the diagnosis of prostate biopsies in clinical practice. Additionally, AI was viewed as a potential for increasing accuracy and reliability of pathologists in this field.
Challenges of AI involve potentially introducing human bias from the resources they learn from and public concern over future obsolescence of humans within diagnostics. Concerns regarding reliability, failsafe measures should AI fail, data protection and workflow issues required addressing before progress towards AI dependent diagnosis can begin.
Method
The study involved digitally scanning 3975 slides from 860 patients biopsied in North Wales. Digital images were reported by the AI and by a consultant histopathologist. Slides were classified based on the AI diagnosis as either likely-benign, uncertain, or likely-malignant prior to examination by pathologists in Betsi Cadwaladr University Health Board. An AI-Model (IBEX, Tel Aviv, Israel) would then highlight points of interest to the pathologist alongside the proposed AI diagnosis. Pathologists would then formulate their diagnosis based on their findings and categorise the case as either benign, uncertain, or malignant.
Parallel to this, numbers regarding the Glan Clwyd histopathology department’s workload; namely caseload, specimens per case, immunohistochemistry requests and final diagnoses comparing cases from 2019 to cases 2022.
Results
Procedures in 2022 were deemed comparable to those carried out in 2019 with 81:19 being biopsy and template biopsy respectively in 2022 compared to a ratio of 86:14 in 2019. Numbers of prostate biopsy cases decreased in this period from 708 cases to 467 cases with number of specimens also decreasing from 2184 to 1632 (Figure 1).
Cases in 2022 had a slightly higher average number of specimens compared to 2019 from 3.08 specimens per case to 3.49.
From 2019 to 2022 the proportion of cancer diagnoses increased from 55 % of cases to 68 % – with a corresponding decrease in the percentage of benign specimens from 43 % to 29 %. Percentages of the intermediate; atypical small acinar proliferation diagnoses remained roughly similar throughout the study (Figure 2).
Figure 1 & 2: Variation of workload and diagnosis of prostate biopsies from 2019 to 2022 in Glan Clwyd pathology department.
Cases requiring additional immunohistochemistry due to diagnostic uncertainty decreased from 8.7 % of total cases to 4.5 % of total cases with the use of AI. This reduction was most prominent within the cases where the end diagnosis was cancer; from 10.3 % of cancer cases requiring immunohistochemistry in 2019 to only 4.2 % of cancer cases requiring immunohistochemistry (Figure 3).
Figure 3: variation of cases requiring immunohistochemistry from 2019 to 2022 in Glan Clwyd pathology department.
Qualitatively; all pathologists involved in the study reported feeling more confident overall making a diagnosis with AI compared to without. Consultants highlighted the benefit of highlighting points of interest in otherwise banal looking areas that carry the risk of being missed. The presence of an AI assistant was used as an aide de memoir to assist with standardisation of reporting when commenting on Gleason score, perineural invasion, high grade prostatic intraepithelial neoplasia et cetera.
Of the 2201 slides that were double reported by AI and a pathologist, the AI was able to definitively diagnose cancer in 808 of the 812 slides ultimately diagnosed as positive for prostate cancer; a sensitivity of 99.5 % and rule out cancer as a diagnosis in 657 of the 660 slides containing only benign tissue, a specificity of 99.5 %. Of the 3 cases incorrectly flagged as benign, 1 contained a single cancerous gland and 2 were out of focus when scanned. Of the cases incorrectly flagged as malignant, 2 were deemed ungradable by the pathologists, 1 was atypical small acinar proliferation and one interpreted Cowper glands as malignant in nature. The unaccounted 729 (33 %) slides were deemed suspicious of, but not definitive for, malignancy. Within this substratum of datapoints, 600 (82.3 %) were benign, 105 (14.4 %) were malignant, 15 (2 %) were atypical small acinar proliferation and 9 (1.2 %) were deemed ungradable by pathologists. Overall, the positive and negative predictive values for AI within prostate biopsies was 0.994 and 0.995 respectively (Table 1).
Table. 1: AI vs actual diagnosis within prostate biopsies in Glan Clwyd pathology department.
AI-Diagnosis | Actual Diagnosis | Count |
|---|---|---|
Likely Benign | Benign | 657 |
Malignant | 3 | |
Likely Malignant | Benign | 1 |
Malignant | 808 | |
Atypical small acinar proliferation | 1 | |
Ungradable | 2 | |
Suspicious | Benign | 600 |
Malignant | 105 | |
Atypical small acinar proliferation | 15 | |
Ungradable | 9 |
Conclusion
For the diagnosis of prostatic carcinoma and its differentials, the gold standard remains diagnosis by a histopathology consultant trained in uropathology. However, the capability of said histopathologists have been enhanced by the assistance of artificial intelligence. The use artificial intelligence to diagnose prostate cancer by itself is not recommended due to the low yet significant numbers of misdiagnoses shown within this study. Importantly, the use of AI within an assistant’s role has been shown to increase the accuracy of pathologists to identify small tumour foci. Even in cases where AI misdiagnoses; or at least suggests a misdiagnosis as more likely; the suspicious sites were always highlighted to the reporting pathologist to make their own conclusion. AI still falls into pitfalls occasionally, such as misidentifying Cowper's glands as malignant cells due to the variety (and rarity) of histology specimens and ultimately will rely on the diagnoses given by the pathologists they learn from much in the same way the pathologists themselves may inherit bias from their educators. AI has been shown as a potential solution to the problem of ever-increasing case to pathologist ratios experienced, increasing reporting accuracy and confidence as well as decreasing associated costs due to diagnostic uncertainty.
Conflict of interest statement
None of the authors has any financial conflict of interest with respect to the content of this article or is a member of an organisation with opinions on the issues dealt with in the manuscript. No external funding.
Authors
