Contributors: Shipra Gupta, Tanusree Chaudhuri, Praveen Korepu, Nag Himakund Mallampalli, Lingaraju Halasiddappa, Uzma Saeed, Jitesh Pillai, Puneet Saxena, Chandra Sekhar Pedamallu
The integration of Artificial Intelligence (AI) and Machine Learning (ML) into medical image analysis has transformed the landscape of cancer diagnosis and treatment. AI and ML technologies enhance the accuracy and efficiency of cancer detection, diagnosis, and treatment planning. This executive summary highlights the transformative potential of these technologies in cancer care, focusing on their applications, benefits, and future prospects.
Figure: Key applications, benefits, and future prospects of AI/ML in Image analysis for Cancer treatment
AI and ML are pivotal in transforming cancer treatment through enhanced image analysis. By improving early detection, characterizing tumours, and personalizing treatment, these technologies offer significant benefits in the fight against cancer. As advancements continue, the integration of AI/ML in oncology promises to enhance patient outcomes further and streamline cancer care processes.
Introduction
Cancer causes tens of millions of deaths every year, is costly to treat, and can be difficult or impossible to cure. In the fight against cancer, early detection is essential to treating cancer effectively. Conventionally, invasive procedures like biopsies have been used in cancer diagnosis. Over the past decades, scientists have applied different methods/tools to detect cancers in early stage before they cause symptoms. Medical image analysis is one such tool that has revolutionized the field of cancer diagnosis, treatment, and monitoring of treatment outcomes [1]. With the advancement of technology, particularly in the field of artificial intelligence (AI) and machine learning (ML), image analysis has emerged as a game-changer in healthcare. The application of AI/ML has persuaded a prominent transformation to enhance high levels of accuracy in detecting and diagnosing a variety of cancers.
Medical image analysis involves four steps: (1) Image preprocessing is used to remove background artifacts from images obtained from various sources and enhance the quality of the images for further
processing. (2) Segmentation involves the process of isolating the regions of interest (ROIs), such as tumor mass, microcalcification, and non-cancerous tissue. (3) Feature extraction is the process of extracting precise details from the ROIs including tumor size, texture, location, and border to aid in their recognition. (4) Pattern recognition or classification, involves evaluation of each ROI for scoring of the probability value for categorizing true positive (TP), false positive (FP), false negative (FN) and true negative (TN) using ML algorithms such as Nearest-neighbour rule, Minimum distance classifier, Artificial neural network (ANN), Radial basis function network (RBF), Support vector machine (SVM), Principal component analysis (PCA) etc [2].
The emergence of AI/ML algorithms have demonstrated the capability to automatically identify and categorize anomalies in various medical images, including X-rays, magnetic resonance imaging (MRI) scans, computed tomography (CT) scans, positron emission tomography (PET) and ultrasound images [3].
Earlier, ML algorithms used low-level methods, such as thresholding, region growing, and edge tracing to process medical images. The growth in size and scope of medical imaging data has fuelled the evolution of deep learning (DL), a branch of ML technique in medical image analysis that employs artificial neural networks comprising multiple layers to grasp and discriminate intricate patterns from extensive datasets [4]. DL techniques, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Long Short-term Memory (LSTM) models, and hybrid models, can be trained using vast datasets with annotated medical images to harness meaningful insights to aid in diagnosis, treatment planning, and therapeutic interventions. Advances in DL technologies already have achieved great success in medical image analysis with high accuracy, efficiency, stability, and scalability [5].
In this article, we explore the current imaging modalities as well as technologies for medical image analysis. We also provide clinical applications with respect to different types of cancers and discuss the existing challenges in the field and offer possible solutions and future directions.
Current Perspectives on AI/ML Applications in Cancer Image Analysis
One of the most impactful applications of AI and machine learning (ML) in cancer detection is in the analysis of mammograms for breast cancer screening. Traditionally, the interpretation of mammograms relies heavily on the expertise of radiologists, which can be susceptible to human error and variability. In contrast, AI and ML algorithms have demonstrated impressive capabilities in detecting subtle patterns and anomalies that may indicate early-stage cancer. Research has shown that AI-driven systems can achieve accuracy levels that match or even exceed those of human radiologists, thereby reducing both false positives and negatives and enhancing diagnostic confidence.
In the context of lung cancer detection, AI and ML models have been developed to analyse CT scans, enabling the identification of nodules and the assessment of their malignancy risk. These advanced models significantly improve the sensitivity and specificity of lung cancer screenings, facilitating early intervention and potentially leading to better patient outcomes. Additionally, AI and ML techniques are increasingly utilized to analyse histopathological images, allowing for the precise classification of cancer subtypes, which is essential for determining appropriate treatment strategies.
AI and ML have also advanced the field of personalized medicine and predicting survival outcomes. By integrating image analysis with genomic data, these algorithms can predict individual responses to specific treatments, allowing for tailored therapy plans that consider each patient’s unique genetic profile. This personalized approach not only enhances treatment efficacy but also helps minimize adverse effects, thereby improving the overall quality of life for cancer patients. The association clinical
& genetic information also helps in estimating factors such as survival rates and the likelihood of cancer recurrence [6, 7]. Recent advancements have enabled models to extract a wide array of quantitative features from histopathological images, generating high-dimensional data for predictive modelling. These models can also integrate diverse data types, including genomic, proteomic, and clinical information from multiple imaging modalities (e.g., PET-CT), to enhance diagnostic accuracy [8].
Furthermore, AI and ML-powered image analysis tools play a crucial role in monitoring treatment responses and detecting recurrences. Advanced algorithms can compare sequential imaging studies to identify subtle changes that may indicate disease progression or response to therapy. This capability empowers oncologists to make informed decisions regarding treatment adjustments, ensuring timely and effective interventions.
Figure 1. Role of AI-ML in image analysis for cancer detection.
Artificial Intelligence (AI) and Machine Learning (ML) algorithms utilize a structured workflow to learn from high-quality data, enabling effective analysis, annotation, and processing of various medical images, including MRI, CT scans, and histopathological images [9]. The process begins with techniques such as noise reduction, contrast enhancement, and normalization to improve image quality. Next, ML algorithms segment these images to isolate regions of interest, such as tumours or specific tissue types, for detailed analysis. This segmentation can be performed using traditional methods like thresholding or advanced deep learning techniques, such as U-Net [10, 11].
After segmentation, the workflow continues with feature extraction, which can involve either handcrafted techniques or deep learning methods like Convolutional Neural Networks (CNNs). Once features are extracted, models are trained to detect tumours, employing CNNs and other deep learning frameworks to identify the presence of tumours in medical images, such as lung nodules in CT scans or breast cancer in mammograms. The subsequent phase involves tumor classification, where ML models categorize tumours as benign or malignant and further classify them into specific cancer types based on the patterns observed in the image data [7].
Key algorithms used in this workflow include Convolutional Neural Networks (CNNs) for image classification, detection, and segmentation tasks; Recurrent Neural Networks (RNNs) for analysing sequential data, such as in longitudinal imaging studies; Autoencoders for image denoising, reconstruction, and feature extraction; Support Vector Machines (SVMs) for classifying high-dimensional radiomic features; and Random Forests and Decision Trees for feature selection and classification in radiomics and pathomics. The accompanying figure above illustrates this comprehensive workflow.
Available tools/packages and details on AI/ML algorithms used in Image analysis
Here are some tools and packages that cater to a wide range of image analysis tasks ranging from basic image manipulation to sophisticated deep learning-based image recognition and segmentation. These tools are essential for general-purpose image processing, developing deep learning frameworks, and executing specialized tasks with specific libraries. Table 1 talks about some of the prominent tools that are widely used in medical imaging applications (specifically cancerous tissues), including radiology and pathology, for segmenting medical images and instances, real-time object detection in images and videos, and key point detection, among other purposes.
Table 1. Medical image analysis tools used in Cancer detection and treatment
AI/ML algorithms used in Image analysis
AI/ML algorithms have emerged as a promising solution for enhancing healthcare accuracy and patient outcomes. These algorithms play a pivotal role in oncology by predicting and automating various aspects, including risk assessment, early diagnosis (such as tumor detection), classification and identification of cancer types, segmentation of cancerous regions, prognosis estimation, and treatment selection based on comprehensive data analysis. Studies have shown that AI/ML algorithms often surpass clinicians in predicting cancer with higher accuracy [22]. These technologies hold significant potential to improve diagnosis, prognosis, and the overall quality of life for cancer patients. Below are some of the key algorithms used in this field.
Figure 2. AI/ML algorithms used in medical image analysis
Several applications have been developed using AI/ML algorithms, demonstrating their effectiveness in cancer detection and diagnosis. Notably, Google’s DeepMind algorithm-based AI system has been reported to detect breast cancer with high accuracy (The University of Pittsburgh has developed a machine learning technique with remarkable precision in diagnosing prostate cancer, achieving a specificity and sensitivity of 98%. Additionally, ViT-Patch, validated on a publicly available dataset, has proven effective in both detecting malignant tumours and localizing them [23]. These advancements highlight the significant potential of AI/ML technologies in improving cancer diagnosis and treatment.
Dependencies and path forward
The development of AI and ML tools in the clinical domain should address vital clinical challenges and require a deep understanding of the clinical context. The increasing amount of biomedical data from various sources can be integrated using AI and ML to support personalized medicine.
AI has the potential to revolutionize cancer image analysis, shifting healthcare organization from organization-centric to patient-centric, which may improve clinical outcomes and reduce healthcare costs and could be transformative in precision oncology, selecting a patient’s therapy based on their tumour’s molecular profile. However, bridging the gap between emerging AI tools and clinical practice is a challenge.
The use of images and integrating these with clinical and molecular data can be a source of real-world data for evidence-generating studies. However, the sensitivity of radiomic feature values to image acquisition and reconstruction techniques and variations in segmentations among different users and software are challenges.
One of the reasons for the lack of translation of AI models to clinical application is the focus on increasing model performance at the expense of explainability. The AI community has started to recognize this limitation and has moved towards the development of explainable AI.
The main dependencies of applying AI driven models to clinical space are
a) Data Annotation and Standardization: AI models require labelled data for training. This necessitates manual annotation or curation of datasets, which can be a time-consuming process. Furthermore, the lack of standardized data on cancer-related health and the absence of uniformity in the collection and storage of unstructured data pose significant challenges.
b) Need for Diverse and Inclusive Datasets: AI algorithms and decision-support systems require diverse and inclusive datasets for training. The lack of such data is a major barrier to the widespread adoption of these systems in cancer care.
c) High Dimensionality and Class Imbalance: Medical datasets often have many features compared to t,īṁhe number of records, leading to high dimensionality. Additionally, there is often an unequal distribution of classes in these datasets, particularly in cancer data, leading to class imbalance. These issues can affect the performance of AI models.
d) Model Generalizability, sparsity and Reproducibility: AI models need to be validated across various sites to enhance their generalizability. However, data sparsity and non-standardized therapeutic approaches pose challenges in reproducing the output of these models across different healthcare systems.
e) Data Drift Over Time: AI algorithms and decision-support systems are susceptible to data drift over time, which can impact their performance. This is particularly concerning when AI and ML are used continuously in treatment or diagnosis.
f) Data Openness and Interpretability: The “black box” nature of AI poses significant challenges with data openness and interpretability. This, along with inherent bias towards minority cohorts, limits the reproducibility of AI models and perpetuates healthcare disparities.
g) Deep Network Training Difficulties: Training deep networks involves several non-linear transformations, which can lead to difficulties. While there are ways to solve these problems, such as using effective and efficient optimizers, thoughtful initialization techniques, and activation functions based on local competition, these methods still need improvement.
Limitations of Current AI Algorithms are its lack of broad intelligence and are not adaptable enough to handle additional clinical diagnostic tasks. However, transfer-learning techniques can be used to modify a fully trained AI algorithm to carry out closely related tasks and offers a solution to these limitations by enabling a pre-trained AI model to adapt to new but related tasks. The transfer learning methods used in image analysis in the clinical diagnostics are:
A. Feature Extraction: This method involves using a pre-trained model as a fixed feature extractor. The model’s learned features from a related task are used to train a new model for the target task. For example, a model trained on general medical images can be used to extract features for a specific diagnostic task like tumor detection [24].
B. Fine-Tuning: In this approach, a pre-trained model is further trained (fine-tuned) on a new dataset. This method adjusts the model’s parameters to better fit the new task while retaining the knowledge gained from the original task. Fine-tuning is particularly effective when the new dataset is small [24].
C. Self-Supervised Pretraining: This technique leverages large amounts of unlabelled data to pretrain a model, which is then fine-tuned on a smaller, labelled dataset. This method improves feature representations and enhances the model’s performance on the target task [7].
D. Hybrid Approaches: Combining feature extraction and fine-tuning can also be effective. For instance, using a pre-trained model to extract features and then fine-tuning the model on the new task can yield better results [24].
Applications in Clinical Diagnostics
I. Medical Imaging: Transfer learning has been widely used in medical image classification, such as detecting pneumonia from chest X-rays or identifying tumours in MRI scans (Litjens et al., 2017).
II. Predicting Clinical Outcomes: Models pre-trained on animal-based molecular data can be adapted to predict human clinical outcomes, enhancing the accuracy and reliability of diagnostic tools [6].
By harnessing transfer learning, AI algorithms can transcend their current limitations, becoming more adaptable and effective in early cancer detection, ultimately enhancing treatment options. In essence, AI-driven systems have the potential to deliver world-class diagnostic tools to remote healthcare settings. As these systems continue to evolve, they offer great promise for improving patient outcomes, minimizing diagnostic errors, and increasing global accessibility to cancer care. Nevertheless, challenges such as data privacy, model interpretability, and regulatory approvals must be tackled to fully realize the potential of AI and machine learning in oncology.
Conclusions
The integration of AI and ML in medical image analysis has revolutionized cancer diagnosis, prognosis, and treatment. These technologies have significantly enhanced early detection, personalized medicine, and monitoring of treatment responses, leading to improved patient outcomes. Despite challenges such as data annotation, standardization, and model generalizability, the benefits of AI/ML in healthcare far outweigh the limitations. The continuous advancements in AI/ML algorithms and tools promise a future where cancer care is more accurate, efficient, and tailored to individual patients’ needs. As we move forward, addressing the existing challenges and embracing the potential of AI/ML will be crucial in transforming cancer diagnosis and treatment.
References
1. Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015). Machine learning applications in cancer prognosis and prediction. In Computational and Structural Biotechnology Journal (Vol. 13). https://doi.org/10.1016/j.csbj.2014.11.005
2. Halalli, B., & Makandar, A. (2018). Computer Aided Diagnosis – Medical Image Analysis Techniques. In Breast Imaging. https://doi.org/10.5772/intechopen.69792
3. Li, M., Jiang, Y., Zhang, Y., & Zhu, H. (2023). Medical image analysis using deep learning algorithms. Frontiers in Public Health, 11. https://doi.org/10.3389/fpubh.2023.1273253
4. Sistaninejhad, B., Rasi, H., & Nayeri, P. (2023). A Review Paper about Deep Learning for Medical Image Analysis. In Computational and Mathematical Methods in Medicine (Vol. 2023). https://doi.org/10.1155/2023/7091301
5. Liu, X., Gao, K., Liu, B., Pan, C., Liang, K., Yan, L., Ma, J., He, F., Zhang, S., Pan, S., & Yu, Y. (2021). Advances in Deep Learning-Based Medical Image Analysis. In Health Data Science (Vol. 2021). https://doi.org/10.34133/2021/8786793
6. Bejnordi, B. E., Veta, M., Van Diest, P. J., Van Ginneken, B., Karssemeijer, N., Litjens, G., Van Der Laak, J. A. W. M., Hermsen, M., Manson, Q. F., Balkenhol, M., Geessink, O., Stathonikos, N., Van Dijk, M. C. R. F., Bult, P., Beca, F., Beck, A. H., Wang, D., Khosla, A., Gargeya, R., … Venâncio, R. (2017). Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA – Journal of the American Medical Association, 318(22). https://doi.org/10.1001/jama.2017.14585
7. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639). https://doi.org/10.1038/nature21056
8. Lee, H., Seo, S., Won, S., Park, W. Y., Choi, J. Y., Lee, K. H., Lee, S. H., & Moon, S. H. (2023). Comparative analysis of batch correction methods for FDG PET/CT using metabolic radiogenomic data of lung cancer patients. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-45296-9
9. Pinto-Coelho, L. (2023). How Artificial Intelligence Is Shaping Medical Imaging Technology: A Survey of Innovations and Applications. In Bioengineering (Vol. 10, Issue 12). https://doi.org/10.3390/bioengineering10121435
10. Rizwan I Haque, I., & Neubert, J. (2020). Deep learning approaches to biomedical image segmentation. In Informatics in Medicine Unlocked (Vol. 18). https://doi.org/10.1016/j.imu.2020.100297
11. Salvi, M., Acharya, U. R., Molinari, F., & Meiburger, K. M. (2021). The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis. In Computers in Biology and Medicine (Vol. 128). https://doi.org/10.1016/j.compbiomed.2020.104129
12. Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J. C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., Buatti, J., Aylward, S., Miller, J. V., Pieper, S., & Kikinis, R. (2012). 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic Resonance Imaging, 30(9). https://doi.org/10.1016/j.mri.2012.05.001
13. Lowekamp, B. C., Chen, D. T., Ibáñez, L., & Blezek, D. (2013). The design of simpleITK. Frontiers in Neuroinformatics, 7(DEC). https://doi.org/10.3389/fninf.2013.00045
14. Bolocan, V. O., Secareanu, M., Sava, E., Medar, C., Manolescu, L. S. C., Rașcu, A. Ștefan C., Costache, M. G., Radavoi, G. D., Dobran, R. A., & Jinga, V. (2024). Correction:Convolutional Neural Network Model for Segmentation and Classification of Clear Cell Renal Cell Carcinoma Based on Multiphase
CT Images(J. Imaging, (2023), 9, (280), 10.3390/jimaging9120280). In Journal of Imaging (Vol. 10, Issue 2). https://doi.org/10.3390/jimaging10020035
15. Stollmayer, R., Budai, B. K., Rónaszéki, A., Zsombor, Z., Kalina, I., Hartmann, E., Tóth, G., Szoldán, P., Bérczi, V., Maurovich-Horvat, P., & Kaposi, P. N. (2022). Focal Liver Lesion MRI Feature Identification Using Efficientnet and MONAI: A Feasibility Study. Cells, 11(9). https://doi.org/10.3390/cells11091558
16. Pati, S., Singh, A., Rathore, S., Gastounioti, A., Bergman, M., Ngo, P., Ha, S. M., Bounias, D., Minock, J., Murphy, G., Li, H., Bhattarai, A., Wolf, A., Sridaran, P., Kalarot, R., Akbari, H., Sotiras, A., Thakur, S. P., Verma, R., … Bakas, S. (2020). The cancer imaging phenomics toolkit (CaPTk): Technical overview. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11993 LNCS. https://doi.org/10.1007/978-3-030-46643-5_38
17. Bankhead, P., Loughrey, M. B., Fernández, J. A., Dombrowski, Y., McArt, D. G., Dunne, P. D., McQuaid, S., Gray, R. T., Murray, L. J., Coleman, H. G., James, J. A., Salto-Tellez, M., & Hamilton, P. W. (2017). QuPath: Open source software for digital pathology image analysis. Scientific Reports, 7(1). https://doi.org/10.1038/s41598-017-17204-5
18. Koh, D. M., Papanikolaou, N., Bick, U., Illing, R., Kahn, C. E., Kalpathi-Cramer, J., Matos, C., Martí-Bonmatí, L., Miles, A., Mun, S. K., Napel, S., Rockall, A., Sala, E., Strickland, N., & Prior, F. (2022). Artificial intelligence and machine learning in cancer imaging. In Communications Medicine (Vol. 2, Issue 1). https://doi.org/10.1038/s43856-022-00199-0
19. Weng, X., Song, F., Tang, M., Wang, K., Zhang, Y., Miao, Y., Chan, L. W. C., Lei, P., Hu, Z., & Yang, F. (2023). MDM-U-Net: A novel network for renal cancer structure segmentation. Computerized Medical Imaging and Graphics, 109. https://doi.org/10.1016/j.compmedimag.2023.102301
20. Wang, J., & Liu, X. (2021). Medical image recognition and segmentation of pathological slices of gastric cancer based on Deeplab v3+ neural network. Computer Methods and Programs in Biomedicine, 207. https://doi.org/10.1016/j.cmpb.2021.106210
21. Vulli, A., Srinivasu, P. N., Sashank, M. S. K., Shafi, J., Choi, J., & Ijaz, M. F. (2022). Fine‐Tuned DenseNet‐169 for Breast Cancer Metastasis Prediction Using FastAI and 1‐Cycle Policy. Sensors, 22(8). https://doi.org/10.3390/s22082988
22. Salinas, M. P., Sepúlveda, J., Hidalgo, L., Peirano, D., Morel, M., Uribe, P., Rotemberg, V., Briones, J., Mery, D., & Navarrete-Dechent, C. (2024). A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis. NPJ digital medicine, 7(1), 125. https://doi.org/10.1038/s41746-024-01103-x
23. Zhang, B., Shi, H., & Wang, H. (2023). Machine Learning and AI in Cancer Prognosis, Prediction, and Treatment Selection: A Critical Approach. In Journal of Multidisciplinary Healthcare (Vol. 16). https://doi.org/10.2147/JMDH.S410301
24. Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. In Medical Image Analysis (Vol. 42). https://doi.org/10.1016/j.media.2017.07.005
