An Interpretable Multi-Modal Ensemble Framework for Breast Cancer Analysis Using Imaging, Omics and Biomedical Literature
DOI:
https://doi.org/10.3991/ijoe.v22i05.60535Keywords:
Breast cancer, medical imaging analysis, biomedical data integration, multi-model system, literature miningAbstract
Although breast cancer is still a concern in the global healthcare domain, there is an immediate requirement for intelligent systems that can help in the early and accurate diagnosis of the disease based on the synthesis of various types of data. This paper proposes AutoMedEnsemble, an artificial intelligence-powered multi-modal ensemble system that integrates the extraction of healthcare literature, gene expression analysis, and histopathological image assessment. The literature processing module with BioBERT has a precision of 91.8% and an F1-score of 90.5%. The omics-based component, analyzing gene expressions from the NCBI Gene Expression Omnibus (GSE45827), achieves an accuracy of 93.5% with an F1-score of 93.7%. The imaging module utilizes a ResNet50 architecture with Grad-CAM for interpretability, achieving an accuracy of 95.2% and an F1-score of 95.5%. While evaluated as independent modules on benchmark datasets, this framework demonstrates a proof of concept for an interpretable, data-driven decision-support dashboard for breast cancer research.
References
[1] Shinde, M., & Dixit, D. (2024). Deep learning approach for breast cancer detection from histopathology images. South Eastern European Journal of Public Health, 2383–2397.
[2] Kaur, A., Kaushal, C., Sandhu, J. K., Damaševičius, R., & Thakur, N. (2024). Histopathological image diagnosis for breast cancer based on deep mutual learning. Diagnostics, 14(17), Article 95. https://doi.org/10.3390/diagnostics14010095
[3] Alom, M. R., Farid, F. A., Rahaman, M. A., Rahman, A., Debnath, T., Miah, A. S. M., & Mansor, S. (2025). An explainable AI-driven deep neural network for accurate breast cancer detection from histopathological and ultrasound images. Scientific Reports, 15(3), 1–34.
[4] Dada, E. G., Oyewola, D. O., & Misra, S. (2024). Computer-aided diagnosis of breast cancer from mammogram images using deep learning algorithms. Journal of Electrical Systems and Information Technology, 11, 38.
[5] M. D. P., M. A., Ali, Y., & V. S. (2025). Effective BCDNet-based breast cancer classification model using hybrid deep learning with VGG16-based optimal feature extraction. BMC Medical Imaging, 25, Article 12. https://doi.org/10.1186/s12880-024-01538-4
[6] Talaat, F., Gamel, S., El-Balka, R., Shehata, M., & ZainEldin, H. (2024). Grad-CAM enabled breast cancer classification with a 3D Inception-ResNet-V2: Empowering radiologists with explainable insights. Cancers, 16(15), 3668.
[7] Wu, J., Li, Y., Gong, W., Li, Q., Han, X., & Zhang, T. (2025). Multi-modality radiomics diagnosis of breast cancer based on MRI, ultrasound, and mammography. BMC Medical Imaging, 25, 265.
[8] Aliouane, S. E., Chehili, H., Boulahrouf, K., Abdelaziz, A., Khlifa, N., & Hamidechi, M. A. (2025). Integrating deep learning and SHAP for breast cancer classification and biomarker discovery using gene expression data. IEEE Access, 13(2), 49693–49709.
[9] Ansari, Z. A., Tripathi, M. M., & Ahmed, R. (2025). The role of explainable AI in enhancing breast cancer diagnosis using machine learning and deep learning models. Discover Artificial Intelligence, 5(4), 75.
[10] Sharma, A., Debik, J., Naume, B., Ohnstad, H. O., Sahlber, K. K., Borgen, E., Børresen-Dale, A.-L., Engebråten, O., Fritzman, B., Garred, B., Geisler, J., Geitvik, G. A., Hofvind, S., Kristensen, V. N., Kåresen, R., Langerød, A., Lingjærde, O. C., Mælandsmo, G. M., Russnes, H. G., … OSBREAC. (2024). Comprehensive multi-omics analysis of breast cancer reveals distinct long-term prognostic subtypes. Oncogenesis, 13(7), 22.
[11] Karam, J., Rejto, P. A., Bienkowska, J. R., Mu, X. J., & Roh, W. (2025). Identification of breast cancer subtypes and drug response prediction through forward and reverse translation. npj Precision Oncology, 9(10), 267.
[12] Neagu, A.-N., Whitham, D., Bruno, P., Morrissiey, H., Darie, C. A., & Darie, C. C. (2023). Omics-based investigations of breast cancer. Molecules, 28(11), 4768.
[13] Rakhshaninejad, M., Fathian, M., Shirkoohi, R., Barzinpour, F., & Gandomi, A. H. (2024). Refining breast cancer biomarker discovery and drug targeting through an advanced data-driven approach. BMC Bioinformatics, 25(8), 33.
[14] Haji, H. E., Sbihi, N., Handri, K. E., Bahaj, A., Elkasri, M., Souadka, A., & Ghogho, M. (2025). Unveiling breast cancer causes through knowledge graph analysis and BioBERT-based factuality prediction. In BIOSTEC: HEALTHINF 2025 (pp. 141–148).
[15] Nie, Y., & Yu, J. (2013). Mining breast cancer genes with a network-based noise-tolerant approach. BMC Systems Biology, 7(14), 49.
[16] Moreau, E., Hardiman, O., Heverin, M., & O’Sullivan, D. (2024). Mining impactful discoveries from the biomedical literature. BMC Bioinformatics, 25(13), 303.
[17] Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020). BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36, 1234–1240.
[18] forderation. (2025). BreaKHis 400X [Data set]. Kaggle. https://www.kaggle.com/datasets/forderation/breakhis-400X
[19] Spanhol, F. A., Oliveira, L. S., Petitjean, C., & Heutte, L. (2016). A dataset for breast cancer histopathological image classification. IEEE Transactions on Biomedical Engineering, 63(7), 1455–1462.
[20] U.S. National Library of Medicine. (n.d.). PubMed. National Center for Biotechnology Information. https://pubmed.ncbi.nlm.nih.gov/
[21] National Center for Biotechnology Information. (n.d.). GSE45827: Breast cancer gene expression dataset. https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE45827
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Sayeedakhanum Pathan, Dhanush Kandagatla, Takkedu Malathi, Syeda Imrana Fatima, Vijay Kumar Gugulothu, Purude Vaishali Narayanrao
This work is licensed under a Creative Commons Attribution 4.0 International License.
