Breast Cancer Detection Using Late Fusion Approach on Multimodal Data with Deep Learning

Abstract

Breast cancer stands as one of the most formidable challenges in contemporary global healthcare, with approximately 18.1 million new cancer cases reported worldwide and an urgent need for revolutionary advances in early detection methodologies. Current breast cancer diagnostic methodologies present significant limitations through unimodal approaches, where mammography suffers from masking effects and false positives/negatives, thermography demonstrates low specificity of 57.8%, and clinical tabular data alone proves insufficient for definitive diagnosis. This research presents a comparative study of four late fusion strategies for breast cancer diagnosis, integrating predictions from deep learning models trained on mammography images, thermography images, and clinical tabular data. The fusion methods evaluated include probability multiplication (product rule), weighted averaging, stacking metaclassifier, and log-opinion pool fusion. The log-opinion pool fusion method achieved superior performance with 97.5% overall accuracy, surpassing the other fusion approaches and all individual. The method has also achieved precision of 0.91–0.95 for benign cases 1.00 for malignant cases, with fusion approaches maintaining zero false positive rate and recall for malignant cases up to 95%.