Multi-Modal Generative AI and LLM-Driven Clinical Interpretation Pipeline for Early Detection of Oncology Markers in Radiological Imaging

Gangadhar Vasanthapuram

doi:10.63519/IJCSERD_15_03_005

Authors

Gangadhar Vasanthapuram Technology Architect, Smartworks LLC, Hillsborough, New Jersey 08844, USA. Author

DOI:

https://doi.org/10.63519/IJCSERD_15_03_005

Keywords:

Radiology, AI, Pipeline, LLM, Oncology, Generative, Multi-Modal

Abstract

In this work, AI pipeline based on generative models and large language models that combine to improve early cancer detection from radiological imaging. The system fuses text and image modalities to increase diagnostic accuracy, reduce report generation and increase clinical efficiency on its way to promise of advancements in precision oncology and automated interpretation.

References

Garg, A., Gupta, S., Vats, S., Handa, P., & Goel, N. (2024). Prospect of large language models and natural language processing for lung cancer diagnosis: A systematic review. Expert Systems, 41(11), e13697. https://doi.org/10.1111/exsy.13697

Ruan, W., Lyu, Y., Zhang, J., Cai, J., Shu, P., Ge, Y., ... & Liu, T. (2025). Large Language Models for Bioinformatics. arXiv preprint arXiv:2501.06271. https://doi.org/10.48550/arXiv.2501.06271

Sultan, R. I., Zhu, H., Li, C., & Zhu, D. (2025). BiPVL-Seg: Bidirectional Progressive Vision-Language Fusion with Global-Local Alignment for Medical Image Segmentation. arXiv preprint arXiv:2503.23534. https://doi.org/10.48550/arXiv.2503.23534

Zhang, X., Zhao, L., Sun, H., Li, Y., Chen, W., & Wang, J. (2025). Multimodal Data Fusion Techniques for Accurate Elderly Mental State Evaluation. https://www.researchgate.net/profile/Xinyi-Zhang-235/publication/389323986_Multimodal_Data_Fusion_Techniques_for_Accurate_Elderly_Mental_State_Evaluation/links/67be946c96e7fb48b9cde545/Multimodal-Data-Fusion-Techniques-for-Accurate-Elderly-Mental-State-Evaluation.pdf

Velez-Arce, A., Li, M. M., Gao, W., Lin, X., Huang, K., Fu, T., ... & Zitnik, M. (2024). Signals in the cells: multimodal and contextualized machine learning foundations for therapeutics. bioRxiv. 10.1101/2024.06.12.598655

Shen, Y., Xu, Y., Ma, J., Rui, W., Zhao, C., Heacock, L., & Huang, C. (2024). Multi-modal large language models in radiology: principles, applications, and potential. Abdominal Radiology, 1-13. https://doi.org/10.1007/s00261-024-04708-8

Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., ... & Pei, Y. (2024). A comprehensive review on synergy of multi-modal data and ai technologies in medical diagnosis. Bioengineering, 11(3), 219. https://doi.org/10.3390/bioengineering11030219

Fountzilas, E., Pearce, T., Baysal, M. A., Chakraborty, A., & Tsimberidou, A. M. (2025). Convergence of evolving artificial intelligence and machine learning techniques in precision oncology. npj Digital Medicine, 8(1), 75. https://doi.org/10.1038/s41746-025-01471-y

Buess, L., Keicher, M., Navab, N., Maier, A., & Arasteh, S. T. (2025). From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine. arXiv preprint arXiv:2502.09242. https://doi.org/10.48550/arXiv.2502.09242

Sun, D., Hadjiiski, L., Gormley, J., Chan, H. P., Caoili, E., Cohan, R., ... & Gulani, V. (2024). Outcome prediction using multi-modal information: integrating large language model-extracted clinical information and image analysis. Cancers, 16(13), 2402. https://doi.org/10.3390/cancers16132402