The next frontier: Advancing foundation models with multimodal data

Aug 31, 2026
Flat illustration of an electronic health record with a data streamAI-generated from publicly available materials.

Researchers are advancing AI capabilities in health by integrating electronic health records (EHR) with genomic data, aiming to enhance disease prediction and personalize care.

The collaboration between Verily Health, NVIDIA, and the NIH's All of Us Research Program has led to the development of a groundbreaking multimodal foundation model. This model merges EHR and genomic data, showcasing the potential to create a more comprehensive understanding of health and improve precision medicine. By combining these two types of data, the model can identify inherited disease risks earlier than traditional methods, addressing the long-standing challenge of integrating biological and clinical information.

Utilizing foundation models allows for the identification of complex disease risk patterns from diverse health data sources. Verily's approach demonstrated significant improvements in predicting Type 2 diabetes risk by integrating genetic risk scores with EHR data. This integration not only enhanced predictive accuracy but also extended risk projection timelines from immediate to clinically relevant periods of five to ten years. The use of NVIDIA GPUs further optimized computational efficiency, facilitating faster insights in research environments.

This pioneering work lays the groundwork for future advancements in health data integration, which could include wearables, medical imaging, and patient-reported outcomes. Verily's ongoing efforts aim to refine these models, ensuring they evolve with the continuous influx of multidimensional health data. By illustrating the power of multimodal AI, this initiative opens pathways for future applications in areas like biomarker discovery and health system precision medicine programs.

Read the original article: BioPharma Dive