Multimodal Transformer Architectures for Neuropathology Study of Alzheimer’s Disease

Focused Research Program

Our Focus

The goal of the Multimodal Transformer Architectures for Neuropathology Study of Alzheimer’s Disease FRP is to improve premortem diagnosis of neuropathologic processes underlying cognitive impairment (mild cognitive impairment and dementia) utilizing a machine learning approach to definite patterns in MRI scans and neuropsychological data that predict underlying neuropathology and then understand these patterns through latent space analysis. The proposed work is highly interdisciplinary and requires expertise from five different departments across the Charles River Campus and the Medical Campus (both from the School of Medicine and the School of Public Health). 

This Focused Research Program is co-sponsored by the Digital Health Initiative at the Hariri Institute for Computing, the School of Public Health Population Health Data Science Program, the Clinical and Translational Science Institute, and the Evans Center for Interdisciplinary Biomedical Research.

Research Program Leaders

• Xin Zhang, PhD, Distinguished Professor of Engineering, Department of Mechanical Engineering, Electrical & Computer Engineering, and Biomedical Engineering

• Chad Farris, MD, PhD, Assistant Professor of Radiology and Section Chief of Neuroradiology

Research Thrusts

1. Creation of Machine Learning Architectures/Models to Accept Neuroimaging, Neuropsychological Data, and Neuropathologic Data 

This goals of this thrust are to 1) create a general data loader that can read, and preprocess multi-standard clinical data simultaneously; and 2) to construct a proper training process so that the encoders can produce cross-modality embeddings and map them in the same latent space. To accomplish this, the researchers will create a program capable of accepting/inputting magnetic resonance imaging scans (in original unprocessed format), neuropsychologic data, and neuropathologic data, and that also includes a transformer-based encoder-decoder architecture design.

2. Prediction of Neuropathologic Changes Underlying Dementia Based on Magnetic Resonance Imaging Scans and Neuropsychological Data

This thrust’s research goal is to train the developed machine learning program in Thrust 1 to identify MRI scans and neuropsychological testing profiles that correspond to specific neuropathological findings. The ultimate goal is to develop an upstream-to-downstream algorithm that can successfully predict the neuropathologic changes present in the brain of a living patient with cognitive impairment or dementia, which could direct therapies and inform prognosis for a patient. This algorithm can then be applied to other datasets unique to BU.

3. Investigation of Model’s Latent Space for Determination of Features in Magnetic Resonance Imaging Scans and Neuropsychological Data that Predict Neuropathology

The goal of thrust 3 is to determine the features used by the program developed in Thrust 2 to identify scans and neuropsychological profiles that fit with different neuropathologies through latent space analysis. This exploration would potentially provide an in-depth understanding of the medical scan analysis, new pattern recognition, and patient case clustering based on neuropsychologic data.

How to get involved?

For program specific inquiries and questions, please contact FRP leaders: Xin Zhang or Chad Farris

Faculty interested in submitting a Focused Research Programs proposal are strongly encouraged to discuss their ideas with Yannis Paschalidis, director of the Hariri Institute for Computing.

To learn more details about the Hariri Institute’s Focused Research Programs, visit here.