Inspiration
The detection of Alzheimer's is Important cause many people cannot afford expensive diagnosis. These models are trained inorder to Help who cannot undergo diagnosis under Expensive Specialists. Rural areas have limited access for appointment with Specialist neurologists who are mostly present in big cities. Also we can help in detecting early-stage disease which can help in slowing the decline.
What it does
I have trained Two Deeplearning models based on different Datasets for the detection of Alzheimer's Disease.
The first model uses the Dataset Given by Hack4Health Alzheimer's Hackathon MRI scans. This only uses MRI scans to train the model achieving 98.83% test accuracy and 98.7% validation accuracy. It classifies into 4 stages of the disease.
The second model is a multimodel which is trained on oasis MRI dataset+clinical dataset. This is combining the clinical features with MRI scans acheiving a per-slice accuracy of 78.18% with ROC_AUC 0.8506 and a per-patient accuracy 81.13% with ROC_AUC 0.8281. This model does binary classification - Healthy vs Impaired.
How we built it
For the Unimodel (Competition Dataset):
- Used EfficientNet-B0 backbone with 5 million parameters
- The competition dataset had 5,120 images with 4 classes
- Main challenge was Class imbalance - Class 1 had only 49 samples
- Split was 85% train, 15% validation
- Trained using Adam optimizer with learning rate 1e-4
- Used CrossEntropyLoss without class weights
For the Multimodal (OASIS Dataset):
- Used dataset of 347 patients, approximately 15 slices of MRI scan of each patient is taken
- The distribution of dataset was Non_Demented-13794, Very_mild_Dementia-3377, Mild_Dementia-902, Moderate_Dementia-88
- The clinical features had Age, MMSE, CDR, nWBV, eTIV as features
- Architecture combines ResNet18 for images and MLP for clinical features
- Image branch outputs 512 features, clinical branch outputs 32 features
- Fusion layer concatenates both and classifies into 2 classes
- Used patient-level splitting to avoid data leakage
Technologies used: PyTorch, torchvision, timm, pandas, numpy, scikit-learn, matplotlib, seaborn
Challenges we ran into
In the Unimodel case: As mentioned earlier the Hackathon Dataset has only 49 samples of class 1. Due to this class imbalance i tried to use class-weighted loss functions but it reduced the accuracy of the model than simply training the model. Simple training was better than using complex techniques.
Architecture Selection: EfficientNet-B0, with approximately 5M parameters, was better than ResNet18 which has more parameters. This shows that well scaled architecture's is better.
Multimodal Integration: There were 347 patients data was available in which for each person 15 slices of MRI scans were available. The per-slice evaluation does not fully capture the disease-level patterns. Aggregating predictions across multiple slices improve the robustness emphasizing the importance of per-patient evaluation.
Data Leakage Prevention: Had to implement patient-level splitting for the OASIS dataset to ensure that slices from the same patient don't appear in both training and validation sets.
Accomplishments that we're proud of
Unimodel Results:
- Achieved 98.83% test accuracy and 98.7% validation accuracy
- The precision of class-1 is 100% even with scarcity of only 49 samples which shows that the model was successful in extracting meaningful features
- Simple approaches worked better than complex class-weighting methods
Multimodal Results:
- Achieved 81.13% per-patient accuracy
- Successfully integrated MRI scans with clinical features
- Implemented proper patient-level evaluation preventing data leakage
- Built complete multimodal fusion architecture
Overall:
- Built two complete systems demonstrating different approaches to Alzheimer's detection
- Created professional documentation with GitHub repository
- Generated comprehensive results with confusion matrices and training curves
What we learned
Technical Learnings:
- Simple Training Performed better: Class-weighted loss functions reduced accuracy compared to standard training for imbalanced data
- Architecture efficiency matters: EfficientNet-B0 with 5M parameters outperformed larger models, showing well-scaled architectures are more important than size
- Patient-level evaluation is crucial: Per-slice evaluation doesn't capture disease patterns properly - aggregating across multiple slices per patient gives more robust results
- Multimodal fusion is challenging: Just combining features doesn't automatically improve results - need better fusion strategies
Problem-Solving Insights:
- Sometimes simpler approaches work better than complex solutions
- Data quality and proper validation methodology matters more than dataset size
- Understanding the medical context (patient-level vs slice-level) is important for evaluation
What's next for Team Hack4
Immediate Improvements:
- Ensemble Approaches: Combining multiple models (EfficientNet, ResNet, DenseNet) could improve robustness
- Better fusion techniques: Use attention mechanisms or cross-modal learning for the multimodal model to improve the 81.13% accuracy
Long-term Goals:
- Larger Datasets: Train on larger datasets like ADNI (Alzheimer's Disease Neuroimaging Initiative) or UK Biobank for better generalization
- 3D Analysis: Current models use 2D slices - implementing 3D convolutional networks to analyze full MRI volumes could capture more spatial context
- Temporal Analysis: Develop models to track disease progression over time using longitudinal MRI data
- Clinical Deployment: Conduct prospective clinical validation studies with radiologists to assess real-world performance
Log in or sign up for Devpost to join the conversation.