Dense-Tune CIFAR-10

• 

  Cifar-10 DataSet

Inspiration

Standard CNNs on CIFAR-10 typically plateau around 85% accuracy. Manual architecture tuning is time-consuming and requires expertise. I wanted to see if PerforatedAI's dendritic optimization could automatically break through this ceiling.

What it does

Dense-Tune automatically adds dendritic connections when model learning saturates, pushing CIFAR-10 accuracy from 84.59% to 86.85%—a 14.7% Remaining Error Reduction with zero manual intervention.

How I built it

1. Implemented a 6-layer CNN with PAI integration
2. Used DOING_HISTORY switch mode to detect plateaus
3. Let PAI automatically add dendrites twice during training
4. Tracked results with auto-generated PAI graphs

Challenges I ran into

• Training on CPU took ~27 hours (no GPU available)
• Discovered a bug: lr_scheduler.step() warning after dendrite additions
• Documented the bug with suggested fixes in BUG_REPORT.md

Accomplishments

• Achieved 14.7% RER on CIFAR-10
• 2 successful dendritic growth cycles
• Created W&B sweep script for future experiments
• Contributed bug report + improved .gitignore to the repo

What I learned

Dendritic optimization works! The model broke through accuracy ceilings that would normally require manual architecture search.

What's next

Run full W&B sweep on GPU to explore hyperparameter impact across 100+ configurations.

Built With

• perforatedai
• python
• pytorch
• weights-&-biases