DendriLite

Inspiration

Modern neural networks achieve high accuracy, but often at the cost of increased size, computation, and energy consumption. Inspired by recent neuroscience discoveries showing that biological neurons perform additional computation inside dendrites, this project explores how dendritic optimization can make neural networks smarter without making them larger.

The PyTorch Dendritic Optimization Hackathon was a perfect opportunity to bridge neuroscience-inspired ideas with practical deep learning models.

What it does

This project applies dendritic optimization to a baseline PyTorch CNN model to enhance internal feature representations while keeping the architecture largely unchanged. The goal is to improve learning efficiency rather than simply increasing model depth or parameters.

How we built it

We started with a simple convolutional neural network (CNN) implemented in PyTorch. After establishing a working baseline model, we integrated dendritic-inspired components using the Perforated AI library.

We trained both the baseline and dendritic-enhanced models under identical conditions and compared their training behavior, loss curves, and parameter counts to evaluate the impact of dendritic optimization.

Challenges we faced

One of the main challenges was understanding how to correctly integrate dendritic optimization into an existing PyTorch workflow without redesigning the entire model. Additionally, interpreting subtle performance differences required careful comparison and visualization.

What we learned

This project demonstrated that biologically inspired computation can meaningfully influence neural network behavior. Even small architectural changes inspired by dendrites can lead to smarter representations without relying on brute-force scaling.

Built With

• matplotlib
• numpy
• python
• pytorch