Dendritic Edge Vision

The Story of Dendritic MobileNet Inspiration Deep learning has achieved incredible feats, but standard artificial neurons are significant oversimplifications of their biological counterparts. In 2012, neuroscience revealed that single neurons perform complex computations in their dendrites—a capability completely absent in the standard $y = \sigma(Wx + b)$ model.

We were inspired by Perforated AI's mission to bridge this gap. Our goal was to take a standard, highly optimized "edge" architecture—MobileNetV3—and see if we could squeeze out more performance not by simply making it deeper, but by making its neurons smarter. We wanted to answer the question: Can bio-inspired dendritic computation improve the accuracy of lightweight models without exploding the computational budget?

What We Learned Working with Perforated AI taught us that "biological plausibility" isn't just a buzzword—it's a viable path to efficiency.

Parameter Efficiency: We learned that dendritic layers add parameters in a structured way that can capture more complex non-linear relationships than simple dense layers. Hyperparameter Sensitivity: We discovered that dendritic models have their own unique scaling laws. Simply adding more dendrites ($D$) doesn't always guarantee better performance; finding the "sweet spot" (e.g., $D=8$ for our use case) requires rigorous experimentation. How We Built It We built Dendritic MobileNet using a robust stack of open-source tools:

PyTorch & Torchvision: We started with a pre-trained MobileNetV3-Small, a standard for efficient mobile vision. Perforated AI: We used the perforatedai library to surgically replace the model's final classification layers. We utilized UPA.initialize_pai(model) to automatically wrap generic layers into PAINeuronModules, and then programmatically injected dendritic structures using create_new_dendrite_module(). Weights & Biases (WandB): To scientifically determine the impact of dendrites, we implemented a Bayesian Hyperparameter Sweep. This allowed us to automate the search for the optimal Dendrite Count and Learning Rate. The core mathematical transformation involved moving from standard summation to dendritic integration: $$y = \sigma(\sum_{i} w_i x_i + b) \quad \xrightarrow{\text{Dendritic}} \quad y = f_{\text{soma}}(\sum_{d \in \text{dendrites}} g_d(\mathbf{x}_d))$$

Challenges We Faced Integration Complexity: Integrating a novel library like PerforatedAI into an existing architecture wasn't plug-and-play. We had to carefully manage PYTHONPATH issues and ensure the library was correctly importable alongside our custom scripts. Shadowing Issues: We encountered a tricky bug where a local folder named wandb shadowed the actual wandb library, causing mysterious AttributeErrors. Debugging this required a deep dive into Python's import system. Training Time: Simulating complex dendritic structures on a CPU (our available hardware) proved computationally intensive. Training a single epoch took over an hour, forcing us to be extremely strategic with our hyperparameter sweeps. We had to balance "exploration" (trying many configs) with the hard reality of "compute time."

Built With

• biases
• computer
• hyperparameter
• perforatedai
• pytorch
• sweeps)
• torchvision
• tqdm
• vision
• weights
• ython