Micrograd Studio: Interactive Neural Autograd Lab

💡 Inspiration

Modern artificial intelligence education faces a critical pedagogy crisis: memorization over intuition. Students in university courses and online bootcamps are taught to write import torch, stack layers, and invoke loss.backward() without ever understanding how a single derivative flows through a neuron. When models diverge, explode, or fail to generalize, students lack the mathematical and visual intuition to diagnose why.

We were inspired by Andrej Karpathy's micrograd and Seymour Papert's constructionist learning philosophy: to truly understand a system, you must build it and see it live. We set out to create Micrograd Studio—a zero-dependency, browser-native AI learning laboratory that brings reverse-mode automatic differentiation, computation graph DAGs, and non-linear decision boundary manifolds to life with interactive 60 FPS visuals.

🚀 What It Does

Micrograd Studio is a complete, interactive, first-principles deep learning laboratory built entirely in TypeScript:

1. Pure Scalar Autograd Engine: Implements reverse-mode automatic differentiation from scratch supporting addition, subtraction, multiplication, division (quotient rule), powers, ReLU, Tanh, and Sigmoid activations.
2. Interactive 2D Decision Boundary Canvas: Renders real-time continuous prediction heatmaps and glowing (\hat{y} = 0) iso-contour manifolds across non-linear benchmark datasets (Two Moons, XOR, Concentric Circles, and Two-Arm Spirals) at 60 FPS.
3. Computation Graph (DAG) Inspector: Visualizes nodes, forward pass scalar evaluations, and backward pass accumulated gradients (\frac{\partial L}{\partial v}), allowing students to step through the chain rule node-by-node.
4. Rigorous Numerical Gradient Verification: An in-browser verification suite comparing analytical gradients computed via backpropagation against two-sided finite difference approximations to within (< 10^{-6}) relative error.
5. Real-Time Dynamic Loss Tracking & Hyperparameter Lab: Live vector SVG loss curve rendering, adjustable learning rates, L2 regularization (weight decay), and dynamic activation switching.

🔬 Mathematical Foundations & Algorithmic Design

1. Reverse-Mode Automatic Differentiation via Chain Rule

Every intermediate calculation produces a scalar Value node tracking its value (v), accumulated gradient (\frac{\partial L}{\partial v}), input parents, and a local gradient closure:

• Product Rule: For (c = a \cdot b): $$\frac{\partial L}{\partial a} += b \cdot \frac{\partial L}{\partial c}, \quad \frac{\partial L}{\partial b} += a \cdot \frac{\partial L}{\partial c}$$
• Quotient Rule: For (c = \frac{a}{b}): $$\frac{\partial L}{\partial a} += \frac{1}{b} \cdot \frac{\partial L}{\partial c}, \quad \frac{\partial L}{\partial b} += \left(-\frac{a}{b^2}\right) \cdot \frac{\partial L}{\partial c}$$
• Power Rule: For (c = a^k): $$\frac{\partial L}{\partial a} += \left(k \cdot a^{k-1}\right) \cdot \frac{\partial L}{\partial c}$$
• Activation Derivatives: $$\frac{\partial \text{ReLU}(x)}{\partial x} = \begin{cases} 1 & x > 0 \ 0 & x \le 0 \end{cases}, \quad \frac{\partial \tanh(x)}{\partial x} = 1 - \tanh^2(x), \quad \frac{\partial \sigma(x)}{\partial x} = \sigma(x)(1 - \sigma(x))$$

2. Topological Sort Graph Traversal

Backpropagation cannot execute until all downstream consumers have contributed their partial derivatives. When loss.backward() is called, our engine runs a recursive Depth-First Search (DFS) post-order traversal to linearize the Directed Acyclic Graph (DAG):

backward(): void {
  const topo: Value[] = [];
  const visited = new Set<Value>();
  const buildTopo = (v: Value) => {
    if (!visited.has(v)) {
      visited.add(v);
      for (const child of v._prev) buildTopo(child);
      topo.push(v);
    }
  };
  buildTopo(this);
  this.grad = 1.0; // Seed: dL/dL = 1.0
  for (let i = topo.length - 1; i >= 0; i--) {
    topo[i]._backward();
  }
}

Built With

• ai
• algorithms
• autograd
• automatic-differentiation
• calculus
• data-visualization
• deep-learning
• edtech
• educational-tech
• finite-difference-method
• graph-theory
• html5
• interactive-ui
• latex
• lucide-icons
• machine-learning
• math
• neural-networks
• numerical-analysis
• react
• svg
• tailwind-css
• topological-sort
• typescript
• vite