Doodle Defense

• 

  When doodles attack, your pencil fights back

Inspiration We were inspired by the chaotic joy of childhood doodles and the powerful accessibility of modern machine learning. We wanted to create a game where strategy meets creativity, where your drawing skills directly impact your survival. Google's QuickDraw project showed us that AI could understand human sketches, and we wondered: what if those sketches could come to life and fight for you? The hackathon theme of "AI and Play" was the perfect catalyst to blend these concepts into a tower defense game where your imagination is your army.

What it does Doodle Defense is a browser-based game where players defend their base by drawing units on a canvas. An AI model analyzes your sketch in real-time, identifies what you tried to draw (e.g., cat, cannon, sword), and spawns a corresponding unit with stats based on your drawing quality. A quick, accurate drawing creates a powerful warrior; a sloppy scribble creates a weak, pathetic fighter. Waves of enemies (like "Ink Blots" and "Erase-ers") march toward your base, and you must draw rapidly and strategically to survive.

How we built it Frontend & Game Engine: React with Kaboom.js for the game loop, rendering, and physics

Machine Learning: TensorFlow.js running a pre-trained Convolutional Neural Network (CNN) based on Google's QuickDraw dataset

Drawing Interface: HTML5 Canvas with custom drawing tools and smoothing algorithms

Game Logic: JavaScript classes for unit management, wave spawning, and resource allocation

Assets & Polish: Custom pixel art from Aseprite and free sound effects from Freesound.org

The core technical integration involves capturing canvas data, feeding it to the TF.js model, and mapping the results to game units: UnitPower=BasePower(class)×confidence^2

This quadratic scaling makes drawing quality dramatically affect gameplay.

Challenges we ran into ML Latency: Running inference in the browser initially caused noticeable delays. We optimized by resizing drawings to 28x28 pixels before processing and implementing a loading animation.

Drawing Recognition Accuracy: The model sometimes misclassified similar shapes (e.g., "circle" vs. "sun"). We implemented a confidence threshold and added a "redraw" suggestion for low-confidence predictions.

Game Balance: Balancing unit stats, drawing time, and wave difficulty was tricky. We created multiple test waves and adjusted formulas until the game felt challenging but fair.

State Management: Synchronizing the game state between Kaboom.js and React components required careful architecture to prevent bugs and performance issues.

Accomplishments that we're proud of Seamless ML Integration: Creating a real-time drawing recognition system that feels magical and responsive

Engaging Gameplay: Designing a core loop that's easy to learn but difficult to master

Polish: Implementing juicy feedback like screen shake, particle effects, and satisfying sound design

Humor: Embracing the AI's misclassifications as a feature rather than a bug, leading to hilarious moments

What we learned TensorFlow.js is powerful but requires careful optimization for real-time applications

Game feel is everything - small details like sound effects and animations dramatically improve player experience

Constraints breed creativity - limiting units to what the ML model could recognize forced us to design more creatively

Browser-based ML is finally viable for consumer applications thanks to WebGL acceleration

What's next for Doodle Defense Multiplayer Mode: Let players compete or cooperate with their doodle armies

Expanded Recognition: Train a custom model on more drawing types and styles

Campaign Mode: Add a story, boss battles, and progressive difficulty

Mobile App: Develop a native iOS/Android version leveraging device GPUs for faster inference

Community Features: Let players submit their own drawings to be added to the recognition database

Built With

• canvas
• html5