Terracatch

TerraCatch turns real-world wildlife discovery into a creature-collecting RPG: spot animals on video, identify species with AI, unlock unique battle stats, and train your squad.

Comment

TerraCatch — Real-World Wildlife Meets Creature-Collecting RPG

Inspiration

Growing up, millions of us dreamed of exploring the wilderness with a Pokédex in hand, discovering creatures in our own backyards. Yet in modern life, people spend more time looking at virtual monsters on screens than noticing the astounding biodiversity right outside their windows.

We asked ourselves: What if the real world was the game? What if filming a red-tailed hawk, a praying mantis, or an eastern box turtle in your local park gave you a real-world digital creature with unique battle stats, elemental typings, and move sets derived from its actual ecological traits?

Inspired by the intersection of citizen science (like iNaturalist) and tactical creature RPGs (like Pokémon), we built TerraCatch: an AI-driven mobile app that turns real wildlife exploration into an immersive creature-collecting and battling experience.

What It Does

  • Instant Wildlife Capture & Identification: Players record a short video clip of an animal. An on-device computer vision pipeline detects, tracks, and classifies the species in real time—even without cell service.
  • Deterministic Lore & Battle Profiles: Every capture generates an individualized combat profile featuring 7 primary stats, 4 hidden stats, and a bespoke 4-move kit tailored to the animal's biological archetype (Avian, Predator, Grazer, Reptile, Aquatic, Insect, Arcane).
  • Turn-Based Tactical Battle Engine: Players train their captures, level them up, and deploy them into tactical matches featuring stamina management, status effects (Bleed, Camouflage, Shock, Daze), priority mechanics, and elemental advantages.
  • Ecological Rarity & Quests: Animals are assigned a dynamic Rarity Index and Tier (Common to Legendary) based on global prevalence, local seasonality, range conformity, and IUCN conservation status. Daily and weekly quests incentivize ethical, outdoor exploration.

How We Built It

TerraCatch was engineered as an end-to-end full-stack ecosystem:

1. Mobile Client (Flutter & Dart)

  • Cross-Platform UI/UX: Built with Flutter for smooth 60fps animations, intuitive camera controls, real-time bounding-box overlays, and a responsive Battle Hub.
  • Offline-First Sync: Local persistence using SQLite/shared_preferences and optimistic synchronization with Firebase Firestore.

2. Edge & Cloud Computer Vision Pipeline

  • On-Device Vision Pipeline: Utilizes quantized INT8 EfficientDet-Lite for animal bounding-box proposals, SORT-style tracking across video frames, and an EfficientNet / MobileNetV3 classification backbone.
  • Temperature Scaling & Confidence Gating: Output logits are calibrated to prevent overconfident hallucinations in the wild.
  • Geospatial & Seasonal Priors: Class probabilities are re-weighted using latitude, longitude, biome tags, and day of year.

3. Backend & Battle Simulation (Python & FastAPI on GCP)

  • Deterministic Gamification Engine: Evaluates capture XP and player level progression monotonically.

Challenges We Faced

  1. Balancing Real Wildlife Against Game Balance: In real life, an apex predator dominates smaller animals. To make every capture tactically viable, we introduced a 6-element advantage grid, speed/evasion archetypes, and strategic crowd-control moves.
  2. On-Device Mobile Inference Latency: Running neural network detection and classification on high-framerate video caused thermal throttling on older mobile devices. We solved this by running the detector every $N$ frames, interpolating with a lightweight tracker, and isolating ML inference on separate Dart workers/background threads.
  3. Open-Set Classification in Nature: Nature is full of rare species, lighting variations, and occlusions. Implementing post-hoc temperature scaling and reliability diagrams allowed us to set reliable reject thresholds and fallback pathways.
  4. Deterministic GPT Move & Stat Synthesis: When leveraging LLMs for lore generation, outputs can be unpredictable. We developed a strict JSON-schema parser, an archetype verb lexicon, and a local deterministic fallback engine that guarantees consistent, reproducible moves and stats from a fixed capture seed.

Accomplishments That We're Proud Of

  • Deep Combat Depth: A fully simulated, 11-stat turn-based battle engine that turns real animal adaptations (e.g., camouflage, venom, echolocation) into viable game strategies.
  • Safe & Ethical Wildlife Engagement: Built-in ethics guardrails that reward respectful observation distances and discourage disturbing natural habitats.

What We Learned

  • Edge ML Optimization: Practical experience in quantization-aware training (QAT), INT8 conversion, and hardware delegate acceleration (NNAPI / GPU / Core ML).
  • Game Balancing via Mathematical Modeling: Translating complex biological traits into robust, exploit-free stat formulas and turn-based mechanics.

What's Next for TerraCatch

  • Citizen Science Partnerships: Creating an anonymized, opt-in data export pipeline to contribute verified biodiversity sightings to global ecological research initiatives.

Built With

Share this project:

Updates