Manu

Manu: The Last Crossing

Inspiration

Our inspiration came from the Hindu story of Manu and Matsya, one of the earliest stories of survival through a great flood.

In the story, Manu saves a small fish that eventually reveals itself as Matsya, an avatar of Vishnu. When a catastrophic flood approaches, Matsya warns Manu to build a boat to protect his family, the seven sages, and the Vedas, preserving life and knowledge for the future.

We wanted to reimagine that story in a modern context. Today, floods continue to isolate communities, destroy infrastructure, and cut people off from essential services. But what interested us most wasn't just the disaster itself; it was the decisions people have to make when resources are limited and every second matters.

What if, instead of watching a disaster unfold on a screen, you could physically change its outcome?

That's how Manu began.

What It Does

Manu is a tangible, AI-powered disaster-response strategy game that transforms a physical map into an interactive, living simulation.

Using a 24 × 18-inch map of Nepal, an overhead camera, and a projector, players take on the role of Manu, protecting communities from rising floodwaters.

As animated water spreads across the map, endangered communities pulse red and disconnected routes become visible. Players place physical bridges, hospitals, and food stations directly onto the landscape.

Our computer vision system recognizes these objects, while the simulation evaluates their impact on road connectivity, access to essential services, and community needs. When an intervention succeeds, routes illuminate green and the map responds to the player's actions.

We also introduced Matsya's Guidance, a golden fish companion that helps players discover strategic placements using recommendations from our simulation.

The experience culminates in a before-and-after impact reveal, showing how the player's choices changed the simulated outcome compared to taking no action.

Rather than simply rewarding players for placing objects, Manu challenges them to understand why those decisions matter.

How We Built It

We combined computer vision, physical hardware, simulation, and interactive storytelling into one experience.

OpenCV handles camera input, ArUco marker detection, board calibration, and tracking physical objects as they're placed, moved, or removed.
Perceptron powers AI-based classification of physical pieces, helping the system distinguish bridges, hospitals, and food stations.
Python + FastAPI power the backend simulation, evaluating infrastructure changes, road connectivity, service accessibility, capacity, and unmet community needs.
React, TypeScript, and SVG power the projected interface, including animated floodwaters, danger zones, accessible routes, mission prompts, and responsive visual effects.
Projection mapping connects the digital simulation to the physical Nepal map, making the tabletop itself the game interface.

One of our biggest design goals was making sure the visuals weren't just decorative. A glowing green route should mean something. Our projection is tied to backend-confirmed world state, so the game reflects the consequences of actual interventions rather than predetermined success animations.

Challenges We Faced

Our biggest challenge was getting the physical and digital worlds to agree.

Detecting an object is one thing. Understanding exactly where it's placed on a physical map, identifying what it is, tracking when it moves, and determining whether it actually improves the simulated situation is a much harder problem.

Projection mapping introduced another layer of complexity. We had to account for camera angles, map alignment, object occlusion, and projected lighting potentially interfering with computer vision.

We also had to balance technical accuracy with gameplay. A realistic simulation can quickly become overwhelming, especially when someone has only 75 seconds to understand the situation and make meaningful decisions.

We approached this by creating a five-chapter experience inspired by Manu's journey, with clear objectives, recognizable physical pieces, animated feedback, and contextual guidance.

What We Learned

This project pushed us to think beyond traditional software interfaces.

We learned how challenging it is to connect computer vision with real-time physical interaction, especially when a system needs to recognize not just what an object is, but how its placement changes an environment.

We also learned that good design is essential to communicating complex technical systems. Animated floodwaters, glowing danger zones, and changing routes aren't just visual effects; they help people intuitively understand accessibility, risk, and the consequences of their decisions.

Most importantly, we discovered how powerful it can be to combine cultural storytelling with emerging technology. The story of Manu and Matsya gave our simulation an emotional purpose beyond simply optimizing a score. What's Next

We want to continue improving physical tracking and projector calibration, expand the range of disaster scenarios, and explore how Manu could be used in classrooms, museums, and community preparedness workshops.

Our long-term vision is to make disaster resilience more approachable through interactive experiences that encourage experimentation, strategic thinking, and empathy.

Thousands of years ago, Manu was guided through the flood. With Manu, we wanted to put that responsibility—and the power to make a difference—into the player's hands.