Edgelord Samurai

A Single Player, Educational Samurai Mixed Reality Fruit Ninja-style experience with added Passthrough Camera Access and Computer Vision capabilities for Social Party Play mode with VR Katana prop.

Comment

Inspiration

Edgelord Samurai is a simple single player bamboo slicing experience in Mixed Reality with an immersive storyline. Originally the concept is inspired by when I was training with my sensei (teacher) in his backyard and after doing tameshigiri (test cutting), we would take turns throwing the remaining tatami mats (igusa straw) at each swordsman, providing a fun and useful moving practice target to test our sword skills on.

Edgelord Samurai's singe player mode follows a young son, grieving the loss of his father who was killed in the war, as he is recruited for Samurai training by his sensei (teacher) to prepare for the growing conflict. After training in single player mode, players are encouraged to play with friends or family in the real world in Party Mode to test out their skills or to impress by having one person throwing a colored foam ball that is tracked in the virtual world, towards the player, allowing the player in Mixed Reality to cut from the different angles, testing their sword cutting skills.

Additionally, players can use a custom created VR Katana prop for a more immersive, realistic experience. This way, players can test their real life sword cutting skills and their progress in a fun and safe environment, without the need of wielding a real sword. With Japanese swordsmanship training where precise cuts matter, the realism of the experience is tested and approved by a high ranking and skilled swordsman which is combined with the instant fun inspired by games like Fruit Ninja and the social chaos of couch party games, ending in an exciting and realistic fun experience for all. The result is a project that blends serious dojo focus with friends laughing, throwing a foam ball at each other, and calling out for one more cut!

What it does

Edgelord Samurai turns any room into a shared mixed reality dojo on Meta Quest. The player wears the headset and holds a katana style controller. A friend throws a giant orange foam ball through the play space. Our system uses OpenCV color tracking on the passthrough video to lock onto the ball and follow its movement in real time.

In the headset, the ball becomes the anchor point for virtual fruit, targets, and impact effects that align with the real world motion. When the player swings the katana through that space, they see satisfying slices, particles, and scoring feedback that feel tied directly to the physical throw. Multiple Meta Quest headsets can join the same session so other players and spectators see the same dojo, the same ball path, and the same hits from different viewpoints.

How we built it

We built Edgelord Samurai in Unity on Meta Quest using passthrough mixed reality and OpenCV. The headset shows the real room, and we layer a stylized dojo environment over it. We use a bright orange foam ball as a physical training orb and feed the passthrough frames into OpenCV.

By defining minimum and maximum values for the orange color, OpenCV isolates the ball from the rest of the scene and calculates its position in each frame. Unity receives that position and attaches virtual targets, trails, and hit effects to it so the physical ball and the digital effects feel like one object. The katana controller is treated as a training sword. The game checks when its swing path intersects the tracked ball and triggers slices, sound, and when the timing and placement are correct. On top of this, we allowed the player to choose between a single player dojo mode and a party mode with a thrower and a cutter.

Challenges we ran into

We discovered that tracking a real object in mixed reality is very sensitive to the environment. Different lighting, wall colors, and clothing made it harder for OpenCV to cleanly isolate the orange ball. We spent time tuning the color ranges, smoothing detection, and reducing jitter, so the ball would stay stable on screen.

We also had to balance responsiveness and reliability. If tracking reacted too fast, it created noisy movement. If it was too strict, the ball would occasionally pop in and out of view. Finally, we had practical challenges around safety and hardware, such as designing a katana prop that felt good to swing while keeping enough distance between the thrower, the player, and the surrounding furniture.

Accomplishments that we're proud of

Creating a realistic sword cutting experience that can be combined with computer vision tracking on the Quest headset that hasn't been possible before. Creating new and innovative experiences, pushing the boundaries of what is possible in Mixed Reality with Computer Vision and creating realistic immersive experiences with a VR Katana prop, adding an extra layer of immersion and realism to the experience, allowing the player to truly feel like a samurai in Mixed Reality and in a safe environment that can be played anywhere, on the go or at home.

What we learned

We learned that mixed reality sits at the intersection of computer vision, game design, and physical space. Simple tools like color tracking can create powerful illusions when combined with thoughtful feedback and real objects. At the same time, MR forced us to pay close attention to safety, comfort, and clarity because players are moving their bodies in real rooms, not just using thumb sticks.

We also learned that social presence multiplies engagement. The moment we added spectators, and turn taking, the project felt more alive. Finally, we learned how important it is to design within real constraints like noisy environments and limited hackathon time, and to prioritize features that work reliably in front of judges and players.

What's next for Edgelord Samurai

Make this work with Mixed Reality Shared Spaces so that multiple people can view the experience from different angles, instead of just experiencing from the person's First Person view. Can also incorporate the idea of Hyperscape environment combined with FBT (Full-body Tracking) to show the player from a third person perspective as a FBT avatar in sudo-MR.

Spend more time improving gameplay such as adding a score and edge alignment feedback to make the gameplay more competitive and give more feedback to the user of how well their technique is improving. Also, adding a storyline progression mode so that the player can improve their technique to show their strength in battle, following the single player storyline idea.

We would like to focus on hardware, for instance, with VR Katana prototype already built, so that the software can optionally be packaged with the hardware and mass marketed/sold as a package to kids and adults, as a fun social/educational/fitness experience and mass marketable.

Built With

Share this project:

Updates