Morning buddy

Inspiration

One part of what inspired this group of Hackers, who have known each other for a while, was a topic that popped up quite often: Forgetfulness. There's a common subtle sense of fear of the brain aging, maybe sooner than it should. One of the team members had a genuine freak-out about early memory issues the long-term cognitive health, becoming hyper-aware of it. They came across this video while doom scrolling on Instagram which sparked the research from there: 3 Brain Activation Exercises for Beginners Another part of the inspiration came from noticing how easy it is to rely on AI in daily life for things that used to challenge our minds. Stopping to push our own thinking stops the support of neuroplasticity, what our brain needs to stay sharp. "Morning Buddy" came from this reminder of why daily, small mental challenges still matter, and that cognitive fitness is just as important as physical fitness and that it's something you train and maintain.

What it does

In this mixed reality experience, you begin by settling into your usual morning routine. You place your coffee cup in front of you, and wherever you set it on the table, a small Buddy emerges beside it, eager for a sip, but only once he reaches the top of the mug.

Your goal is to complete a sequence of gestural tasks. They may look simple but performing them accurately and consistently within a set time limit is surprisingly challenging. When you succeed, the little Buddy finally earns his sip of coffee, and you get your well-deserved first sip too.

How we built it

For this application's logic, we utilized:

Unity + Meta XR SDK for hand tracking, and scene understanding +Hand gesture classifiers for sequences like peace sign, palm open, rock on, and index up.

For this application's narrative virtual layer:

We designed the character "Buddy", he's brainy fella.
Modeled "Buddy" using hyper3d -Rigged and animated "Buddy" using Mixamo
We modeled on Maya the stairs that "Buddy" uses to climb us the coffee cup.
We made timeline sequences in unity for the intro/outro.
We made voiceovers using elevenlabs

Challenges we ran into

During the hackathon, we faced the challenge of implementing cutting-edge technology with components still in beta (Meta AIO SDK), with the incompatibility issues between packages and versions that this entails.

For some of us it was the first time doing a hackathon so time was running by so quickly, we had to keep the scope of the project within clear limits and not get carried away by the chaos of this type of environment, analyzing and prioritizing all the situations we encountered, such as wasting too much time implementing or improving components that do not have a primary impact on the experience, or building stable checkpoints to which we could return in case of a major problem.

Accomplishments that we're proud of

The members of this team for years have been primarily focused on building VR experiences with a specific SDK XROAM so we’re quite proud to create a mixed reality experience in just two days using building blocks we hadn’t worked with extensively and not as familiar with the recent updates.
This was the first hackathon for 4 people out of 5 in this team, so being extra agile was a great exercise. A lot of creative solutions came from this while keeping focus on the main goal, the core mechanic.

What's next for Morning buddy

Include context-aware behavior: Tea/Coffee/Milk/Juice, etc..... + Text-To-Speech for the character (elevenlabs)
Bigger/ wider library of gestural tasks.
Personalization: Adaptive to your progress with time / your weak points.
Add difficulty levels: Play with time frame