JaydedEcho

You found it
you got hit
GameOver
warning
Game over
You found it
looking

Inspiration

I was supposed to attend this hackathon with my friend Jayde, but they somehow did not get in. Since we were planning on teaming up, I decided to go solo and create their echo to guide me through social interactions, help me avoid people and find my lost items. No AI can replace them, but it is fun to have to live without seeing anything and be guided by introvert person.

What It Does

You lost your sight and you are relying on Jayde's echo telling you where to go. The catch is that Jayde hates people, so you will die after looking at 3 people.

How I Built It

Using Ollama/LLaVA, the user will start the game by telling the program which item they are looking for. The backend is python interacting with LLMs.

Challenges I Ran Into

Trying to solve the issue of streaming live video and getting real time response from the backend. This was a headache for me and in the end I improved this by lowering image quality and using different factors like how confident the model is. I spent the whole day testing different settings and letting others test the game to get the perfect settings for it to work with less latency.

Accomplishments That I Am Proud Of

Solving the problem of real time video streaming with accurate object classification. I ran into this issue a few years ago while working on a drone project, but this time I managed to solve it which I am really proud of.

What I Learned

User will not notice the reduced quality if you do not mention it lol

What's Next For JaydedEcho

Making it a VR game to keep the player from seeing things but also keep them safe from hitting objects, adding the ability to find hearts to refill their health, and more!