Inspiration

We were inspired to use hardware to explore the possibility of entering into a new world, determined to show the user an engaging experience.

What it does

You control a Cutebot with a handheld controller, which holds a phone to see the buggy's surroundings. The phone's camera feed gets fed into a laptop and at this point an API call is made to Decart.ai, transforming the feed into a new world. If music is playing at the same time, a call is made to Gemini to determine what kind of world should be shown.

How we built it

We used a python localhost and used a websocket to connect the phone's feed.

Challenges we ran into

  • Being stuck with one micro:bit
  • Me (Jonathan) not having any internet connection on my laptop and having to hotspot
  • Shaky connection between the phone and laptop
  • Running out of API credits for Gemini and Decart

Accomplishments that we're proud of

Getting it all working

What we learned

How to connect a phone's feed to a laptop and transform it in real-time

What's next for World Overlay Mapper

  • Make the world interactive through the laptop
  • Pre-rendering - This would reduce the amount of calls we would have to make to Decart, but would require more powerful hardware

Built With

Share this project:

Updates

Submission history