Inspiration

We are surrounded by lots of "things" on a daily basis. Ordinary things like mugs, plants, or bottles. Yet we are still bored because those items are merely everyday objects. So I thought, "Why not make a scavenger hunt using those items?" And so I did. I made NestCraft, which not only allows you to partake in an IRL scavenger hunt using those items, but also decorates themed birdhouses with the things that you find.

What it does

NestCraft features a collection of (currently 3) themed birdhouses. These birdhouses are missing some items. The user can take (or upload) pictures using their smartphone, and then the app uses Gemini to analyze the photo for any of the missing items. If any are found, they are added into the birdhouse. It also uses ElevenLabs to perform TTS on Gemini's text-based feedback on the images.

How I built it

I used Gemini a lot for this project. I started with Google AI Studio, but quickly switched over to Gemini in Android Studio. I also had Gemini generate the images that were used, and I even used regular chat-based Gemini to write a script for the demo video. However, I did still have to do a significant bit of manual editing as well (especially for the demo video script). I also used ElevenLabs for the narration in my demo video.

Challenges I ran into

Generating images was quite challenging. I had a lot of trouble getting Gemini to generate me images of items that fit a consistent theme and had the right angle and lighting to fit into the scene. I had to get clever with my prompts, and I also had to use other tools, such as Inkscape or the Magic Eraser tool in Google Photos, to get some of the images.

Accomplishments that I'm proud of

This was my second Hackathon I went to solo, and I was also only given 24 hours (the other Hackathon I went to solo was a 48-hour Hackathon). Although I may not have achieved much, I am proud of what I was able to accomplish in the 24 hours of working alone that I was given.

What I learned

I learned the importance of not being too ambitious. I knew when I had to compromise, and that allowed me to create a functioning application, rather than scrambling to finish stuff at the last minute and ultimately still having to compromise on my ambitions due to time constraints.

What's next for NestCraft

I had some ambitious ideas for this project that I was never able to implement. For example, a feature that uses Gemini's image generation capabilities to let users create their own birdhouses. Or, varying the colors or styles of the items that get added into the birdhouses based on the color or style of the IRL item. A simple next step would be to just add more birdhouses, but a larger next step would be to implement some of these features that I had to compromise on due to time constraints.

Share this project:

Updates

Submission history