Sparrow

• 

  Menu

• 

  Choose

Inspiration

Our journey with Sparrow began when we noticed a common issue: many language learners focus solely on vocabulary memorization, losing sight of real-world application. We sought a solution that would reignite motivation by immersing users in authentic language experiences. Sparrow emerged from this need—a platform that transports users to foreign environments, fostering practical language use and meaningful learning.

What it does

Sparrow offers users a unique language learning experience by providing small lessons similar to those on other platforms but in an open-world format. Users can explore neighboring locations while practicing language skills, creating an effective educational journey that blends immersive exploration with practical language application. Through advanced voice-to-text recognition technology and ongoing conversations with trained AI agents, Sparrow enables users to have real-time, natural conversations as they navigate the immersive world.

How we built it

We built Sparrow using Unity as our main game engine, with Python and C Sharp for backend development. We also Postman to test our API calls. We sculpted, modeled, rigged, and animated assets in Blender. We integrated advanced voice-to-text recognition and AI capabilities for a seamless language learning experience. Google's Gemini AI and OpenAI handled natural language processing and dialogue generation, while Fetch.AI provided intelligent NPC behavior and the use of agents to represent these NPCs. Our design process focused on iterative prototyping and a user-focused product, ensuring Sparrow's edge cases and efficiency.

Challenges we ran into

We faced some tough challenges while building Sparrow. Using AI agents as NPCs that fit the environment was trickier than we expected. Also, getting the hang of different APIs like Gemini and making the most of Fetch.AI took some serious learning. Mapping and animating scenes and assets were extremely intimidating at first, but together we found work-around solutions. Putting together all of the components was also extremely difficult since multiple people were tasked with different parts of the project. Also, the wifi at the venue was super unreliable and was a roadblock worth considering.

Accomplishments that we're proud of

Many of us were new to Unity and game development, but we tackled the challenge with enthusiasm. It was a refreshing learning experience, diving into new technologies and gaining insights from sponsors and mentors. Honestly, we're just proud to have tried our best till the end.

What we learned

Hmmmmmmmm. One thing that was a huge lesson for us was to definitely seek professional help. We spent hours trying to configure fetch.ai and found mentors that guided us in a great direction. Most importantly, we learned to never underestimate game developers and their craziness for doing this as a career.

What's next for Sparrow

For our future plans for Sparrow, we hope to continue the true immersive experiences with fields in VR and AR to truly emulate the real environment. We hope to also add a lot of cultural educational factors that could help improve the authenticity of learning the language, and fix graphics and bugs, ofc.

Built With

• blender
• c#
• deltav
• fetch.ai
• google-gemini
• openai
• postman
• python
• unity