alt text

💡 Inspiration

The idea is to be able to explore a new level of immersion within a decision-making game using the power of the player's voice. The way the idea has been developed is with a police mystery story that little by little shows that the secrets of the past always come to light.

The story takes inspiration from the police thrillers of television but adding more elements such as science fiction, mad scientists, etc.

📕 Story.

A rookie FBI agent must solve the kidnapping case of two children in a small rural town, but what begins as a small investigation reveals darker secrets that were buried in the past.

👓 Pitch deck

You can see the pitch deck presentation here: Link of presentation

🤔What it does

The game us the voice of the user 🧏‍♂️ , as an input, to move over the story of the game, and depends of what the user says, the course of the history change.

💻 Tech Stack

  • Frontend: React Native
  • Backend: Cloud Functions , NodeJS, Google Cloud Speech API
  • DB: Firebase Firestore
  • Auth: Firebase Auth
  • NPL Engine:

🧙‍♂️ How we built it

First step was coming out with an idea of a story out of the conventional decision games. provides the necessary tools to achive this goal so we developed the API using Node JS to manage sound data requests, handle all natural process language operations from, looking for the point in the story, from our DB, and finally converting the text to speech, with Google Cloud. For the process of building a cross platform application we used React Native so the whole environment was written using JavaScript and the integration was less complicated. Next steps were finding out how to record audio from the physical device and send this data to our API so it could be processed by, and receive a response depending from the API which data comes processed as text to speech, so the reponse from the API after the voice message is processed can be reproduced by our application and players are able to listen to it.

🧱 DB Structure

Each dialog of the story, has an ID related to it. and every character, has and id, of voice configuration, for the Google Cloud API, so we have a collection for those things.

And the story moves foward, by passing the ID of the point in the story, from the API to the APP.

Collection Story

alt text

Collection Voices

alt text

🧠 NPL Flow

alt text

alt text

🦾Challenges we ran into

Firstable we had to find a way to capture voice sound because none of us had previous experience working with sound data, so after some research we found a way to accomplish this but the next complication was handling the data generated from this recording in order to send it as the data structure needed for the tool. We had no idea how to manage this raw information until we investigated and found out how to convert it into Buffer data type which was what we needed. Now we had to manage the sound data from the API response and in order to do it we found out by the experience we had converting sound data into Buffer that we had to do the reverse process and decode this data into a playable sound file.

🤓 What we learned

Building a game application was a whole new challenge for us. As we had no previous experience on the area, we learned from scratch how to handle sound files information and send it as raw data through http requests and make the reverse process of this data to convert it into playable sounds on mobile devices. We discovered the potential usage of this technology in other areas and how the proper implementation can make a really good impact in the society.

🧐What's next for Game - Castle Fortress

  • More story lines
  • Clues inside the game
  • More user interaction with new technlogies
  • iOS App

🎮Game Play


Built With

Share this project: