Voice Verse

Inspiration

The idea of having an immersive experience when listening to a book, and a change from the old monotonous tone of reading, is what inspired our team to develop character-specific voices at the user's will.

What it does

The app adds an immersive feature of converting text to Audio in a way never done before. It uses different voices for different characters, which is up to the user's discretion. It allows the user to upload books of their choice and choose a voice from the many available options for characters (prompted when the book is started).

How we built it

• Overall, we used languages such as TypeScript and CSS for front-end and back-end dev, as well as using VS Code IDE and React Native for developing the mobile application. We also used API integration for aspects such as DeepSeek (credits used for context in pages for who the characters are and what they say- used for audio voice purposes).
• For front-end, we developed on React and ExpoGo (during the development stage and part of testing). We used CSS for images and icon sizes, as well as using TypeScript for user-app interactions and workflow between panels, screens, or tabs.
• For the back-end, we first came up with a prompt that had proper parameters to handle the test case. Using the API, there are some edge cases, but for a demo, it is fine. We then coded using the API, and got it to identify which character is talking, then we worked towards integrating this with the Fish Audio, and using Fish Audio's API together to give each of the characters a voice in our mobile app.
  ## Challenges we ran into
• We struggled to get the right prompt and had to prompt the engineer for a good prompt before successfully doing it for our DeepSeek API.
• At the end of the hackathon, we realised that Expo CLI and Expo Go do not support Audio file downloading, which led to having code and no way to execute the program. The Mac development wall prevented the use of many mobile app dev apps, leading to execution of code becoming a hassle (especially when time is running out). ## Accomplishments that we're proud of
• We did manage to integrate DeepSeek LLM API into the app to make it read a page, take into context the dialogues and who says it, and create characters' list and dialogues.
• We started a couple of hours late and still managed to do all that we wanted to do with the project, along with some features in the front and back-end. ## What we learned
• Learned to implement AI into mobile app development - a useful feature for the hard task we wanted to do (learned prompting AI specifically to perform the required task - training).
• Importing API such as FishAudio and using them in voice for characters. ## What's next for Voice Verse
• Dealing with our edge cases, because as we expand our book size, the margin in which the name of the character talking would increase.

Built With

• deepseek
• fishaudio
• react.js
• typescript