VANCE - IOT Integrated AI Assistant

With state-of-the-art language processing, VANCE is your personal AI to be at your side, write code for you, and answer any question you've ever had.

Comment

Inspiration

To improve every day productivity, to provide a source of reliable and immediate information, and to be readily available for all walks of life. This app is targeted to help the people who don't have reliable access to educational texts or resources, to the people who want to add more productivity into their lives, and to flex the current abilities of Artificial Intelligence.

What it does

The app takes voice input from the user, processes it through various APIs and AIs to create relevant output and speaks back to the user with useful information, and potentially code that the user asks for. It can also control an arduino and adjust smart home devices.

How we built it

We built this application by utilizing the built in voice recognition and text to speech software in android using react-native, and integrating it with data scraped from chatgpt to enable a complete AI voice assistant experience. Additionally, this AI also has custom commands that allow it to send and receive messages from various IOT devices such as connected laptops or lights, and also provide the user with up-to-date live information such as weather.

Challenges we ran into

The biggest challenge we faced when creating this application was the lack of an official chatgpt api, as a result we had to use various workarounds and web scrapping using puppeteer to successfully send and receive data from chatgpt.

Accomplishments that we're proud of

  • We were able to complete all the milestones we set for the project within the hackathon time limit.
  • We were able to extensively test it various realistic scenarios successfully. These scenarios include asking information about a specific topic, asking for code for a specific problem(eg. Fizz Buzz) and copy pasting the code to any device connected to the network, interfacing with home devices such as turning on/off lights.
  • We were also able to make intuitive interface so that future developers can easily integrate their applications into our system

What we learned

We learned about the value of a well-organized plan to tackle various different parts of the code, and the communication and teamwork required to co-develop a project of this scale.

What's next for VANCE - IOT Integrated AI Assistant

  1. NLP to generalize custom commands rather than having hard coded phrases
  2. We want to be able to support more complex scenarios, reduce latency for responses from the bot.
  3. Add in ability to receive code from connected devices
  4. Improve the UI/UX design to be more familiar and easily usable.
Share this project:

Updates