Neurodog

Manage your service dog
GIF
Interact with guided tutorials
Landing Page
GIF
Use natural language to navigate pages
Share your emergency profile information
Dashboard

Inspiration

For individuals suffering from conditions like mental health or physical disabilities, service animals can offer immense improvement to a person’s quality of life. Yet, tragically, many are unable to own a service animal due to the limitations imposed by their conditions.

In 2023, 74% of pet owners believed their mental health improved after getting a pet; however, less than 1% of Americans with disabilities own service dogs due to cognitive/physical incapacities that may result in animal neglect and differences in the attributes of dogs.

We began to think:

How can people with disabilities receive emotional support and physical assistance from service animals, without needing to shoulder the responsibility of caring for the animals' needs?

What it does

Designed as a self-functioning robotic service animal, Neurodog has the capability to recognize familiar brainwave patterns, including emotional states like "sadness" in its user, and is equipped to automatically execute corresponding commands based on those emotions, such as approaching the user for comforting interactions through play or engaging in entertaining tricks. Neurodog is a service dog that offers therapy and a range of essential medical services.

By merging AI with voice commands we were able to implement into Neurodog to recognize natural language voice prompts— providing a more realistic feel while extending the range of voice functionalities available.

How we built it

Frontend was built using Next.js (React) with Shadcn UI for the components. We included a Github OAuth for dashboard login, and Neurosity OAuth to gain access to the tab containing brainwave scans and functionalities.

The backend consisted of serverless functions using NextJS’s app router. This app router allowed us to develop with SSR that improved the performance of the application. We used Drizzle as the ORM and lucia-auth for the session management. Through Lucia’s documentation, we were able to spin up our oauth quickly.

In addition to those tools, we used Kirimase which bootstraps NextJS applications with all the tools above with one command.

For natural language comprehension, we used OpenAI’s whisper to understand speech. We then processed the response into ChatGPT’s function calls to allow us to derive a command and run it on the frontend or backend. This enabled us to do Voice Actions on the frontend.

With Spot, we used Elevenlabs to generate the speech and ChatGPT’s whisper to understand recorded audio. We also used Neurosity’s SDK to allow Spot to receive brain waves from our EEG.

Challenges we ran into

This project encountered a significant number of integration challenges. Here are some interesting tidbits:

Kinesis, the Motor Imagery Brain-Computer Interaction (BCI) method that allows us to trigger actions through trained brainwave patterns, needs to be consistently trained to maintain the accuracy of the brain scans and work on command.
Sharing a singular Boston Dynamic SPOT dog between numerous teams, significantly reduced the time we had to execute our programs.
Figuring out the Neurosity SDK, and connecting it with the SPOT SDK proved to be difficult because of the complex intricacies in both documentations.

What's next for Neurodog

Unlike competitor products such as Tombot which is currently only touch-activated and real live dogs, Neurodog will interact and safeguard users through voice and based on their brain activity using an electroencephalogram (EEG) allowing immediate care reliably.
Neurodog will include precautionary SOS features to ensure it is protecting its user in case of emergencies
We plan to target households, hospitals, and nursing homes to assist with caretaking.
Neurodog will have computer vision capabilities so it will act as the eyes for its user. Neurodog will be able to detect these dangerous situations and analyze traffic signals allowing them to alert the user while protecting them.