RecoverAI

An AI that can help diagnose an injury and provide at home care and attention, all while saving you thousands of dollars in the ER

Comment

Inspiration

A few weeks ago, my mom had to go to the ER with cardiac problems. She had been complaining the entire day about jaw pain and arm pain, and later that night, she ended up having to go to the ER. She was told she had three arterial blockages in her heart and was one step away from a heart attack. While we thankfully caught my mom's problems in time, a lot of people don't. What most people just rule out as arm pain can in fact turn out to be one of the leading symptoms of a heart attack. And that is why I wanted to build something like RecoverAI, where you don't need to be in doubt whether your arm pain is from playing too much pickleball or the knock of a heart attack. So many people have scrolled and searched for hours on Google what their symptoms mean, going through years of medical research to figure out what hurts them. Now, it's much easier, and all in one place.

What it does

It takes in your injury by either voice or text, and from that, you can get a few results; the assessment result, the 4 week recovery plan of exercises you can do, a progress tracker, a red flag checker to see what the threat level of your injury is and if you need to go to the ER or can manage at home, and lastly a movement tracker than tracks the angles of a few of your joints to see how your range of motion is affected. All of this is then read aloud to you as well, in addition to having the text on screen, to ensure the maximum amount of people can access this vital information, even if they are blind or deaf, or otherwise disabled.

How I built it

I built the backend using Node, with the key feature being the three Claude agents that classify the injury, check if there is an emergency, and make the recovery plan. Then, I made the text-to-speech and speech-to-text transcription using Deepgram, which ended up working really well after I fixed everything up. I used Sentry to find any bugs that I had in the code, which was a lifesaver at 3 a.m. Lastly, I deployed using Railway to make the site live.

Challenges I ran into

The speech part was hard to figure out. I went through a few different versions of it. I started out with just having text, then switched it to just speech, and then decided to have both speech and text in the final product. The shift between having one, completely pivoting, and then deciding to implement both was challenging because I had to basically turn around every time I thought I was finished. Another big problem was trying to get the Claude agents to work. I spent nearly an hour trying to figure out why they weren't working, only to realize that it was because I was using an older version of Claude, and that's why. Another hurdle was when I tried to deploy the site, but I kept running into issues. For some reason, my Redis wasn't properly tied to the project in Railway, and no matter what I did, it didn't work. I ended up having to unfortunately axe Redis from my final version, and use it without a cache. But if I had to pick one, I would say that the speech and text hurdle was the hardest, because I had to shift my code around fully every time, and mentally, every time I was done, it felt like I had to go back to ground 0.

Accomplishments that I'm proud of

I'm most proud of the fact that I managed to turn it into a voice feature. When I thought of this project, it hadn't really occured to me that I could use a speech feature in it. When I actually went in to use it, it was much harder than i thought and expected it to be. This was my first time working with a speech feature in any way, and it was much more different than I thought it would be. The fact that I got it running and that it actually works how it was intended is something I'm really proud of.

What I learned

I would say the biggest thing I learnt was how to incorporate a voice and speech feature. It was my first time working with any sort of speech feature, and I really liked how it turned out.

What's next for RecoverAI

I want to turn this into a proper tool that people can use in the real world. Right now, RecoverAI can't be used because of the many concerns with patient confidentiality. For the hackathon it isn't that big of an issue (there's no actual patients, and there aren't any real injuries). However, if I want to take this to the next stage, I would need to add a user authentication system and data encryption to ensure that HIPAA standards are met. And that's the lower end. Eventually, I want to integrate this with telehealth, and link this so that once people get their diagnosis, they can proceed with booking their appointment, right from the website/app itself, and don't have to go to another place to do that.

Built With

Share this project:

Updates