Inspiration
I was inspired when I read about how AI was being used to help students without access to teachers have a personalized education. Connecting this to blind or visually impaired people who don't have access to service dogs, I combined my idea with a 3blue1brown video I watched recently about image detection and object separation. I came up with a goal to harness AI and object detection while making an algorithm that could be used to improve the lives of others.
What it does
ASD recognizes objects across frames, tracking movement. It detects when objects are moving fast or growing larger (approaching.) The fast-moving object is to prevent users from stepping on busy roads or into crowds while the approaching object helps prevent things like walking into walls or trees, without the burden of a cane. ASD can be ran on a phone or similar device, then held out, attached to a belt, to clothing, or to a headband.
How we built it
I used Python, specifically OpenCV and Flask, as well as HTML and CSS. I do not have much experience with Flask or OpenCV in general, so it was a new experience. Anyways, the program works like this: When the movement of a specific object over a threshold size is going much faster than normal, ASD plays a audio warning. Then, to detect when the camera goes closer to another object, we track the color variance through the frame. If it is less than a threshold, we warn the user with audio too.
Challenges we ran into
Coming up with an idea was a larger challenge than expected, and tight timing was an issue as well. However, coding and debugging the actual program was tough without much prior experience. For example, one problem I encountered was too many audio warnings from ASD when it would quickly switch from “Object fills up screen” to “Object doesn’t fill up screen”, back and forth. To fix this, I put a timer so that a brief flash of “Object fills up screen” wouldn’t trigger the warning. (However, I of course excluded this timer from the fast-moving object cue because fast objects would indeed only be in the frame for fractions of seconds.)
Accomplishments that we're proud of
I was very proud when I actually got the object detection to work, and seeing the green squares move around as I moved was awesome. Applying Gaussian blur successfully was also a highlight, as well as every time I finally figured out how to prevent a bug. Overall, I'm happy with how ASD came out for having been coded from scratch in 2 days!
What we learned
I definitely learned better familiarity with Python tools and OpenCV. But I also now know that time management is very important (I pretty much spent the first day just brainstorming) and I hope to do better with that in the future. Also, on the more technical side, I was able to learn more in-detail how AI can tell what's in an image. It's pretty fascinating.
What's next for AI Service Dog
Next, I hope to add text-to-speech for website accessibility, add CSS to a website design, and of course actually host a website with my ASD implemented within it- a toolbox of sorts for the visually impaired. I'm also trying to track object sizes in my program to get more accurate "approaching object" reads, without the objects having to be mostly one color.
Log in or sign up for Devpost to join the conversation.