Inspiration
The last time Danial called 911 was because he was in a car accident. Despite being involved in the crash, he never got through to 911 because the line was presumably flooded by bystanders also calling to report the accident.
What it does
The Incident Reporter is an AI that asks a 911 caller for details about car crash that just occurred. It asks questions such as if the caller is involved, how many cars are involved, if there is any fire, and if the caller is injured. All the information that the caller provides is stored in a database.
How we built it
We used Assembly AI’s speech recognition API to transcribe user’s speech in real time, including multiple language support for accessibility. We then process it with our own algorithms in order to determine the artificial intelligence’s response, or we choose for open AI’s completion algorithm to respond. We get location via the google API to imitate how 911 traces callers. We hosted a server on a flask backend, which goes to the tech domain we registered with Domain.com. We then pass the information from the artificial intelligence to the database. We query this priority queue to display which calls are the most important/relevant to the 911 operator, with a frontend in html/javascript/css using bootstrap.
Challenges we ran into
Making sure the separate files can interact, the structure of our data so it can be analyzed and scored but also fit into our database. Two people working on the same file leading to a github merging issue. Solving parallel processing issues and displaying the information on a website.
Accomplishments that we're proud of
Firstly, working together so well! Who knew that four students from completely different backgrounds and universities who just met on the day of could work so well together?
Secondly, we’re super proud of the added bells and whistles we have on this project, including the multiple language support.
What we learned
Coming up with the steps of the project is fundamental to the process, it allows every teammate to know what tasks they can do and help the project run. The process also showed a lot about the process of transcribing audio.
What's next for Incident Report
The next steps would be to expand it to record more incidents than just car crashes and to have the ai interview a caller while they’re on hold. To perfect the questioning and add more useful factors for importance/relevance, we could train an ai to make a question pool and scoring system.
Log in or sign up for Devpost to join the conversation.