David always recognising as two people
First example of working object detection
Metro Train detection
Prior to Unihack, the team identified many existing problems and settled on one, which each member was able to relate to. This problem was the difficulty in finding vacant study spaces during study hours, often forcing students to continue searching for one. This wastes valuable study time. After a brainstorming session, the team decided to create a solution using computer vision and machine learning to detect vacant study spaces with just a basic webcam. As the team continued discussing the potential of the solution, the team realised the plethora of benefits for object detection, anywhere from space available in buildings (lecture theatres, study spaces etc) and public transport (train, trams, bus etc) to complex object recognition such as lost or possibly hazardous items in public areas. This drove the team to create SeatMe.
What it does
SeatMe detects the capacity of a venue and how full it is. SeatMe does this by first taking an image of an empty scene such as a class room or a train carriage, and sending it to Accenture's smart vision API to determine the number of chairs. Images will be taken throughout the day in intervals and processed using Smart Vision to determine the fullness of a venue, providing end users this information in real time.
Metro Carriage Example
Study Space Example
How we built it
Challenges we ran into
- Detecting people, the computer vision API counted David's shoes as an extra person.
- Trying out 3 different Material UI libraries on react before Erfan saved the day.
- Transitionally cleanly between colours in custom made svgs.
- Changing the backend from Js to TypeScript 8 hours in.
- David always detecting as two people
Accomplishments that we're proud of
- Built a fully functioning prototype in 24 hours.
- Built a solid backend which was extensible to two different use cases (and can be extended for more)
- Using new technologies such as a computer vision API, mobx, leaflet.
What we learned
- Feedback is extremely valuable. Everyone we spoke to and presented our core idea to, had their own unique ideas and insights to innovate SeatMe. These ideas drastically changed how we approached and iterated on the project.
- Pick our stack and stick to it
- Sleep soothes the soul...
- Ask for help and utilise the mentors (Accenture reconfigured their API on several occasions for us. Including phone calls to colleagues overseas crossing the border for US to Canada!)
- yarn > npm
What's next for SeatMe
As the machine learning model is trained and the classifications become more accurate and descriptive, SeatMe can expand its api to:
- Determine your demographic via capturing age and gender, allowing statistics about who is using your using your product. This will help in refining your target audience and assist in you in advertising efforts.
- Refined object detection to detect objects in the scene. This has profound benefits in a security landscape, for example, detecting lone packages at a train station to avoid bomb scares or help with lost property.
- Complex positional awareness to accurately determine where the object is in real world space.