As individuals and communities continue to work tirelessly in keeping each other safe throughout the pandemic, we must continue innovating with technology so we can go back to doing the things we love, safely.
What it does
adiona is an all in one system that can be adapted to a diverse range of contexts including a hygiene station, access control method as well as a security management system. By extension, the data gathered from these stations can reveal important insights on user behavior for managers in industries such as healthcare, entertainment and education.
How we built it
Our team is made up of a multidisciplinary set of skills and experience that has enabled us to bring this solution to life. In 48 hours, we have combined software and hardware to create something we're truly proud of from the ground up.
The front end was designed in Figma and implemented with React. The mask detection system used machine learning, with Python, OpenCV and TensorFlow to detect if the user was wearing a mask and whether they were using it properly. We used socket.io as a communication tool for the backend system which was made up of Express.js, Node.js and Twilio.
The main hardware platform was the Microsoft Surface tablet, and the ESP32 and Arduino Mega microcontrollers, along with 3D printed components. The interface consisted of various sensors, including ultrasonics, and DC and stepper motors connected through IoT to the front end, via websockets.
Challenges we ran into
Abiding by council regulations especially in regards to noise control! We left our drilling and other noisy assembly tasks to a very unfortunate time and may have annoyed some of the surrounding neighbours. This ultimately delayed our video production as we needed the product to be assembled before taking footage.
3D presented a set of problems that were sometimes completely out of our control as the products printed aren't guaranteed to look and feel exactly as we modeled it.
While the Twilio API vastly simplified the implementation of streaming video between the kiosk and admin interfaces, we ran into a few issues with the SDK. Between the documentation and a massive helping hand from Phil the Twilio mentor, we were able to diagnose the issue as having duplicate user IDs. This allowed us to implement a complex feature we had previously ruled out as infeasible.
Communication between the hardware platform and the front-end was difficult as there were many differing ideas as to how the components would interact and send messages. In the end, we settled on a bi-directional approach using websockets to send messages from the ESP32 to the server which controlled the front end. Similarly, websockets were also used to communicate between the machine learning platform and the front end.
Accomplishments that we're proud of
The machine learning system used was quite complicated however Kush was able to get it made and working quite seamlessly and in a well-optimised manner. Adding nose detection to verify whether the mask was being worn properly was one of the more ingenious aspects of the machine learning UI. Allowing the front end to remotely control the IoT platform to be able to open gates, dispense masks and pump hand sanitiser was a great accomplishment since it allowed us to decouple the implementation of the different aspects of the Adiona device, allowing us to iterate quickly and delegate different development tasks to different people. This also allows for easy iteration in terms of the modular components of the Adiona device. We quickly implemented a UI prototype using Figma that allowed for quick iteration of the user experience. We then rapidly translated the design into a React interface thanks to Figma’s advanced export tooling. Creating large, sophisticated, and inter-relating assemblies in CAD, rendering them and then animating.
What we learned
- Planning is integral to a good project. We owe much of our successes to the valuable time we spent discussing each aspect of the project before diving into implementation.
- Hardware is incredibly unpredictable and needs to be started on from the very beginning
- Like every aspect of designing, CADing is an arduous and iterative process, and that is why it is essential to adopt best practices to maximize efficiency.
What's next for adiona
We hope to improve the efficiency of the electronic control systems and we plan to begin interviewing key stakeholders and users to design a more user-centric product.
We envision the Adiona platform as allowing modularity in development. By decoupling the different components from each other, the Adiona platform could be used to scan barcodes/QR codes for ticket check in rather than ID scanning. Additionally, the computer vision could be used to detect for coloured wristbands or similar to enforce flow control.
Similarly, integration with employee scheduling systems such as Deputy could allow for Adiona to also double as a way for employees to safely clock on to work.