Inspiration
Many older adults want to manage their medication independently, but missed doses can create stress for both patients and their families. Existing solutions often rely on the patient manually confirming that they took their medication, which may not always be reliable.
We wanted to build a system that could recognize when medication was removed without requiring the patient to press a button or open an app. Our goal was to support independence while still giving caregivers reassurance that a dose had been taken.
What it does
Pill Check monitors a medication area using an ESP32-CAM. The camera regularly captures images, which are processed by a Python computer-vision program.
By comparing recent frames, the system detects movement around the medication and determines when a dose has been removed. Once this happens, it:
- Records the time of the event
- Updates the caregiver dashboard
- Sends an SMS notification through Twilio
This creates a feedback loop between the patient’s medication area and the caregiver, allowing medication activity to be monitored in real time.
How we built it
We used an ESP32-CAM as an image-capture server and connected it to a Python application. OpenCV and NumPy were used to process and compare images through a rolling frame buffer.
We created a multi-threaded Tkinter dashboard so new camera data and medication events could be displayed without freezing the interface. Twilio was used to send real-time SMS notifications, and we designed a mobile-app concept in Figma to show how the system could eventually be used by caregivers.
Challenges we ran into
Our biggest challenge was getting the hardware, image-processing program, dashboard, and notification system to communicate reliably.
The ESP32-CAM produced raw image captures, but we still needed to process those images, identify meaningful movement, update the interface, and send a notification at the correct moment. Since we built the project in under 24 hours, we had to debug several parts of the system at the same time.
Accomplishments that we're proud of
We are proud that we created an end-to-end prototype combining embedded hardware, computer vision, a caregiver interface, event tracking, and SMS communication.
Rather than building each part separately, we were able to connect them into one working system that responded to activity in the real world.
What I learned
This project taught us how difficult, but rewarding, it is to connect hardware and software into one reliable system. We gained experience with camera-based motion detection, ESP32 communication, multi-threaded interfaces, API integration, and designing around the needs of both patients and caregivers. We also learned how important it is to divide tasks, communicate clearly, and adapt quickly during a short hackathon.
What's next for Pill Check
With more time, we would improve the vision system so it could distinguish between someone taking medication and unrelated movement near the container. We would also like to add scheduled-dose reminders, missed-dose alerts, medication history, support for multiple medications, and a mobile caregiver application. Eventually, the system could use a more personalized detection model while still protecting the patient’s privacy.
Log in or sign up for Devpost to join the conversation.