Memora: Clear Your Mind, Conquer Your Day.

Top View of the Hand accessory
Front view of the Camera module
Image analysis using ML
ML training model confidence report
Side view of the Hand accesory

Inspiration

Have you ever had this feeling where you walk into a room and just forget why you entered, lose an idea you really wanted to remember, or just forget an assignment until it’s too late to submit? This is a problem that every single person goes through in normal day-to-day activities, whether it’s something serious like taking your meds or just a simple reminder of lowering your caffeine intake. Having something that will give you a physical reminder when you need it is a necessity for almost everyone. That’s why we created Memora. Being inspired by the problems we face currently and what our parents have been facing for a long time is exactly why our team of four passionate engineers who are eager to grow and learn came up with an incredible solution. Digging deeper, we found that this solution is revolutionary for people facing challenges like memory impairment. Imagine the fear, the isolation, of losing your memories, your connection to the world around you. This is the reality for millions living with dementia. We've seen heartbreak in families, the strain on caregivers, and the silent struggle of those affected. It's this shared human experience, coupled with our team's deep-seated passion, that fuels our drive to make Memora a reality, a tangible solution for a universal challenge.

What it does

Memora operates on a foundation of robust, localized machine learning. To begin, we utilize a standard ESP32 overclocked, enabling us to capture and process high-resolution images and 2D scans of faces and objects. This initial data collection phase is crucial, forming the basis of our proprietary machine learning dataset. For instance, in a real-world application, this allows a user to rapidly train the device to recognize frequently misplaced items like keys or wallets or to identify the faces of loved ones, ensuring they are never forgotten.

Subsequently, we leverage the power of Edge Impulse and TensorFlow to develop a custom machine learning algorithm. This algorithm is then distilled into a dedicated, efficient library optimized for the ESP32's architecture. This enables Memora to accurately identify faces and objects with a predetermined level of precision, even in diverse lighting and environmental conditions. Imagine a user, in a busy market, needing to quickly identify a family member; Memora's algorithm can do this swiftly and reliably.

Upon successful identification, the ESP32 initiates a wireless data transfer to the user’s bracelet. This is achieved through a secure, locally hosted private Wi-Fi connection, transmitting encoded numeric values. This ensures data privacy and speed. In a practical scenario, if a user is reminded to control their caffeine intake when it sees a Red Bull can, the identification of that can triggers the reminder.

On the bracelet, the received numeric values trigger a script that sends the programmed reminder text to Google Cloud’s Text-to-Speech (TTS) AI. This conversion to audio happens rapidly, allowing for seamless communication. For example, a user might be reminded to "take your medicine now" via a clear, natural-sounding voice.

The Google Cloud TTS output is then converted into an encoded audio format compatible with the ESP32’s I2S amplifier, ensuring high-quality audio playback. Simultaneously, the onboard LCD display on the bracelet displays the text of the reminder, providing a visual cue. This dual-sensory approach ensures that users with varying needs can effectively receive and understand the reminders.

All of these processes, from image capture and ML processing to audio and text output, are executed within a remarkable three seconds in our current prototyping phase. This rapid response time is critical for real-life applications, ensuring that Memora provides timely and effective support for users, whether it's reminding them of daily tasks or helping them navigate their surroundings with confidence.

How we built it

The journey to create Memora was a testament to our team's ingenuity and relentless problem-solving. Faced with the challenge of building a sophisticated assistive device with limited resources, we began by meticulously evaluating every available component. Through rigorous brainstorming and testing, we arrived at an optimal configuration, carefully selecting our key components and processors for their balance of performance, affordability, and power efficiency. This initial phase was crucial, setting the stage for the innovative approach we would take.

Next, we delved into the intricacies of programming these processors. Recognizing the need for offline machine learning capabilities, we focused on integrating ML processing into basic, readily available Chinese microcontroller boards. This required extensive research and experimentation, pushing the boundaries of what these boards could achieve. After countless hours of coding and testing, we successfully developed our first working, basic single object detecting algorithm. This breakthrough was a pivotal moment, validating our approach and providing a foundation for further development.

Building upon this success, we expanded our dataset and refined our ML algorithm using the powerful tools provided by Edge Impulse. This platform enabled us to iterate quickly, improving accuracy and reliability. With a robust ML algorithm in place, we shifted our focus to the bracelet component. We established a seamless wireless communication system, enabling the necklace to transmit signals to the bracelet when a designated face or object was recognized. This involved setting up secure local Wi-Fi networks for both ESP32s, ensuring reliable and private data transfer.

With wireless communication established, we turned our attention to the user interface. We implemented a system for setting reminders associated with identified objects and faces. Leveraging Google Cloud's Text-to-Speech (TTS) AI, we converted these text reminders into encoded audio files, ensuring high-quality playback through our I2S amplifier and speakers. Simultaneously, we displayed the reminder text on an LCD 16x2 display with an I2C shield, providing a clear visual cue. This dual sensory approach ensures that users can receive and understand reminders effectively, regardless of potential sensory impairments. Throughout this process, we prioritized efficiency, ensuring that all operations, from object detection to audio and text output, are executed rapidly. This commitment to speed and reliability is essential for creating a truly useful and impactful assistive device.

Challenges we ran into

We encountered several key challenges during Memora's development. Primarily, we faced significant hardware limitations. Optimizing machine learning algorithms for an entry-level microcontroller demanded extensive resource management and pushed the device to its processing limits. Achieving reliable facial recognition with a 2MP camera required meticulous data collection and algorithm refinement, ensuring accuracy in diverse lighting conditions.

A critical challenge arose just before our presentation: a sudden LCD display failure. This forced us to quickly diagnose the issue and implement a DIY solution using a variable resistance, showcasing our ability to adapt under pressure. These practical obstacles tested our resourcefulness and problem-solving skills, demanding rapid, creative engineering. Each challenge reinforced our commitment to Memora, proving that ingenuity and perseverance are essential for overcoming technical hurdles and bringing innovative ideas to fruition.

Accomplishments that we're proud of

We're immensely proud of several key accomplishments that define Memora's journey. First, we achieved robust face and object recognition on a platform never intended for such complex tasks. By building our own ML library, we pushed a cheap microcontroller to its maximum limits, demonstrating that sophisticated assistive technology doesn't require expensive hardware. We created something incredible from scratch, proving that innovation can thrive with limited resources.

Our commitment to making extremely affordable assistive technology is a significant achievement. We've developed a solution that's accessible to a wider demographic, addressing a critical need without financial barriers. Finally, we're proud to have developed something that was never thought of before, a unique integrated system that combines advanced ML with practical, user-friendly design. We've proven that groundbreaking technology can be both powerful and accessible, and we're excited to bring Memora to those who need it most.

What we learned

We pushed the limits of what we believed to be feasible with assistive technology today as we started a steep and quick learning curve. By learning firsthand about the enormous potential of optimizing algorithms for resource-constrained situations, we expanded our knowledge of embedded machine learning. We gained an understanding of the complex interplay between software creativity and hardware constraints, learning how to get the most performance possible from widely accessible microcontrollers.

In order to ensure effective and confidential data flow between devices, we acquired vital experience in developing a seamless, localized wireless communication system. We learned the value of modular architecture and strong error handling from the challenges of integrating many technologies, from audio output and display to camera input and machine learning.

We also improved our user-centered design abilities, realizing how important accessibility and ease of use are in assistive technology. We understood the subtleties of turning intricate algorithms into user-friendly interfaces and realized that the real worth of technology is found in its capacity to empower and assist. Additionally, we understood the value of iterative development and rapid prototyping, which let us quickly adjust to obstacles and improve our strategy in response to immediate feedback.

Lastly, we reiterated our conviction that affordable innovation can be powerful. We discovered that innovative technology can be created with little funding, demonstrating that drive and creativity can get beyond even the most difficult challenges. We departed today with a fresh perspective on how technology can improve people's lives and a greater comprehension of the commitment needed to turn an idea into a workable, game-changing solution.

What's next for Memora

Our immediate focus is on solidifying Memora's foundation, ensuring it's ready for widespread impact. We will meticulously refine the user experience, prioritizing clarity and ease of use. This involves a comprehensive front-end development cycle, concentrating on the LCD display and audio feedback to guarantee seamless interaction for all users. We are also committed to enhancing the core functionality through rigorous machine learning model training. By significantly expanding our dataset with diverse faces and objects, and by exposing the model to numerous real-world prompts, we will achieve a marked increase in recognition accuracy and reliability.

Crucially, we will also begin integrating more advanced and convenient hardware components. This includes exploring next-generation microcontrollers with faster processing prowess, enabling us to execute our machine learning algorithms even more efficiently and reduce latency. We will also look into more compact and power-efficient camera modules, and audio components, making Memora more comfortable and discreet for everyday wear. These hardware upgrades will allow us to further enhance the device's performance, ensuring seamless and reliable operation in diverse environments.

To protect our unique innovation and pave the way for sustainable growth, we will actively pursue patent protection for our technology and design. This will solidify our market position and ensure our groundbreaking approach remains exclusive. We are also dedicated to minimizing online dependencies, striving for a truly self-contained, offline solution that respects user privacy and guarantees uninterrupted functionality.

Our initial market entry will concentrate on individuals living with dementia and their caregivers. We will forge strategic alliances with Alzheimer's associations, senior living facilities, and healthcare providers to facilitate targeted adoption and gather crucial user feedback. We will also prioritize building a supportive online community. To expand Memora's reach to the general public, we will develop a versatile user interface that caters to a wide range of needs, from daily organization to enhanced personal safety. We will explore partnerships with retail channels and online marketplaces to broaden our distribution.

From a business standpoint, we will prioritize building a robust and scalable model. This includes refining our pricing strategy, optimizing manufacturing processes, and investing in continuous research and development. We will also explore subscription-based services for premium features and ongoing support, ensuring long-term value for our customers. Through strategic partnerships and a steadfast commitment to innovation, we are confident that Memora will become a leading force in the assistive technology market, empowering individuals and fostering greater independence.