Remembron

Login/Signup Screen
Homescreen Layout (Showcases user info ,reminders and other accessibility tools)
Settings Screen
User Setup
GeoFencing feature in action

What is Remembron?

Dementia affects more than 55 million people worldwide as of March 15, 2023. Alzheimer's disease, the most common cause, accounts for nearly 70% of these cases. The impact on individuals and their families is profound, with symptoms that gradually erode memory and cognitive functions. These symptoms include:

Frequent forgetfulness: Forgetting recent events, names, and important dates.
Getting lost: Disorientation while walking or driving, even in familiar areas.
Difficulty recognizing familiar people or environments: Social interactions and daily activities become increasingly challenging.
Inability to follow conversations: Struggling to keep up with or understand ongoing conversations.
Decreased problem-solving abilities: Everyday tasks become more difficult to complete.
Changes in mood and behavior: Increased anxiety, confusion, and mood swings.

This led to the inception of Remembron, an app tailored to support individuals with Alzheimer's through intuitive features like facial recognition, geo-fencing, audio tracking with Speech-to-Text (STT), and reminders via Text-to-Speech (TTS). Remembron is designed to help users overcome daily challenges posed by their condition and act as a virtual caretaker.

Remembron Features

Facial Recognition

Alzheimer's disease can impair memory, making it difficult for individuals to recognize loved ones. To assist in such scenarios, Remembron includes a Facial Recognition feature. It identifies faces and their relation to the user, providing information about the person shown.

How It Works:

A machine learning Convolutional Neural Network (CNN) model compares images to identify and determine relationships.
The model is trained using live video input, broken down into frames to extract facial data with Haar Cascade.

Geo-fencing

Alzheimer’s can cause disorientation, leading to users getting lost or wandering into unsafe areas. To mitigate this, Remembron uses geo-fencing with two boundaries around a designated home location.

Features:

Users can set the radii of the inner and outer fences.
Notifications are sent when users pass the inner fence, and guidance is provided when they cross the outer fence.
IP tracking for geolocation is achieved via the ipgeolocation API.

UI & UX

Given that Alzheimer’s predominantly affects those aged 75 and older, Remembron features a simple and user-friendly interface.

Salient Features:

Secure access through FireAuthentication for sign-up and sign-in.
Frontend built with Flutter/Dart; backend uses Node.js with Python scripts for API requests, and data is stored on Firebase and Google Cloud.
A mutable reminders list for adding, editing, or removing reminders, with real-time updates.

Speech-to-Text (STT) and Text-to-Speech (TTS)

Alzheimer’s often leads to forgetting daily tasks. Remembron addresses this with STT and TTS features.

Features:

STT tracks and extracts keywords from daily conversations to create automatic reminders.
TTS provides verbal reminders for tasks and geo-fence notifications.

Challenges We Ran Into

Issues in Creating a Pipeline from Firebase to Flutter for Python Scripts

Developing a seamless data pipeline between Firebase and the Flutter frontend for executing Python scripts was a significant challenge. The main difficulty lay in ensuring reliable communication and data flow between the two systems. Synchronizing real-time updates and maintaining data consistency required careful integration of Firebase services with Flutter's data handling capabilities, often necessitating intricate configuration and debugging.

Finding an Ideal Geolocation API for Accurate Tracking

Finding a geolocation API that provided precise and reliable location tracking with minimal error proved to be a challenge. The API needed to deliver accurate real-time data to effectively manage geo-fencing and guide users safely. We had to evaluate various APIs to ensure they met our accuracy requirements while also performing efficiently, minimizing the potential for errors and improving the reliability of location-based notifications.

Overfitting of the Facial Recognition Machine Learning Model

The initial machine learning model for facial recognition faced overfitting issues, where it excelled with the training data but struggled with new, unseen images. This problem occurred because the model became too specialized to the training set, resulting in poor generalization. To address this, we employed techniques such as data augmentation, regularization, and enhanced validation methods to improve the model's ability to recognize faces accurately across different scenarios and lighting conditions.

Accomplishments That We're Proud Of

Effective Integration of Advanced Technologies

We are proud of successfully integrating a range of technologies to create Remembron. Combining Flutter for the frontend, Python scripts for backend processing, Firebase for real-time data handling, and advanced machine learning for facial recognition has resulted in a comprehensive and user-friendly application. Overcoming the challenges of ensuring smooth data flow and real-time updates between these technologies was a significant achievement.

Robust Facial Recognition System

Our facial recognition feature, powered by a Convolutional Neural Network (CNN) model, has been a major accomplishment. Despite initial challenges with overfitting, we refined the model using data augmentation and regularization techniques to enhance its accuracy and reliability. This feature helps users identify and remember their loved ones, which is a crucial support tool for those with Alzheimer's.

Innovative Geo-fencing Solution

Implementing an effective geo-fencing system with customizable boundaries around a home location has been a key achievement. This system helps prevent users from getting lost by sending timely notifications and providing directions when they move outside set boundaries. The use of the ipgeolocation API for accurate tracking further enhances this feature’s effectiveness, ensuring user safety and peace of mind.

User-Friendly Design

Given the high prevalence of Alzheimer’s among older adults, we’re proud of our focus on creating a simple and intuitive UI/UX. The design considerations make Remembron accessible and easy to use, helping users navigate the app effortlessly and manage their condition more effectively.

What We Learned

Challenges of Technology Integration

We learned the importance of seamless integration when working with diverse technologies. The process highlighted the need for careful planning to ensure that Firebase, Flutter, and Python scripts communicated effectively. This experience underscored the complexity of managing real-time data and maintaining consistency across different components of the application.

Balancing Machine Learning Performance

Our work with the facial recognition model taught us valuable lessons about model training and generalization. We discovered that preventing overfitting and improving model performance requires ongoing refinement and a deep understanding of machine learning techniques. This experience emphasized the need for continuous testing and adjustment to achieve optimal results.

Importance of Accurate Geolocation

Finding a reliable geolocation API that provides accurate tracking data was a significant learning experience. We realized that precision in location-based services is crucial for effective geo-fencing. The process of evaluating and selecting the right API taught us the importance of thorough testing to ensure minimal errors and reliable performance.

What's Next for Remembron

Expanding Feature Set

We plan to enhance Remembron by adding new features and improving existing ones. Future updates will focus on incorporating additional cognitive support tools, such as personalized activity recommendations and advanced memory exercises, to better support users in managing their condition.

Incorporating User Feedback

We will actively seek and incorporate feedback from users and caregivers to refine and enhance the application. By addressing user needs and making iterative improvements, we aim to ensure Remembron remains effective and responsive to the needs of those with Alzheimer’s.

Broadening Accessibility

Our goal is to expand Remembron’s reach to a global audience. This includes developing support for multiple languages and adapting the application for various devices and platforms. By making Remembron more accessible, we aim to provide valuable support to users worldwide.