Inspiration
Restoring Dignity The initial spark for Cognito-Weave was deeply personal, rooted in observing the quiet crisis of cognitive decline. While many brilliant projects focus on physical sight or mobility, we were drawn to the mission of restoring dignity and independence to the elderly and cognitively impaired—a mission aligning with the spirit of the Prakash initiatives. The challenge was: how can technology do more than just manage decline? How can it actively help a mind function better?
This led us to the core concept of the Cognitive Prosthetic: a lightweight, non-invasive system that uses AI to dynamically augment the user's existing mental capacity, specifically targeting memory and attention, rather than just testing them.
What it does
It actively helps the elderly and cognitively impaired maintain their mental abilities and independence and fundamentally shifts care from passive management to active augmentation through three core functions:
\(1.\) Augments Memory Recall (The Adaptive Story-Weaver) It uses a Voice-First interface (Speech-to-Text) to allow the patient to tell a story or memory.
An AI/NLP engine analyzes their fluency and content in real-time.
The system then generates a dynamic, personalized scaffolding prompt ("Who did you go with?" or "What did you eat?") to gently guide and complete the memory retrieval process.
\(2.\) Augments Attention and Focus (The Haptic Feedback Loop) The app monitors user engagement during a task (e.g., measuring silence or response time).
If focus drops, it triggers a subtle, non-intrusive haptic vibration (a custom cue) on the mobile device.
This cue gently draws the user's attention back to the screen without the distraction or annoyance of a loud alert, maintaining task engagement.
\(3.\) Provides Objective Data for Caregivers (The Longitudinal Cognitive Profile) All user sessions are securely logged to a cloud database.
This data is visualized on a dedicated Caregiver Dashboard (M4) to provide the progress of the patience through the graphs.
How we built it
The project was structured as an MVP proof-of-concept across three integrated modules, relying solely on Python, JavaScript, HTML, and CSS:
The Mobile Client (HTML/CSS/JavaScript): We built a progressive web app (PWA) shell for the front-end using HTML, CSS, and JavaScript to achieve cross-platform capability. This handles the Elderly-First UI and integrates the device's native Speech-to-Text (STT) APIs via JavaScript to capture the patient's voice input, fulfilling Milestone 1 (M1).
The Augmentation Engine (Python/Flask): This is the core of the Adaptive Story-Weaver (M2). We built a lightweight Python/Flask back-end that receives the transcribed text. A simple NLP/rule-based engine quickly analyzes the input for fluency (word count, pauses) and generates the therapeutic scaffolding prompt (e.g., "What did you eat?").
The Feedback and Data Loop (JavaScript, Python, & PostgreSQL):
We implemented the Haptic Feedback Loop (M3) logic using the JavaScript Haptics API, triggering a distinct haptic pattern if silence exceeded five seconds (simulating lost focus).
The Caregiver Dashboard (M4) was built using simple HTML, CSS, and JavaScript, fetching and visualizing the key metrics for proactive care.
Challenges we ran into
The development involved overcoming two significant challenges:
Real-Time Data Latency: The largest technical hurdle was ensuring the Augmentation Loop was seamless. The chain involves: Voice → STT → Internet → Flask API → NLP Analysis → Flask Response → Internet → Text-to-Speech (TTS) Output. Minimizing latency across these steps—especially on local networks for testing (using 10.0.2.2 for the Android emulator)—was critical to making the experience feel conversational and non-robotic.
Ethical Design Constraint: The biggest design challenge was avoiding "deception" and "surveillance." The app had to be transparent that it was a tool, not a companion. This was solved by making the haptic cue subtle but purposeful, and designing the core interaction as a collaborative storytelling session, maintaining user dignity above all else. The ethical constraint drove the innovation.
Accomplishments that we're proud of
We are immensely proud to have delivered a functional, full-stack proof-of-concept for the Cognitive Prosthetic using only Python, JavaScript, HTML, and CSS. Our key accomplishment was successfully integrating the novel AI Augmentation Loop: the Python-powered Adaptive Story-Weaver takes live voice input and generates real-time memory prompts, while the JavaScript-driven Haptic Feedback Loop provides non-intrusive attention cues. Finally, we established a complete, secure data pipeline that translates these interactions into clear, objective metrics (Fluency and Haptic Count) on a working Caregiver Dashboard, validating our mission to deliver dignified, data-driven support.
What we learned
This project was a deep dive into the practical application of AI and accessible design, yielding several key lessons:
AI for Augmentation: We learned to pivot the use of AI/NLP from traditional analysis (like sentiment scoring) to dynamic augmentation (scaffolding a memory). The "Adaptive Story-Weaver" model taught us that a simple, rule-based NLP framework, when combined with Speech-to-Text (STT), can create a powerful conversational experience that is far more therapeutic than a complex, black-box model.
Elderly-First UX: We gained a profound understanding of designing for high cognitive load environments. The solution wasn't just large buttons, but the deliberate removal of visual-search cognitive load through Audio-First Navigation. Every interaction must be clear, high-contrast, and audibly confirmed.
The Power of Haptics: Integrating the mobile OS Haptics API showed us how a subtle, distinct vibration pattern can be a highly effective, non-annoying tool for cueing attention, proving more useful than flashing lights or loud sounds.
What's next for Cognito-Weave
Our immediate next steps are focused on graduating the proof-of-concept into a robust, clinically relevant platform ready for a pilot program:
\(1\))Enhance AI/NLP Granularity: We plan to move beyond our rule-based system to integrate more sophisticated machine learning models (e.g., fine-tuned open-source NLP models) to analyze semantic content and topic coherence, providing a more nuanced fluency_score and generating more contextually relevant scaffolding prompts.
\(2\))Formalize Caregiver Alerts & Intervention: Develop real-time alert logic on the Caregiver Dashboard. This will allow the system to push notifications when a patient's haptic_trigger_count suddenly spikes (indicating a bad day) or when their average fluency_score dips over a 7-day period, enabling immediate and targeted caregiver intervention.
\(3\))Expand Accessibility and Localization: Fully implement multi-lingual support, recognizing that patients often revert to their mother tongue during cognitive decline. We will formalize the PWA for easier installation and explore integrating more custom accessibility features beyond the core high-contrast UI.
\(4))Clinical Pilot Preparation: Partner with a retirement community or elder care facility to begin a small-scale pilot study. This will be crucial for gathering real-world, longitudinal data to validate the utility of our augmentation metrics and refine the user experience based on genuine patient and caregiver feedback.
Log in or sign up for Devpost to join the conversation.