Inspiration:
Our inception was merely a shared frustration we all experienced - not remembering all of the details of our own lives. A modern concern for most is that they are busy enough to not keep a daily journal, or regularly reflect - still, memory, emotional intelligence, and self-reflection each day are all extremely important for personal growth, mental wellbeing, and pushing ourselves cognitively.
We were also inspired by a group of memory challenged people, neurodivergent folks, like those with ADHD, or early onset dementia, who often quite simply have trouble recalling the events of their days. What if AI could help reconstruct your day as a story - with context, feeling and meaning?
That is really how NeuroScribe was formed - as an AI memory companion. It was designed to passively listen, learn, and relate your life so you can reflect, recall, and formatively grow yourself even further.
What it does:
NeuroScribe is a mobile and web app that will transform your day into a cohesive journal entry using:
- Voice Notes and ambient audio (utilizing Whisper AI)
- Calendar events, to dos, and geolocation data
- Phone usage data and sensor data
- Journal narratives that are tuned to fluctuate in real time based on your perceived emotions (using LLMs)
It then turns out a journal entry that resembles a story, with:
- Daily mood/emotion time line
- Highlights and behavior patterns
- Personal insights and memory triggers
This provides users with a fully passive and artificial intelligence driven journaling exercise that is valuable for self reflection, with no effort on the part of the user.
How we built it:
NeuroScribe was entirely developed in VS Code using:
- React Native for the cross-platform mobile application;
- Next.js for the responsive web dashboard;
- Node.js + Express for the backend API;
- PostgreSQL for structured event storage;
- MongoDB for unstructured narrative logs;
- OpenAI's Whisper for voice-to-text transcription,
- HuggingFace Transformers + fine-tuned GPT to generate narratives and emotion,
- Firebase for authentication and user session management,
- Google Maps & Calendar API for evidence of location and activities.
We created a Temporal AI Fusion Engine to align multi-modal data (e.g., audio, time, geolocation), and a Narrative Compression Algorithm that only selects meaningful events (user-value).
Challenges we ran into:
- Context Alignment: We needed to design new fusion logic for arbitrary data fusion (e.g., text, time-series, audio) to ensure a consistent narrative.
- Privacy and Consent: Our design needed to facilitate ethical data collection, provide user control, facilitate opt-out processes in real time, and to be conscientious of limited data on the user's device.
- Mobile Performance: Running AI models (e.g., Whisper) locally takes considerable time and resources. For this reason we optimized the AI for hybrid edge-cloud use.
- Narrative Coherence: The early outputs felt robotic and we had to work to adjust the models, remove non-verbal crosstalk, and inject emotional arcs to produce human-style storytelling.
Accomplishments that we're proud of:
- Effectively reconstructed full-day memory logs for test users with over 90% perceived accuracy.
- Developed an emotionally intelligent journaling system that adjusts based on user mood and behavior overtime.
- Developed a fully working prototype that is made entirely with open-source and cloud-native tools.
- Developed a solution that is similar to reality, in terms of relevance to mental health, productivity, and cognitive assistance.
What we learned:
- Ways to use multi-modal data for a single personalized experience
- The need to consider privacy-by-design to the passive AI systems
- Best practices for fine-tuning LLMs for narrative coherence
- How to optimize AI pipelines across mobile and web applications.
What's next for NeuroScribe – Your AI memory companion:
- 🧠 Memory Search Engine: Users can search back in their journaling using queries that use semantic language such as “When was I last feeling anxious before a big meeting?”
- 🔒 Zero-Knowledge Storage: Allow encrypted journaling which is unreadable even to us.
- 🌐 Therapist Dashboard: Share insights with mental health professionals (but only when consent is provided).
- 🎧 Wearable Interactivity: Smart glasses or earbuds will allow users to collect ambient context more accurately.
- 📚 Research Publication: We are writing a paper called "Temporal Narrative Reconstruction with Emotion-aware AI Agents."
NeuroScribe is not just a journal, but a step towards creating the digital memory prosthetic of the future.
Log in or sign up for Devpost to join the conversation.