JOLO | Devpost

JOLO's Inspiration

The idea for JOLO was born from the desire to make journaling more accessible, fun, and engaging. Traditional journaling can be time-consuming and hard to maintain, especially for those with busy schedules or specific challenges like disabilities. We took inspiration from the rising trend of voice-based interfaces, platforms like BeReal, and the growing need for mental health solutions that fit seamlessly into daily life. With JOLO, we aimed to create a voice-first journaling experience that encourages self-reflection while using technology to provide deeper insights into one’s emotional well-being.

What it does

JOLO is a voice-first journaling app that allows users to record their thoughts through spoken entries, making the process quick and effortless. The app features time-based prompts to nudge users to reflect at different points in the day. It leverages AI to analyze entries, providing users with mood insights, sentiment analysis, and keyword tagging, which helps users better understand their emotions over time. JOLO also incorporates gamified self-reflection, rewarding users for consistent journaling habits, and providing daily, weekly, and monthly summaries of their emotional state.

How we built it

We built JOLO using React Native for a seamless, cross-platform experience on both iOS and Android devices. The backend was developed in Python, leveraging cloud services for scalable storage and processing. We integrated AI models to perform natural language processing (NLP) for sentiment analysis and keyword tagging, utilizing frameworks like TensorFlow and PyTorch. We also implemented voice-to-text technology to convert users' spoken entries into text for analysis. The app's interface is intuitive, featuring a clean, minimal design with easy navigation between journaling sessions, insights, and achievements.

Database

MongoDB: Prompt management Ease of connection with APIs like SingleStore and quick retrieval AWS S3: Object storage and user file management Long term mp3 archive storage and scalability AWS Amplify to interact with the S3 bucket in the app

APIs Used

VAPI Voice API; website: vapi.ai

Screens: Home Screen Speech to Speech Interaction, stored as Speech to Text + Speech Journaling with a fine tuned VAPI AI assistant that has been trained on past user journal entries to provide customized support and guidance Journal mp3 recording and transcription is automatically sent as a public entry to the Social Media page and the Archive under the relevant question prompt, creating a new prompt card if necessary

Hume Voice AI API; website: hume.ai

Screens: Home Screen, Archive Real-time speech-to-text transcription of journal entries SingleStore Screens: All Screens SingleStore Helios for extremely fast JSON processing, allowing the app to handle data retrieval and insertion without noticeable delays. This is particularly beneficial when managing large amounts of prompts in MongoDB Vector searches and live data streams for real-time data updates when a user interacts with a prompt, adds a journal entry or edits existing content

Hyperbolic API; website: hyperbolic.xyz + Gemini

Screens: Home Screen, Archive, Calendar, Social Media Gemini, integrated alongside the Hyperbolic API, powers the app's live transcription and natural language processing (NLP) capabilities Real-time keyword tagging, language understanding, and context extraction, enhancing the accuracy and relevance of journal prompts and entries; Hyperbolic's inference capabilities are used for advanced voice and text processing. JOLO utilizes this to perform deep analysis on transcribed speech, extracting key phrases, keywords, and tags, which then enrich the journal entries and provide a refined, searchable structure. This allows users to revisit their past entries with context, using semantic analysis to make connections and reflect on their thoughts more effectively.

Deepgram - Voice AI; website: Deepgram

Screens: Calendar + Private Journaling Retroactive private journaling for past journal prompts by talking to a DeepGram voice assistant Future impl: https://mem0.ai/ for more personalized and accurate responses based on past user journal entries since its a self improving memory layer for applications like this Cartesia Voice API Screens: Archive, Calendar and Social Media Page Automatic language translation of journal recordings and transcriptions based on user-specified language

Google Gemini

For NLP tagging and semantic search Google Gemini supports JOLO’s natural language processing (NLP) tasks. By tagging important keywords and phrases from journal entries, JOLO provides insights and personalized suggestions to users. With Gemini, we can ensure entries are parsed for sentiment, themes, and topics, which enhances the user's reflective experience and offers deeper insights into their journaling journey.

Singlestore: Best Real-Time App; website: Singlestore

Singlestore powers the backend database for JOLO, enabling real-time data processing. This integration ensures users' journal entries are stored, accessed, and analyzed instantaneously, offering a smooth, lag-free experience. This real-time capability is essential for providing immediate feedback during live transcription and enabling instant search and retrieval of past entries.

Cartesia: Best AI Voice Project; website: Cartesia Sonic

Cartesia’s sonic technology enables JOLO’s multilingual support, allowing users to journal in multiple languages seamlessly. This makes JOLO accessible to a wider global audience, ensuring everyone can reflect, express, and capture their thoughts, no matter their preferred language. The integration with Cartesia Sonic also enhances audio processing, delivering accurate transcription for diverse speech patterns.

The Future of Speech and Text Interaction

JOLO exemplifies the future of voice and text integration by seamlessly merging transcription, inference, and AI-based tagging to provide a holistic journaling experience. We leveraged Calhacks-only APIs, including Hyperbolic and Cartesia, to create a solution that empowers users to express themselves naturally, reflect effectively, and engage with their thoughts meaningfully. The integration of voice AI, NLP, and multilingual support demonstrates JOLO's potential as a universal journaling tool, making speech-to-text interactions more intuitive, empathetic, and accessible.

Challenges we ran into

One of the biggest challenges was accurately converting voice to text in various conditions, such as background noise or different accents. We had to experiment with different models and tune the settings to optimize for accuracy and responsiveness. Another challenge was building an effective AI model that could provide accurate sentiment analysis across diverse phrases and expressions, ensuring it could pick up on subtleties in users' entries. Lastly, ensuring seamless cross-platform functionality with React Native and dealing with memory management issues during audio processing proved to be quite complex.

Accomplishments that we're proud of

We’re proud of creating an AI-powered journaling app that makes self-care more accessible, especially for users who may struggle with traditional text-based journaling. Our voice-first approach makes the process of self-reflection quicker and easier, and the integration of AI ensures users receive valuable insights from their entries. We also successfully implemented features like time-based prompts and gamified achievements, which enhance user engagement and retention. Our app has a sleek, user-friendly design that we believe will appeal to a broad audience.

What we learned

Through building JOLO, we learned a lot about the nuances of voice interaction and natural language processing, especially the importance of handling edge cases like slang, mixed languages, and abbreviations. We also learned how crucial it is to build an intuitive user experience that guides users naturally through the process, as journaling can be a deeply personal activity. Additionally, we learned how to balance complex backend AI processing with a smooth frontend experience, ensuring the app remains responsive even with real-time voice-to-text conversion.

What's next for JOLO

Moving forward, we want to expand JOLO’s capabilities by introducing more advanced features, such as mood tracking based on tone analysis, integration with smart assistants for hands-free journaling, and deeper insights through customizable tags. We also plan to implement multilingual support, allowing users to journal in their preferred language. Finally, we see potential in forming partnerships with mental health platforms, providing additional resources to users who may need them, and launching JOLO+ as a premium subscription with advanced analytics, unlimited entries, and personalized journaling prompts.