Launch screen - describing an ideal use case for the application in a workplace
Audio record and live-transcribe screen
Screen showing smart AI-generated summaries with action items and key points
AI-based conversation screen to go deep into the summary generated from the voice recording
Screen showing many different share options including various file formats
A representational screen, showing that the app supports most of the major spoken language in the world
App home page screenshot without device frame
app icon

Inspiration

In the age of AI, privacy is often an afterthought. While there are hundreds of note-taking apps on the marketplace, almost all of them send sensitive user voice data to the cloud or third-party servers for processing. The primary inspiration behind our AI Note Taker was to build a powerful, privacy-first alternative. We wanted to prove that users don't have to sacrifice the security of their personal thoughts, meetings, and ideas to enjoy the benefits of modern AI-based note-taking solutions.

What it does

AI Note Taker is a secure, intelligent voice recording application that keeps user voice data exactly where it belongs: on their device locally. The app captures audio and performs high-quality transcriptions completely locally on the user's device - meaning a user's voice never leaves their phone. Once the text is safely transcribed locally, the app generates smart, concise summaries of the transcription using cloud-based AI services. To further protect user identity, AI Note Taker requires zero sign-in or account creation, ensuring no personally identifiable information is ever collected.

How we built it

To make efficient offline processing a reality, we fine-tuned the open-source Whisper model specifically for on-device transcription. This allows the app to process raw audio directly on the smartphone without relying on an internet connection for the heavy lifting. Once the secure, local transcript is generated, we utilise the OpenAI API strictly for the text summarisation phase, providing users with a neatly organised, actionable and searchable overview of their recordings.

Challenges we ran into

Our biggest hurdle was achieving market-leading transcription quality while strictly limited to on-device processing. High-end AI models typically rely on massive cloud computing power. Condensing that capability to fit within the memory and processing constraints of a mobile device, without compromising transcription accuracy or draining the battery, required rigorous optimisation and creative model fine-tuning. We also faced some challenges in the Android-native JNI communications for smooth audio processing to provide the live-transcribe feature.

Accomplishments that we're proud of

We successfully bridged the gap between strict data privacy and high-end utility. We are incredibly proud to have built a highly accurate voice-to-text pipeline that proves you do not need to harvest user audio in the cloud to provide an elite, AI-driven experience. Establishing a zero-login, zero-audio-tracking architecture is a massive win for user trust.

What we learned

We learned a tremendous amount about edge computing and the deep technical nuances of deploying suitable state-of-the-art machine learning models on mobile architectures. Optimising the Whisper model taught us how to delicately balance computational efficiency with high-fidelity output. More broadly, we gained valuable insights into designing products where user privacy is the foundational feature, rather than just an updated policy.

What's next for AI Note Taker: Voice Recorder

We are focused on expanding the app's global utility and speed. Our immediate next step is building an advanced, real-time live transcription system capable of supporting major global languages, all while strictly maintaining our uncompromising, on-device privacy standards.