• 

  minimalistic design and apple native ui

• 

  Home page widgets

• 

  completely offline

• 

  perfect for meetings

• 

  perfect for planning

Ear — Hackathon Submission

Track: Track 3 — Mobile AI
Demo: YouTube
Source: github.com/jashwanth0712/ear

Inspiration

Important meeting context is often lost between the conversation and the follow-up. We wanted a private, mobile-first assistant that turns spoken decisions into useful next steps—without sending a meeting to a cloud bot.

What it does

Ear records a meeting on iPhone, creates an on-device transcript, separates speakers, and produces a searchable summary, decisions, open questions, and action items. Every output is reviewable against the original conversation.

How we built it

Ear is a native SwiftUI app for Arm-powered iPhones. AVFoundation captures audio; Apple Speech handles on-device transcription; a Core ML CAM++ embedding model and local clustering identify speakers; and quantized Gemma models run through llama.cpp with Metal for structured meeting notes. SwiftData keeps the meeting record on-device.

Challenges we ran into

Making a full meeting-AI workflow work reliably on a phone required careful resource management. We handled speech-recognition request limits with rolling windows, optimized diarization for short audio chunks, added RAM and thermal guards around local LLM inference, and retained a grounded heuristic-notes fallback when the model is unavailable.

Diarization optimization

• Started with real-time speaker embeddings; short windows caused unstable speaker labels.
• Added speaker-turn timing/spacing signals to improve clustering stability.
• Added completed-recording post-processing for more audio context.
• Final approach: Core ML CAM++ embeddings plus post-processing and re-diarization.
• Result: more consistent speaker grouping on-device, with fewer incorrect speaker splits.

Accomplishments that we're proud of

• Delivered an end-to-end local-first meeting workflow instead of a cloud-dependent demo.
• Ran the app successfully on physical iPhone 15 and iPhone 16 devices.
• Added reproducible benchmark tooling, device evidence, and a reviewer-ready Track 3 proof-artifact trail.
• Made AI outputs practical with editable, evidence-linked tasks, decisions, and questions.

What we learned

On-device AI is a systems problem: model quality matters, but memory, thermals, audio segmentation, structured outputs, and a graceful fallback matter just as much. We also learned that transparent, evidence-linked results earn more trust than a polished but opaque summary.

What's next for Ear

We plan to expand benchmark coverage across more Arm devices and longer meetings, improve speaker-label correction and offline language support, and refine model-selection controls so users can balance quality, speed, and battery use for each meeting.

Built With

• coreml
• gemma
• swift