MotionMemo

Motion Memo turns recorded workouts into structured training records, helping you remember every exercise, track progress, and ask AI questions about your training history.

Comment

Inspiration

I work out regularly, but I am not always confident that I am training effectively. During personal-training sessions, I often rely on my trainer to remember what I did, how much weight I used, which movements were difficult, and what I should improve next time.

After working with several trainers, I noticed that everyone tracks progress differently. Some trainers keep detailed notes, while others rely on memory or scattered messages. Even with a great trainer, the workout history often belongs to the trainer—not the person doing the training. When someone changes trainers, takes a break, or begins training independently, much of that valuable context disappears.

I built Motion Memo to give people ownership of their training history.

Motion Memo acts as a persistent memory for every workout. It listens during a session, turns the conversation into an organized training record, and helps users understand how they are progressing over time.

What it does

Motion Memo turns a recorded workout into a structured, editable, and searchable training history.

A user can start recording and focus entirely on the workout without repeatedly stopping to type notes into a phone. After the session, Motion Memo:

  • Transcribes the workout conversation.
  • Identifies exercises, sets, repetitions, weights, and equipment.
  • Captures personalized coaching cues and technique feedback.
  • Highlights accomplishments, improvement areas, and discomfort observations.
  • Produces a concise recap with recommendations for the next session.
  • Lets the user review and correct the generated workout before finalizing it.

The review experience is important because workout conversations are often informal and ambiguous. Users can rename exercises, change sets, repetitions, weights, or coaching cues, and add or remove movements before accepting the record.

Motion Memo also includes Ask AI, a conversational assistant grounded in the user’s finalized workout history. Users can ask questions such as:

  • “What have I improved recently?”
  • “How has my squat weight changed?”
  • “Which coaching cues does my trainer repeat?”
  • “When did I last mention shoulder discomfort?”
  • “What should I focus on next time?”

Users can continue asking follow-up questions in the same conversation. Instead of responding with generic fitness advice, Ask AI retrieves relevant information from that specific user’s previous sessions and uses it to produce a grounded answer.

Motion Memo also provides body-measurement tracking, recording-credit management, session history, and a companion web portal.

How we built it

We built Motion Memo as a local-first mobile application with a companion web portal.

The iPhone app uses Expo, React Native, and SQLite. When the user starts a workout, the app records one continuous compressed audio file and saves the session locally before attempting any network operation. This protects the recording when connectivity is unreliable and allows failed uploads to be retried.

After the workout:

  1. The mobile app uploads the audio directly to Vercel Blob through a presigned URL.
  2. The backend transcribes the recording.
  3. GPT-5.6 transforms the transcript into structured workout data.
  4. The app presents that data for review and correction.
  5. The finalized workout—not the unreviewed transcript—is embedded for future retrieval.
  6. Neon Postgres and pgvector store and search each user’s training history.
  7. Ask AI combines semantic similarity with historical relevance to retrieve useful sessions.
  8. The mobile app reconciles completed processing when it returns to the foreground.

Long transcripts are processed in smaller windows with neighboring context. This reduces model and serverless-processing pressure while helping preserve exercises that cross processing boundaries.

The web portal uses Next.js and provides access to session history, finalized workout details, Ask AI, and credit management. Clerk handles authentication, while Stripe supports web-based credit purchases and subscriptions.

We also used Codex as an engineering partner throughout development. Codex helped us inspect the architecture, plan changes, implement features across the mobile and server codebases, validate builds, diagnose device-specific failures, and deliver focused changes through pull requests.

After introducing Codex on July 15, Motion Memo’s development velocity increased dramatically. In the five days before Codex, three pull requests were merged. During the following five days, 33 pull requests were merged—26 through Codex-managed branches—covering 108 files across the mobile app, backend, retrieval pipeline, reliability systems, documentation, and product design. This represented an 11× increase in pull-request throughput.

Challenges we ran into

The greatest challenge was making mobile recording and cloud processing reliable under real-world conditions.

Our first design divided long workouts into multiple audio files. During device testing, we discovered that iOS could not reliably reactivate audio recording in the background after each segment. Locking the phone could therefore interrupt a session. We redesigned the system to record one continuous file while still presenting recording status through the lock-screen Live Activity.

Other challenges included:

  • Uploading large recordings without exceeding serverless request limits.
  • Protecting unfinished recordings when the app is force-closed.
  • Recovering from interrupted uploads and slow processing.
  • Making synchronization jobs persistent, retryable, and idempotent.
  • Avoiding duplicate transcription or extraction work.
  • Processing long transcripts within serverless execution limits.
  • Preserving context when exercises cross processing boundaries.
  • Keeping locally cached data isolated when users switch accounts.
  • Ensuring that finalized user corrections replace the original AI interpretation.
  • Retrieving relevant history without mixing data between users.
  • Making AI-generated answers readable and appropriately cited.
  • Designing credits so they can be reserved, consumed, or refunded safely.

Many of these issues only became visible during full-device testing. The architecture evolved substantially as we tested complete workouts rather than isolated screens.

Accomplishments that we’re proud of

We are proud that Motion Memo is not simply a concept or interface prototype. It is a functioning application that can be used during a real workout.

A user can record a training session, lock the phone, finish the workout, wait for asynchronous processing, correct the generated record, and then retrieve that information through a multi-round AI conversation.

We are especially proud of:

  • Building a resilient, local-first recording workflow.
  • Supporting continuous background recording on iOS.
  • Converting natural workout conversations into structured records.
  • Preserving context across long transcript-processing windows.
  • Giving users control over AI-generated information before it becomes permanent.
  • Embedding finalized records rather than unreviewed model output.
  • Building user-isolated hybrid retrieval with Neon and pgvector.
  • Supporting conversational follow-up questions in Ask AI.
  • Creating consistent mobile and web experiences.
  • Adding body measurements, account management, credits, and subscriptions.
  • Building something useful enough to use ourselves and recommend to others.

What we learned

Our biggest lesson was that architecture and data-flow planning must happen before polishing the interface.

Motion Memo combines recording, background execution, local storage, cloud processing, AI extraction, retrieval, synchronization, authentication, and billing. A decision in one area can affect the entire product. A seemingly straightforward choice—such as rotating audio files every few minutes—can fail because of an operating-system behavior that only appears on a physical device.

We also learned that AI output should be treated as an interpretation, not unquestionable truth. Prompt quality matters, but validation, editable review screens, safe retries, and careful handling of malformed output matter just as much.

For retrieval, we learned that semantic similarity alone is insufficient. Questions about workout progress frequently contain a historical component, so relevant answers require a combination of meaning, recency, dates, and finalized session data.

Finally, we learned to test the entire user journey early: starting a workout, locking the phone, losing connectivity, ending the recording, waiting for processing, reviewing the result, asking historical questions, and switching accounts. End-to-end testing exposed problems that individual component tests could not reveal.

What’s next for Motion Memo

Our next goal is to turn Motion Memo from a functioning product into a polished training companion that more people can use confidently.

Planned improvements include:

  • Completing TestFlight and App Store distribution.
  • Adding native mobile purchases and subscriptions.
  • Strengthening the remaining recording and synchronization recovery paths.
  • Adding speaker identification for trainer and client voices.
  • Creating progress charts for weight, volume, consistency, and technique.
  • Supporting exports and integrations such as Apple Health.
  • Improving exercise identification and optional instructional media.
  • Expanding body-measurement and progress-tracking features.
  • Testing with more trainers and athletes at different experience levels.
  • Exploring trainer-facing collaboration tools.

Ultimately, we want Motion Memo to give every person a reliable and portable memory of their training—regardless of who they train with, where they exercise, or how their fitness journey changes.

Built With

Share this project:

Updates