Ride Storyteller

Turn motorcycle telemetry into questions, video into evidence, and evidence into an explainable travel story.

Comment

Problem

A motorcycle trip can produce hundreds of gigabytes of footage. The memorable story is buried across long video files, camera timestamps, and a separate GPS record. Manually finding the useful moments is slow, while treating GPS events as visual facts would invent evidence that the camera may never have captured.

Solution

Ride Storyteller converts a GPS route into explainable candidate events. A Story Agent decides when route context is insufficient and requests video evidence. A media-search boundary resolves the relevant source interval, Gemini analyzes only that interval, and the Story Agent accepts, rejects, or escalates the candidate. The edit remains blocked until every selected clip is timestamp-matched and its visual evidence is explicitly confirmed with an attributed source.

The track then gives the film its shape: a day is cut into legs, halt to halt, and every leg is a chapter titled with where it went from and where it went to. Moments the track proves — setting off, each stop's arrival and re-departure, joining and leaving a highway, riding off a ferry, arriving — are always shown, and the model's own answers choose which neighbouring window actually shows the moment. The turns of the day are said in the lower third with the local time.

The output is a three-to-seven-minute film per riding day with no voice narration and existing copyright-free music. Twelve consecutive days of one tour have been cut this way. The public three-minute hackathon video demonstrates the functioning workflow rather than substituting a cinematic trailer for the product demo.

Why This Matters

Motorcycle riders and travel creators often have far more footage than editing time. Ride Storyteller reduces the search space while keeping the human in control of the final visual claim. GPS says where to look; the camera evidence determines what can honestly enter the story.

How We Used AI

  • Gemini 2.5 Flash watches every candidate window of a real ride's own footage (small local proxies, never the 4K source) and answers a fixed schema: road, scenery, interest, story relevance, how much of the rider is in frame, whether the bike moved, what the road is doing, how photogenic the frame is and of what, and what kind of place the bike is standing at. Across twelve riding days that is 4,039 stored judgements for about ¥764 in total — some windows were bought a second time as the questions asked of the model grew.
  • A Google ADK agent receives one fixed synthetic event, invokes a typed evidence tool, and produces a structured final response with Gemini 2.5 Flash.
  • A synthetic-only Google Cloud Agent Platform Runtime in Tokyo verifies the hosted ADK tool/final-response loop without accepting GPX, coordinates, video, Box data, credentials, or arbitrary prompts.
  • A Vertex AI Gemini video transport prepares schema-constrained analysis for an already-approved gs:// clip. It does not upload local media.
  • The Story Agent treats malformed output, missing media, model unavailability, and rejected visual evidence as human-review states instead of inventing a successful edit.
  • An optional Gemini Story-copy boundary rewrites only fixed synthetic chapter copy while preserving chapter IDs, order, and count.

How We Built It With AI Assistants

Two coding assistants were used, in two phases. Until 2026-09-02 Codex was the primary implementation partner: it turned the product constraints into frozen data contracts, deterministic GPS and story planning, explicit evidence-state transitions, Google ADK and Agent Platform adapters, the bilingual local/public-safe UI, Cloud Run deployment safeguards, tests, and submission documentation. From 2026-09-02 Claude Code took over core implementation, review, and the design record, with Codex handling small scoped tasks; the story structure, the judged-film pipeline, the reference layer, and the film's own rules were built in that phase. Both ran regression and secret/private-media checks and kept the Japanese design, decision, history, and test records synchronized with the repository.

IBM Bob was used separately during development to review the earlier codebase. Its findings about missing ADK wiring, evidence transitions, video transport, and boundary tests were then implemented and mapped to focused regression tests. Bob also implemented one slice on its own task plan: the fail-closed ScriptExecutor and the Director pipeline behind the director_mode opt-in, with their tests, which landed on 2026-09-01 and still pass.

Both Codex and Claude Code were development-time coding assistants only. No non-Google AI model, agent framework, or AI API runs in the project itself. The on-device OCR and face detection (Apple Vision) used before publishing the demo video is a local pre-publication privacy check, not part of the agent pipeline.

Key Features

  • Explainable GPS event extraction and stable story identifiers.
  • Agentic evidence loop: decide, search, analyze, update, or escalate.
  • Explicit awaiting, confirmed, and rejected evidence states with source attribution.
  • Half-open source intervals and camera/GPS clock correction, read from the cameras' own GPS where present so that no one is asked at all.
  • A day cut into legs, halt to halt, each a chapter card titled from → to, with "Day N" on a multi-day trip detected from sibling packages.
  • Moments the track proves are always shown, and the model's answers pick the window that shows them.
  • Lower thirds with the local time for towns, scenic routes, named roads, passes, highways, ferries, and each stop by what kind of place it was.
  • An opening of four one-second highlights chosen by a photogenic score, of distinct subjects, spread across the day.
  • Offline map references (touring routes, highways, ferry lines, towns, passes) fetched once per country from OpenStreetMap.
  • Fail-closed privacy rules in the cut itself: no window that could identify a private home, and no window where the rider fills the frame.
  • Inspectable multi-clip FFmpeg planning, and a single normalised render pass that leaves no half-written film.
  • Japanese/English UI with invariant status and domain contracts.
  • Synthetic-only Google cloud path separated from private local media workflows.
  • Public-demo mode that removes cloud, GPX, Maps, and private-media controls.
  • Root-license, secret, ignored-file, and private-media submission preflight.

Architecture

private/local route or synthetic event
  -> GPS parser and explainable event extraction
  -> Story Planner
  -> Story Agent decides whether evidence is needed
  -> media-search tool boundary
  -> Gemini video-analysis boundary
  -> attributed evidence decision
  -> candidate edit quality gate
  -> inspectable FFmpeg render plan

Google ADK exposes the safe synthetic decision loop as an agent/tool workflow. The deployed Agent Platform Runtime accepts only a fixed non-private event. The public Cloud Run image is a separate lean web demonstration and contains no Google SDK or credentials. IBM Bob is evidenced as a development-process tool, not falsely presented as a runtime integration.

Known Limitations

  • Real footage reaches Gemini only as small local proxies (one frame a second, 480 lines, no audio). The 4K source never leaves the rider's machine, and the hosted agent has still only received synthetic events.
  • The hosted service shows the agent's decision loop on synthetic data only; the judged-film pipeline runs on the rider's machine (the judge package cuts one real day there), because real footage cannot be placed on a public server.
  • The demo video was read frame by frame on the owner's machine for faces and number plates before publication and none were found; that reading is sampled on-device OCR and face detection, a reduction of risk rather than a guarantee.
  • Place names come from a Google Geocoding request per rounded coordinate; a film cut with that service disabled keeps the titles the track alone allows.
  • Stop kinds and ferry crossings are read partly from the model's words about older judgements; windows bought before those questions existed answer "unknown" and fall back to word matching.
  • Box is optional future media infrastructure and is not a valid contest track.

For judges

The hosted console (URL in the submission's Additional info) asks for the credential given in the attached file. The product itself runs on your own machine: download ride-storyteller-day-7.zip from the repository's release day-7-package and follow the four commands in the attached file or the archive's README.txt. No credential is needed for that path.

Bob also implemented one slice on its own task plan: the fail-closed ScriptExecutor and the Director pipeline behind the director_mode opt-in, with their tests, which landed on 2026-09-01 and still pass.

Built With

Share this project:

Updates

Submission history