-
-
見るだけで、終わらせない。
-
Beyond the surface — 90 works, 4 languages, 360 source-backed audio guides.
Inspiration
Museum visits often stop at the surface: a title, a date, a material, and a short wall label. Traditional audio guides usually cover only a few highlights, while online explanations often make it difficult to tell where a claim came from.
We wanted to build a guide that helps visitors understand not only what they are looking at, but why it matters.
GUAN started with one rule: if we cannot trace a factual claim to a real source, we would rather show an honest gap than fill it with plausible-sounding text.
What it does
GUAN is a multilingual, source-backed deep art guide for museum visitors.
Its first complete guide covers all 90 works in Kūkai and the Treasures of Shingon Buddhism at the Tokyo National Museum. For each work, GUAN organizes interpretation into six perspectives:
- Historical and social context
- The creator and moment of creation
- Contemporaries and related styles
- Later reception
- Curatorial perspective
- Connections to other works in the exhibition
Visitors can:
- Plan around real exhibition dates and object rotations
- Read and listen in Simplified Chinese, Traditional Chinese, Japanese, or English
- Play localized narration and follow its transcript
- Open the references supporting each interpretive lens
- See explicit research gaps instead of fabricated certainty
- Search exhibitions, venues, works, and full guide text
- Save language, playback, and reading-mode preferences
- Use the responsive web app or the native iOS companion
The catalog contains 90 works and 360 localized audio tracks. The first 12 works can be explored anonymously; the staging judging build enables the complete 90-entry catalog without login.
Build Week scope
GUAN existed as an early prototype before OpenAI Build Week. This submission covers only the substantial extension completed during the Build Week submission period.
During Build Week, we:
- Expanded deep-reading coverage from 12 to all 90 catalog entries
- Expanded Web and iOS from two to four languages
- Expanded localized audio from 24 Chinese and Japanese files to 360 files across four languages
- Productionized a GPT-5.6 Batch narration pipeline
- Built repeatable model and multilingual TTS evaluations
- Expanded the existing operations console with recurring model and multilingual TTS evaluations
- Added native iOS settings, StoreKit 2 purchases, and a paper-inspired reading mode
- Deployed automated staging infrastructure and CI/CD
Timestamped implementation evidence includes the GPT-5.6 Batch pipeline, recurring evaluation contract, full 90-work catalog, four-language audio, and the native iOS localization and Settings experience.
How we built it
GUAN separates verified evidence from generated narration.
The content catalog stores exhibition metadata, display rotations, six interpretive lenses, references, and explicit gaps as structured data. The build pipeline validates schemas, reference relationships, localized assets, and catalog integrity before deployment.
GPT-5.6 Sol transforms the frozen, source-backed content into natural spoken narration. GPT-5.6 Terra independently generates a shadow candidate for comparison. They run as separate OpenAI Batch jobs, and outputs are matched by stable IDs rather than response order. Source and prompt fingerprints prevent stale output from being published after the underlying content changes.
Only Sol output is eligible for canonical files; publishing still requires reviewed git changes and staging validation. Terra remains a shadow result and cannot automatically replace production content. GPT-4o mini TTS converts reviewed narration into localized audio.
We also built a repeatable evaluation system. Every candidate receives the same frozen source and prompt. Deterministic checks measure length, formatting, language consistency, unsupported claims, and other product requirements. Blind model judging and human review are layered on top, but model promotion always remains a deliberate human decision.
The web application uses React, Vite, Hono, Cloudflare Workers, D1, R2, Queues, and Cron. The native iOS app uses SwiftUI, AVPlayer, URLSession, StoreKit 2, XcodeGen, and Xcode Cloud.
Codex acted as our engineering collaborator across the web, iOS, catalog, and operations repositories. We used it to inspect architecture, turn product decisions into issues, implement and test changes, debug CI/CD and deployment problems, review pull requests, and keep contracts synchronized across repositories.
The human remained responsible for the important decisions: the source-first trust rule, privacy boundaries, product scope, language quality, the Sol-versus-Terra rollout, and the rule that no model can promote itself into production.
Challenges we ran into
The hardest problem was not generating more text. It was defining what trustworthy museum narration means.
Museum sources vary greatly in authority and structure. Exhibition lists also contain rotations, split object numbers, changing display periods, and occasional transcription errors. We had to preserve provenance and uncertainty throughout the pipeline.
Model evaluation exposed another challenge. GPT-5.6 Terra was faster and received a strong overall score, but initially produced narration that was too short for the intended listening experience. GPT-5.6 Sol was more consistent with the required narration length. This led us to choose Sol as the production model while keeping Terra in shadow mode for continued measurement.
Localized audio added another failure mode: a missing track must never silently fall back to the wrong language. We added locale-specific text, audio hashes, pronunciation probes, and explicit no-fallback checks.
Finally, the product spans multiple repositories and deployment environments. Keeping the web app, iOS app, content catalog, evaluation console, database migrations, and CI/CD contracts aligned required careful automation and review.
Accomplishments that we're proud of
- A deployed, usable museum guide covering all 90 exhibition works
- Four complete language and audio experiences, totaling 360 audio tracks
- Six source-linked interpretive perspectives per work
- Date-aware handling of real museum display rotations
- A 78-work Sol and Terra Batch narration run completed without failed items
- A reproducible evaluation system instead of subjective model selection
- A native iOS companion and a production-like Cloudflare architecture
- A workflow that treats uncertainty and missing evidence as product information
Most importantly, GUAN demonstrates that AI-generated cultural interpretation can be useful without asking visitors to blindly trust the model.
What we learned
A high general model score does not guarantee that a model satisfies a specific product experience. Length, spoken rhythm, unsupported facts, pronunciation, and language fallback all need separate evaluation.
We also learned that offline cultural content is a strong fit for the OpenAI Batch API. Most museum narration does not need to be generated in real time, so we can spend more effort on validation while keeping production costs controlled.
Frozen sources and prompts turn model comparison from an anecdotal demo into an engineering process. Codex was most effective when the human supplied clear constraints and retained control of product and promotion decisions.
What's next for GUAN · Deep Art Guide
Next, we plan to:
- Expand from one exhibition to a reusable Tokyo museum catalog
- Add more exhibitions and cities through venue-specific data adapters
- Run the same evaluation suite whenever a promising new model appears
- Connect offline evaluation results with real listening completion and visitor feedback
- Complete App Store and Android distribution after provider validation
- Improve accessibility, visit planning, and personalized exhibition routes
Our long-term goal is simple: help anyone look beyond the surface of an artwork, with every factual step traceable back to evidence.
Log in or sign up for Devpost to join the conversation.