Research Cinema

Inspiration

University research is trapped in spreadsheets and search results. Thousands of papers, scholars, and global collaborations — reduced to tables nobody walks up to. We asked: what if a research database could tell its own story, cinematically, on a wall display — and anyone could talk to an AI about what they're seeing?

What it does

Research Cinema turns a publication database (Typesense-indexed, OpenAlex-enriched) into a cinematic wall experience. A 3D globe shows live search queries worldwide. When cinema plays, each scholar or topic gets 5 stories × 6 scenes each — animated counters, concept trees, collaboration webs, paper cards, SDG impact, and frontier questions. Visitors scan a QR code on their phone to open an AI companion that answers questions grounded in the exact data on screen, in real time.

How we built it

Edge layer: Cloudflare Worker serves the wall display, proxies Typesense searches, and relays WebSocket state through a Durable Object hub — no API keys ever reach the browser.

Wall display: Three.js globe with custom shaders (day/night, transparent oceans, grid lines), 400 twinkling stars, 9 orbital comets, trending panels from live Typesense data, and floating search query labels. Cinema plays through a playlist controller that loads 10 story HTML pages (5 scholar + 5 topic) in iframes with bridge transitions and background music.

Story pages: Each of the 10 story templates renders 6 canvas-driven scenes — particle bursts, force-directed concept trees, Three.js collaboration globes with arc animations, paper card carousels with citation counters, avatar rings with breathing animations, and typing-effect closings. All data is received via postMessage from the playlist controller.

AI companion: Google ADK agent on Cloud Run with gemini-3.1-flash-lite-preview model. 12 built-in tools (Typesense search, OpenAlex lookup, scholar comparison, wall signaling). The companion receives live wall state via WebSocket and sends grounded prompts through the Cloudflare Worker proxy to GCP.

Data pipeline: Typesense collections (33K+ papers, researchers, networks) at ITB, enriched with OpenAlex global context. Wall payloads are built on-demand with derived timelines, collaboration maps, SDG alignment, and "gem" facts.

Challenges we ran into

Iframe-within-iframe message passing. The wall loads a playlist controller iframe, which loads story iframes inside it. Getting postMessage to flow correctly across three layers — wall → playlist → story — without race conditions or lost data took careful orchestration.

Story pages crashing silently. Each story page has its own requestAnimationFrame loop. If any data field is null — a missing collaborator latitude, an empty paper array, an undefined topPaper.title — the loop dies and the scene freezes. We had to guarantee every payload field exists with sensible defaults.

Relative path hell. Moving story pages from /scholar/story-01.html to /stories/scholar/story-01.html broke every ../image/ reference. SDG icons, sample portraits, hero images — all 404'd silently until we caught and fixed every relative path.

Browser autoplay policy. Cinema music plays inside an iframe. Browsers block autoplay in iframes unless the parent explicitly grants permission with allow="autoplay". Finding this took longer than it should have.

Playlist looping infinitely. The playlist controller was designed to loop forever for a kiosk. In our architecture, it needed to stop after one cycle and return to the globe. We had to intercept cinema-complete, send cinema-stop, and add an early-exit path for embedded mode.

Accomplishments that we're proud of

Zero-config deployment. wrangler deploy puts the entire wall + companion on Cloudflare's edge. One gcloud run deploy puts the AI agent on GCP. Two commands, globally available.

The globe-to-cinema transition. Clicking "Shuffle Scholar" fades a 3D interactive globe into a full cinematic experience with opening titles, 5 stories, narrative bridges, and background music — then fades back to the globe. It feels like a real installation.

AI companion grounded in live context. The companion doesn't hallucinate — it receives the exact wall payload (scholar data, paper list, citation counts) and answers from that first. It knows what scene is playing and what data is on screen.AI may expand data search to Typesense database or OpenAlex if the question complex beyond what already shown in story

*30 topic hero images and 41 example scholar profile posters (using any available profile picture) generated and matched automatically. The system picks topics and scholars that have pre-generated visuals so the demo always looks polished.

What we learned

Edge architecture changes everything. Putting Cloudflare Workers between the browser and every backend service (Typesense, ADK) means API keys never leak, rate limiting is trivial, and CORS is solved once.

Durable Objects are underrated. A single Durable Object holds the live wall state and relays it to every companion via WebSocket. No Redis, no pub/sub infrastructure — just one persistent object on the edge.

The hardest part of a cinematic data app isn't the visuals — it's the data shape. Every story scene assumes specific fields exist with specific types. The gap between "Typesense returned 5 papers" and "the story page needs topPaper.authors, gem.rapidFacts[2].icon, and sdgAlignment[0].image" is where most bugs lived.

What's next for Research Cinema

Any institution, any database. Swap the Typesense endpoint and the wall tells a different university's story. The architecture is institution-agnostic.

Voice companion. The AI companion currently uses text. Adding Gemini Live voice would let visitors speak to the wall directly.

Generative UI. The AI companion response currently is text. since the story based on rich data, AI can respond in the form of interactive charts/table/cards

Real-time image generation. generate a cinematic one on-the-fly using Gemini's image capabilities before the story plays.

Multi-wall orchestration. Multiple wall displays in a building, each showing different stories, all sharing companion state through the Durable Object hub.

Deeper OpenAlex integration. Use OpenAlex concepts, citation networks, and institutional comparisons to generate richer "gem" facts and more accurate global rankings.