Ethni-CITY

Your photos. Their music. Ethni-CITY uses Gemini AI to turn travel photos into interactive Sonic Story-Zines, soundtracking your journey with hyper-local artists instead of global pop hits.

Comment

Ethni-CITY: The Soundtrack of Travel

Inspiration

Travel is a privilege, but it’s often one-sided. We travel to the Global South, stay in global hotel chains, and post our social stories with the same trending pop hits that everyone else uses. In the process, the actual pulse of the city—the niche local artists and the independent music economy—is left behind.

We built Ethni-CITY to decentralize the soundtrack of travel. We wanted to create an agent that acts as a "Cultural Bridge," helping travelers "slow down" and discover the authentic sounds of the streets they are walking on. Our goal was to turn every travel photo into a tool for local support and authentic storytelling.

What it does

Ethni-CITY is an AI-powered Creative Storyteller designed to transform travel photos into immersive Sonic Story-Zines.

  • Multimodal Analysis: The app uses Gemini Vision to "read" your photos, identifying specific neighborhoods, architectural motifs, and cultural markers.
  • The Sonic Zoom: I implemented a cinematic transport system using CesiumJS and Google Photorealistic 3D Tiles, grounding the digital experience in a physical 3D space.
  • Agentic DJ (Vibe Curation): I built an interface where you interact with an AI director who understands "the vibe" you're looking for and sifts through hyper-local niche artists to find a perfect match.
  • Interleaved Story-Zines: The final output is a rich, mixed-media "Zine" that weaves together AI-generated lore and streaming audio in a single, fluid flow.

How I built it

I built Ethni-CITY using a modern, high-performance stack:

  • Core Framework: Next.js 15 (Turbopack).
  • AI Engine: I developed a custom Hierarchical Gemini Logic that cycles between gemini-3.1-pro-preview, gemini-2.0-flash, and gemini-1.5-flash.
  • Spatial Layer: I integrated CesiumJS with Google Photorealistic 3D Tiles.
  • Multimedia: I used the iTunes Search API for dynamic, real-time audio discovery.

Technical Formulation: Hierarchical Resilience

To ensure 100% uptime during the hackathon, I implemented a cascading fallback logic represented by the probability of a successful request ($P(S)$), where $n$ is the number of models in my tier:

$$P(S) = 1 - \prod_{i=1}^{n} (1 - A_i)$$

Where $A_i$ is the availability of each model tier. With this 4-tier system (Paid 3.1 $\rightarrow$ Regular 2.0 $\rightarrow$ Regular 1.5 $\rightarrow$ Flash-8b), my system availability remains near-absolute even under extreme quota pressure.

Challenges I ran into

The biggest technical hurdle I faced was Model Grounding Accuracy. I wanted to ensure that the "Vibe Curation" wasn't just a generic search. I developed a Cultural Alignment Score ($C_a$) to weight Gemini's visual tokens ($T_v$) against the artist's metadata tags ($M_a$):

$$C_a = \sum (T_v \cdot M_a) \cdot \omega_{region}$$

Where $\omega_{region}$ is a regional cultural weight I implemented to prioritize local niche genres (like Jùjú or Forró) over global pop trends. Managing the CesiumJS deployment within a Next.js environment also required complex build-time configuration to ensure 3D assets were served correctly.

Accomplishments that I'm proud of

  • True Interleaved Output: I moved beyond simple text-based AI. My agent generates a structured, mixed-media narrative that feels like a published zine.
  • Cultural Accuracy: I was stunned by Gemini's ability to distinguish between very specific regional cultures solely from an uploaded photo.
  • Neo-Brutalism UI: I designed and built a custom, premium UI from scratch that gives the app an "agency-grade" feel.

What I learned

I learned that Gemini isn't just a language model—it's a cultural observer. Its ability to act as a "Creative Director" allows it to make curatorial choices that feel human and intentional. I also discovered that "Spatial Storytelling" (combining 3D maps with AI) creates a profound sense of "being there" that static apps can't replicate.

What's next for Ethni-CITY

I want to take Ethni-CITY from a desktop zine-factory to a global mobile companion:

  • Mobile Export: Generating social-ready video stories with one click.
  • Real-time Live API: Transitioning the "Agentic DJ" into a full Gemini Live voice interaction.
  • Direct Support: Enabling users to support the local artists they discover through direct digital payments.

*Ethni-CITY: Promote the unseen, soundtrack the world. *

Built With

Share this project:

Updates