A Castle in the Sky

Inspiration

The project began with the question of how literature can be experienced in new, immersive ways. Traditional reading is intimate and deeply reflective, but not always accessible to wider audiences. At the same time, advancements in generative media and sound design now make it possible to create dynamic, multi-sensory interpretations of stories without altering the original text. A Castle in the Sky offered the perfect narrative to explore this, rich in symbolism, allegory, and emotional depth, while allowing us to preserve the integrity of the short story and present it in a new experiential form.

What it does

This project transforms a short literary work into a new dynamic literary experience. Instead of adapting the story into a screenplay, the original text is maintained word for word. Moving visual clips, soundscapes, and narration are layered around it, similar to a picture book but brought to life in motion. The experience is designed to feel like reading with senses engaged, where imagery, audio, and pacing enhance the meaning and emotional resonance of the story.

How we built it

We began by mapping the story’s symbolic and emotional beats directly to sensory elements. Visual fragments were curated and stitched together to mirror the narrative’s tone, including dust, ash, distant machinery, flickering lights, and atmospheric transitions. A narration track was recorded to preserve the story’s cadence, accompanied by a custom soundscape that evokes tension, exhaustion, and collective endurance. Captions were added to anchor key phrases. The entire structure was aligned to the exact text, ensuring no adaptation, only augmentation.

Challenges we ran into

One major challenge was defining the project’s identity. Because it uses video and audio, it could easily be mistaken for a traditional short film, even though the intent and lineage are literary. Balancing cinematic tools with a reading-like experience required careful pacing and restraint. Another challenge was translating symbolic imagery into motion without overshadowing the text, ensuring the visuals supported rather than replaced the story.

Accomplishments that we're proud of

We created a format that respects literature while embracing contemporary media. The story remains entirely intact, yet the experience has the potential to feel innovative and emotionally immersive. We also developed a model that can be applied to many types of written works, from parables to poems to cultural narratives, making literature accessible to audiences who may not engage with traditional reading. The final piece demonstrates that it is possible to modernize storytelling without abandoning its roots.

What we learned

We learned that the boundary between film and literature is more porous than expected, but also that lineage and intention matter. By starting with the text rather than a screenplay, and by treating visuals as companion layers rather than primary drivers, the experience retains a literary identity even while using cinematic tools. We also learned how powerful sound design and pacing can be in shaping the participant’s emotional interpretation of a story.

What's next for A Castle in the Sky

Next steps include developing a fully individualized version of the experience, ideally consumed with headphones for deeper immersion. We are also considering expanding the format into a series, adapting more short stories and exploring cultural narratives from diverse communities. Eventually, the goal is to build a platform for dynamic literary experiences, offering a new mode of storytelling that sits between reading, visual art, and immersive audio design.

Built With

• elevenlabs
• hailuo
• openai
• runway
• skyreels