Inspiration
Nature documentaries are incredible at showing us the natural world, but some of the most fascinating parts of biology are difficult to capture on camera.
You can watch a hummingbird hover in slow motion, but understanding the aerodynamics of its wings is another story. You can observe an animal feeding, but not necessarily see the anatomy that makes it possible. And you can follow a species through its habitat without understanding its evolutionary relationships or migration patterns.
Scientific visualizations can reveal these hidden details, but producing custom 3D explanations for every documentary is a significant amount of work.
That's what inspired Phylum.
Rather than creating another immersive documentary player, I wanted to build a system that connects existing nature films with reusable spatial biology tools.
Phylum is built around an agent-ready production workflow that connects documentary footage, scientific research, and a library of reusable spatial visualizations. These range from phylogenetic trees and geographic distributions to anatomical models and biomechanical breakdowns.
By organizing these visualizations into configurable tools with structured inputs, Phylum establishes a foundation for AI agents to eventually identify relevant biological concepts, retrieve supporting research, and help assemble spatial documentary experiences.
The result is a documentary experience where viewers can move naturally between watching wildlife and examining the biology behind what they're seeing.
What excites me most is the possibility of bringing this to independent wildlife filmmakers. They already have the footage, the stories, and the knowledge of their subjects. Phylum provides a way to add another layer of scientific storytelling without requiring filmmakers to build their own XR applications.
What It Does
Phylum consists of two connected experiences.
1. Phylum: The Mixed Reality Experience
A nature documentary plays in mixed reality, accompanied by spatial visualizations that expand on the biology being discussed.
Instead of displaying explanatory graphics on a flat screen, Phylum lets viewers manipulate biological concepts directly. These visualizations are synchronized with the documentary, appearing alongside the film when relevant to the narration or subject matter.
For this submission, I created a sample nature documentary about hummingbirds using stock wildlife footage. This serves as a demonstration of how an existing film can be adapted into a spatial episode without requiring its original creator to produce additional 3D content.
The hummingbird episode explores several areas of biology:
- Evolution: Navigate a spatial phylogenetic tree to understand how hummingbirds relate to other birds and to each other.
- Biomechanics: Examine hummingbird wing trajectories and flight mechanics using animated 3D models.
- Anatomy: Explore the specialized structures of hummingbird beaks, tongues, and feathers.
- Ecology: Follow geographic distributions and seasonal migration patterns on a 3D Earth.
- Physiology: Investigate the biological mechanisms that enable hummingbirds' remarkable abilities.
Each visualization is connected to its corresponding documentary segment, allowing viewers to move between observing an animal and examining the science behind its behavior.
Scientific explanations can also include DOI references to their original research papers, connecting the spatial visualizations to the biological research behind them.
2. Phylum Studio: The Agent-Ready Production System
Phylum Studio is a web-based authoring system designed to reduce the work required to produce spatial documentaries.
Rather than building every documentary experience from scratch, Phylum Studio provides a structured workflow for connecting documentary content with a reusable library of spatial UI systems.
These include:
- Spatial phylogenetic trees
- Geographic and migration visualizations
- Anatomical models and cross-sections
- Motion and animation breakdowns
- Other configurable biological visualizations
Each visualization is designed around configurable parameters and structured scientific data, allowing the same underlying tools to support different species, biological concepts, and documentary episodes.
The production system is designed to be agent-ready. Its structured visualization library and episode format provide a foundation for future AI agents to analyze documentary content, identify opportunities for spatial explanations, retrieve relevant scientific research, and configure visualizations.
Rather than relying on AI to generate arbitrary 3D experiences, this approach gives future agents a defined set of purpose-built spatial tools to work with.
Creators can preview and configure spatial visualizations through a web interface before preparing an episode package.
That package contains structured JSON, visualization parameters, timestamps, references, and any required 3D assets. I can then import it into Unity, where the documentary and its spatial components are assembled into an experience for Meta Quest.
How I Built It
Phylum is organized around a reusable spatial visualization system and a structured episode format.
Meta Quest and Unity
I developed the mixed reality application in Unity for Meta Quest. The documentary playback system is synchronized with spatial visualizations that can be configured, animated, and manipulated through hand tracking.
Instead of building each scene as an entirely separate Unity experience, I designed individual visualization systems to be reusable across different documentary subjects.
The goal was to separate the underlying spatial tools from the content of any particular documentary. For example, a phylogenetic tree can display different evolutionary relationships depending on the data provided, while a geographic visualization can represent the distribution or migration patterns of different species.
This allows the same spatial systems to be reused across future episodes without rebuilding them from scratch.
Agent-Ready Content Architecture
I designed the production workflow around structured inputs, reusable visualization templates, and configurable episode data.
Instead of treating every documentary as a completely new XR development project, Phylum organizes spatial explanations into individual tools that can be selected, configured, and connected to specific moments in a film.
This architecture is designed with future AI integration in mind. An agent could eventually analyze documentary footage or transcripts, identify relevant biological concepts, select appropriate visualization templates, retrieve scientific literature, and generate configuration data for each segment.
The important distinction is that the underlying spatial components and their configuration interfaces already exist independently of an AI agent.
By separating content decisions from the implementation of individual visualizations, Phylum creates a predictable environment where AI could eventually assist with production without needing to generate entire Unity scenes.
Research and Scientific Data
Scientific references are incorporated into the content structure through research citations and DOI metadata.
Biological visualizations can be associated with relevant scientific papers, allowing viewers to connect spatial explanations with the research behind them.
The system is also designed to support future research retrieval, where an AI agent could locate relevant scientific literature and use supporting information to help populate visualization parameters.
Keeping scientific content separate from visual presentation makes it easier to update references, reuse visualizations, and adapt the same tools to different documentary subjects.
Episode Packaging
The output is a structured JSON-based episode specification that describes when visualizations appear, which assets they require, and how their behavior should be configured.
Each visualization is associated with a documentary timestamp or sequence, allowing the Unity application to synchronize spatial content with video playback.
This separates much of the content-authoring process from the Unity implementation, making it possible to reuse the same visualization systems across future episodes.
The structured episode format also provides a clear interface for future automation, allowing AI agents to work with predictable data rather than directly modifying Unity scenes.
Challenges I Faced
Making 2D Footage Meaningful in 3D
The goal wasn't simply to put a nature documentary on a floating screen with extra graphics around it.
The challenge was finding moments where spatial visualization could explain something better than conventional footage, whether that's the shape of a feeding structure, the trajectory of a wing, or the relationships within an evolutionary tree.
Not every scene benefits from a 3D representation. Part of the design process involved deciding when a spatial explanation adds meaningful information and when the documentary should speak for itself.
Translating Scientific Information Into Spatial Models
Research papers contain measurements, diagrams, observations, and descriptions, but that information doesn't automatically become a useful 3D visualization.
Different biological concepts require different representations. Migration patterns belong on a globe, evolutionary relationships belong in a branching tree, and biomechanics often require animated models.
This motivated my library-based approach, where each biological concept can be expressed through a specialized visualization system rather than requiring an entirely new 3D scene.
It also creates a clearer foundation for future AI integration, where an agent could select and configure existing spatial tools rather than attempting to generate arbitrary experiences.
Designing for Future AI Integration
One of the central technical challenges was designing a production workflow that could eventually support AI agents without depending on them for its core functionality.
A web-based authoring interface and a real-time mixed reality application have very different requirements.
Creating a structured episode format allows content and visualization parameters to be defined independently of Unity. The resulting JSON acts as a bridge between documentary content preparation and the final headset experience.
This also makes the system agent-ready. Future AI tools could generate or modify episode configurations through the same structured interface, while Unity remains responsible for rendering and interacting with the spatial content.
The challenge was designing visualization systems flexible enough to accept different scientific datasets while remaining predictable and performant inside Unity.
Balancing Scientific Detail With Documentary Storytelling
The scientific visualizations need to provide meaningful explanations without overwhelming the original footage.
I designed Phylum around the relationship between watching and exploring, allowing spatial explanations to complement the documentary rather than compete with it.
The spatial models are meant to expand on the filmmaker's storytelling, not interrupt or replace it.
What I Learned
Building Phylum has highlighted how much biological information exists beyond what documentary footage can communicate.
I also learned the value of designing systems for AI integration without making AI responsible for every part of the experience. A curated library of spatial visualizations with clearly defined inputs and capabilities provides a much stronger foundation for future automation than attempting to generate entirely new XR scenes from scratch.
Working with scientific material also reinforced the importance of choosing the right visual representation for a concept. A branching evolutionary tree, a geographic distribution map, and a biomechanical animation communicate fundamentally different types of information. The challenge isn't just generating 3D content, but finding the representation that makes the underlying biology easier to understand.
Most importantly, I found that reusable spatial tools and a structured production workflow make this approach practical beyond a single handcrafted episode.
Instead of manually developing an entirely new XR experience for every documentary, I can focus on expanding the library of biological visualizations and connecting them to new films.
As AI capabilities continue to improve, this architecture also creates opportunities to automate more of the production process while keeping the underlying spatial experiences consistent and reliable.
What's Next
The next step for Phylum is finding the right documentary creators to work with and expanding its agent-ready production capabilities.
The core spatial visualization systems and episode workflow are already in place. Future development will focus on integrating AI agents that can analyze documentary footage, identify biological concepts worth exploring, retrieve relevant scientific literature, and help configure spatial visualizations through Phylum Studio.
I'd also love to collaborate with independent nature and wildlife documentary filmmakers, particularly smaller creators who have produced beautiful footage and compelling stories but may not have the resources to build mixed reality experiences themselves.
Through Phylum, I want to showcase their existing films in a new format, preserving the original storytelling while adding spatial explanations that help audiences understand the biology behind what they're watching.
I'm especially interested in working with creators who focus on wildlife behavior, ecology, evolution, and the lesser-known details of the natural world.
Ultimately, I want Phylum to become a place where independent wildlife documentaries can reach new audiences, and where filmmakers can give their work another dimension without having to become XR developers.
