Revere XR prototype

Immersive Interactive Footprints:

Inspiration

We are inspired by the stories and rich history of the past. Seattle birthed, raised and launched the careers of music legacies that went on to shape music around the world. We were inspired to bring that history to life for all to experience creating a sense of pride and inspiration for current and future generations. Stories that strengthen our connections across vast spaces/distances, languages. technology allows us the opportunity to introduce that sense of belonging in tech. its true, we can uplift and share a new connection through experiencing history that inspires a new human story. Our inspiration stems from the vibrant tapestry of Seattle's cultural legacy, particularly its profound influence on music that resonates globally. Seattle nurtured and propelled iconic musicians whose impact reverberates through generations, fostering a rich history that embodies creativity, resilience, and innovation. Motivated by this heritage, we embarked on a mission to translate this legacy into a AI digital experience, inviting individuals of all backgrounds to immerse themselves in the city's cultural narrative.

At the heart of our endeavor lies the belief that AI technology serves as a bridge, transcending barriers of time, space, and language to foster a shared sense of belonging. Through leveraging AI technology, we endeavor to weave the threads of history into the fabric of the digital age, empowering users to not only witness but actively engage with pivotal moments that have shaped our collective identity. Our aim is to ignite a spark of inspiration, forging new connections and perspectives as we navigate the intersection of past and present.

In essence, our project seeks to revive and reimagine the past, inviting users to traverse the corridors of history, interact with its luminaries, and partake in its enduring legacy. By preserving and celebrating these invaluable memories, we aspire to cultivate a deeper appreciation for our history and propel it into the future for generations to come. Join us as we embark on a journey to honor our past, celebrate our present, and inspire a future illuminated by the timeless stories that define us.

What it does

we merged AI and extended reality to resurrect music legend Jimi Hendrix. Using NVIDIA GPUs and InWorld's capabilities, we created a custom AI NPC that embodies Hendrix's spirit, offering interactive experiences in a virtual world inspired by his era. This project features immersive, historically rich 3D environments, artistic visuals, and effects like smoke and light, set to Hendrix's music. The NPC, mimicking Hendrix, plays guitar, dances, and interacts with users, complete with eye contact, to deepen engagement. Our aim was to craft an experience that's not just visually striking but emotionally connected to Hendrix's legacy and his time, highlighted by psychedelic visual effects. Our demo allows you to interact a Seattle music legend, see him perform, ask him questions and learn about his history. This demo brought a historic guitar legend back to life where you can talk with him, ask him questions and hear him talk about how Seattle inspired him and his music journey. We trained him w/ AI so he could answer questions about his history and how his neighborhood and city inspired him. we can create immersive learning experiences that make history accessible in unprecedented ways, fostering a deeper connection to our past and a better understanding of the figures who have shaped our world.The technology to interact with holographic projections of historic figures has the potential to change the world in several profound ways, touching upon education, cultural preservation, and our collective memory. we merged AI and extended reality to resurrect music legend Jimi Hendrix. Using NVIDIA GPUs and InWorld's capabilities, we created a custom AI NPC that embodies Hendrix's spirit, offering interactive experiences in a virtual world inspired by his era. This project features immersive, historically rich 3D environments, artistic visuals, and effects like smoke and light, set to Hendrix's music. The NPC, mimicking Hendrix, plays guitar, dances, and interacts with users, complete with eye contact, to deepen engagement. Our aim was to craft an experience that's not just visually striking but emotionally connected to Hendrix's legacy and his time, highlighted by psychedelic visual effects.

How we built it

Our endeavor began with harnessing the power of machine learning techniques to breathe life into our virtual counterpart of Jimi Hendrix. Through meticulous training on a comprehensive dataset comprising Hendrix's speeches, writings, interviews, and archival materials, our AI was primed to emulate his mannerisms and cognitive patterns with uncanny precision. Advanced natural language processing (NLP) technologies facilitated seamless interactions, enabling users to engage in meaningful dialogue with our AI recreation.

The cornerstone of our development process was the utilization of Cloud XR, a cutting-edge platform that facilitated high-fidelity VR streaming, elevating the immersive experience to unprecedented levels. Coupled with the capabilities of InWorld AI for imbuing our digital rendition of Hendrix with depth and personality, we embarked on a journey that blurred the boundaries between past and present, weaving technology seamlessly into the fabric of our human narrative.

Unity served as our development platform, providing the canvas upon which we painted the vivid landscapes of Hendrix's era and choreographed his mesmerizing performances. Leveraging Eleven Labs for realistic voice synthesis and spatial audio for directional immersion, we crafted an experience that transcended mere entertainment, offering users a portal into the rich tapestry of music history.

This project stands as a testament to the transformative potential of AI and XR, extending beyond the realms of entertainment to emerge as powerful tools for education, cultural preservation, and interactive storytelling. By commemorating the legacy of Hendrix and other iconic music figures, we illuminate a path towards a future where technology serves as a bridge between past and present, enriching our understanding of history and inspiring generations to come. Tools Used:

NVIDIA Cloud XR: For high-fidelity VR streaming.
NVIDIA GPU
InWorld AI/AZURE: Facilitating AI training with depth and personality.
Unity: Development platform for VR world creation and animations.
Eleven Labs: Realistic voice synthesis for authentic Hendrix experience.
Spatial Audio: Enhancing immersion with directionally accurate sounds.
Community Insights: Incorporating historical perspectives to enrich AI interactions.

Azure: Inworld’s expertise in working with generative AI models for character development, Microsoft’s cutting-edge cloud-based AI solutions including Azure OpenAI Service, Microsoft Research’s technical insights. Our approach to AI is based on three principles: meaningful innovation, empowering people and organizations, and responsibility. creating responsible AI by design, building on Microsoft’s AI principles and Microsoft’s Responsible AI Standard. And we continue to bring intentional, inclusive designs to AI.

Our utilization of Cloud XR proved highly effective, culminating in a high-fidelity experience that redefines the boundaries of possibility. By reimagining Jimi Hendrix through the lens of modern technology, we have forged a path towards a future where the past is not merely remembered but vividly experienced, setting a new standard for interactive storytelling and educational exploration in the digital age.

Challenges we ran into

Throughout our journey, we encountered a myriad of challenges that tested the limits of our creativity and adaptability. Initially envisioning our project as an augmented reality (AR) experience utilizing NVIDIA Omniverse and Unreal Engine, we encountered significant hurdles with AR passthrough functionality, which threatened to derail our progress. With time constraints looming, we faced the daunting task of pivoting our approach to showcase our vision within a virtual reality (VR) framework.

The transition from AI+AR tovAI+VR was not without its obstacles, as we grappled with the intricacies of adapting our concept to a new medium. Leveraging Unity, we crafted a captivating mural scene that served as the backdrop for our immersive Hendrix experience, channeling our team's collective ingenuity to transform setbacks into opportunities for innovation.

One of our primary challenges arose from the limitations of the training data input for Eleven Labs, which hindered our ability to customize the AI's voice synthesis. Despite diligent efforts, we encountered persistent issues with accent discrepancies, a perplexing anomaly that defied conventional troubleshooting methods. Undeterred, we explored alternative solutions and brainstormed strategies to optimize the voice cloning process, determined to deliver an authentic and seamless user experience.

Moreover, integrating InWorld AI technology presented unforeseen complexities, as compatibility issues surfaced with other systems, necessitating meticulous adjustments to ensure seamless integration. While navigating these technical hurdles demanded patience and perseverance, our team's unwavering commitment to excellence propelled us forward, fueling our resolve to overcome every obstacle in our path.

Although our original vision of an AI+AR experience proved unattainable within the given constraints, our journey of adaptation and resilience culminated in the creation of a AI+VR masterpiece that transcends traditional boundaries. By embracing innovation and embracing the unexpected, we not only conquered adversity but emerged stronger and more resourceful than ever before.

In hindsight, our challenges served as catalysts for growth, pushing us to explore new technologies, refine our methodologies, and ultimately, deliver a product that surpasses expectations. As we reflect on our journey, we take pride in our team's unwavering spirit and collaborative ethos, demonstrating that with perseverance and determination, there are no obstacles too daunting to overcome.

Accomplishments that we're proud of

We are teleporting people to the past to experience moments they otherwise would not have access to. Proud isn't the word. The best way to describe it: In our endeavor, we're not merely crafting technology; we're infusing it with soul, ushering in a new era where innovation transcends the confines of mere functionality. Our journey delves beyond the realms of conventional achievement; it's a testament to our unwavering commitment to imbue the tech landscape with meaning and purpose.

While our project may not push the boundaries of technological extremity, it excels in its visual artistry and conceptual depth, offering a feast for the senses that resonates with community spirit and historical resonance. Our meticulous attention to aesthetic detail, from the vibrant hues that evoke the essence of the '70s to the psychedelic landscapes that dance with life, transports users to a realm where imagination knows no bounds.

Yet, amidst the visual spectacle lies a deeper narrative—a narrative woven with the threads of history and the fabric of technology. Through our innovative use of AI, we're breathing life into the legends of yesteryear, forging connections that bridge the chasm of time. By recreating iconic figures like Jimi Hendrix, we're not merely preserving history; we're redefining it, making it accessible and engaging for generations to come.

Our project represents more than a technological marvel; it's a gateway to a future where learning becomes an immersive journey, where students and curious minds can engage with the past in ways never before imagined. Through dialogue with AI-driven personas, users gain intimate insights into historical contexts, fostering a deeper understanding of the human experience.

As we reimagine Jimi Hendrix through the lens of modern technology, we blur the lines between past and present, weaving a tapestry where history and innovation intertwine. The use of Cloud XR for development catapults us into a realm of high-fidelity experiences, setting a new standard for immersive storytelling.

In essence, our project embodies the convergence of artistry and technology, weaving a narrative that resonates with the soul of humanity. It's a testament to our unwavering dedication to push the boundaries of what's possible, to inspire, to educate, and to illuminate the path forward in this ever-evolving landscape of innovation.

Infusing Soul into Technology: Our project transcends mere technological innovation; it imbues the digital realm with a sense of soul and meaning, elevating the tech experience to new heights.
Visually Compelling Aesthetics: We've meticulously crafted visually striking environments and VFX that captivate the senses, transporting users to the vibrant era of the 70s with a psychedelic vibe that's both exciting and immersive.
Solid Character Design: While our character art may not conform to conventional standards, we've crafted a persona that exudes charisma and authenticity, enriching the user experience with community and historical character.
Advanced Visual Effects: Pushing the boundaries of technology, our team has implemented cutting-edge visual shader effects and lighting techniques, setting a new standard for immersive experiences in the VR space.
Exciting Potential of AI: Our project showcases the transformative potential of AI in building believable interactive characters, sparking excitement for the future of AI-driven storytelling and historical recreation.
Recreating Historic Legends: By breathing life into iconic figures like Jimi Hendrix, we're revolutionizing the way history is experienced and understood, making it accessible and engaging for diverse audiences.
Impact Beyond Gaming: Beyond the realm of gaming, our project resonates with communities, educators, and history enthusiasts, showcasing the broader societal impact of immersive technology in preserving cultural heritage.
Transformative Learning Experience: Our use of AI enables users to engage in dialogue with historical figures, fostering a dynamic learning environment where history comes alive in a personalized and interactive manner.
Preserving Human History: Through the convergence of AI and historical personas, we're preserving and sharing the rich tapestry of human history, ensuring that the achievements and legacies of past generations endure for future ones.
Blurring Past and Present: Our project blurs the lines between past and present, seamlessly weaving technology into the human narrative, and inspiring a new generation to connect with their heritage in innovative ways.
Setting New Standards: With our high-fidelity VR streaming and innovative use of Cloud XR, we're setting a new standard for immersive experiences, pushing the boundaries of what's achievable in the realm of extended reality.

In essence, our project represents more than just a technological feat; it's a testament to the power of human creativity and ingenuity to reshape our relationship with history, technology, and each other. Through the seamless integration of AI, we're crafting a new chapter in the human story—one that's inspiring, engaging, and profoundly impactful for generations to come.

Visual Impact:

Imagine stepping into a world where history comes alive before your very eyes—a world where the past seamlessly intertwines with the present, creating a mesmerizing tapestry of sights and sounds that captivate the senses. Our project harnesses the power of art, VFX, AI, and imagination to bring history to life in the most compelling and profound way imaginable.

Picture this: as you walk down a bustling city street, you're surrounded by the sights and sounds of the past and present merging effortlessly. Historic buildings stand tall, their facades adorned with interactive, AI-enhanced murals that transport you back in time. These murals aren't static paintings; they're dynamic, alive with scenes from the city's rich history, populated by figures from different eras engaging in activities that once filled these streets.

But the experience doesn't end there. Along the sidewalks, you encounter "immersive footprints"—markers that, when activated through a device, lead you on a guided tour of the city's historical sites. With each step, you're immersed in narratives, reenactments, and significant events, experiencing the city's history firsthand as if you were there when it happened.

This seamless blend of AI technology and historical preservation isn't just educational; it's transformative. It redefines the way we engage with history, turning passive learning into an immersive and interactive experience that's irresistible to all. By interacting with simulated history in the real world, we're not just preserving the past; we're bringing it to life in ways that spark curiosity, inspire exploration, and ignite a passion for history in even the most skeptical of hearts.

In essence, our project is more than just a visual spectacle; it's a journey through time, a testament to the power of technology to bridge the gap between past and present, and a celebration of the stories that define who we are. Get ready to embark on an unforgettable adventure—one where history isn't just something you read about; it's something you experience firsthand, leaving you forever transformed into a history junkie hungry for more.

What we learned

Through this transformative experience, we've uncovered a profound truth: people are hungry for history beyond the confines of traditional gaming experiences. Our journey has shown us that by leveraging technology to bring history to life, we have the power to reshape how we perceive and value one another. By immersing ourselves in the past, we not only gain a deeper understanding of our shared heritage but also find inspiration to push the boundaries of the future.

Our project exemplifies the fusion of technology and storytelling, where augmented reality bridges the gap between past and present, transforming ordinary city streets into vibrant tapestries of historical narrative. By preserving historical sites and stories in this innovative manner, we ensure that they remain integral parts of our urban landscape, vibrant reminders of our collective past.

The marriage of AR with historical preservation not only offers an educational experience but also captivates and engages users in a way that is both tangible and immediate. Imagine walking through a bustling city street, where every building tells a story and every mural comes to life with the touch of a screen or the donning of AR glasses. It's an immersive journey through time, where history becomes more than just a lesson; it becomes an experience to be lived and cherished.

Our use of AI to recreate historical legends, such as Jimi Hendrix, adds an extra layer of depth and authenticity to our project. By bringing these figures to life with flair and character, we elevate the immersive experience to new heights, captivating audiences beyond the confines of the gaming sphere.

Ultimately, our project isn't just about technology; it's about bringing soul and meaning to the tech landscape. It's about pushing the boundaries of what's possible and inspiring communities to embrace their history in innovative ways. As we showcase the impact of our project on the community and the broader hackathon audience, we're confident that its power to transcend gaming and inspire real-world connections will resonate deeply with Microsoft and position us as true innovators in the field.

What's next for Revere XR prototype

Transformative Exploration Educational Experiences:

Interactive Learning: By bringing historic figures to life, this technology can revolutionize education, making it more engaging and interactive. Students can engage in conversations with historical figures, asking questions and receiving answers as if they were interacting with them in person. This immersive experience ignites a passion for learning and makes history feel relevant and alive.
Empathy and Understanding: Interacting with figures from different cultures and eras fosters empathy and a deeper understanding of diverse perspectives. It brings to life the context behind historical events, enabling learners to grasp the human aspect behind the facts and develop a more nuanced understanding of history.
Keeping History Alive: This technology serves as a powerful tool for cultural preservation, ensuring that the stories and lessons of the past are preserved for future generations. By making historical interactions accessible, it prevents valuable cultural heritage from being lost and keeps history alive in an engaging and immersive manner.
Global Accessibility: With virtual and augmented reality experiences being distributable globally, this technology democratizes access to cultural education. It brings important historical and cultural figures to audiences worldwide, including those who may not have access to museums or historical sites, thereby broadening access to cultural education.

Enhanced Storytelling and Entertainment:

Revolutionizing Entertainment: In the realm of entertainment, this technology opens up new avenues for storytelling. Historical figures can actively participate in films, plays, and other performances, creating a new genre of historical documentaries and educational content that is both informative and engaging.
Personalized Experiences: Individuals can have personalized experiences with historical figures, whether it's through learning specific skills, exploring an era's culture, or understanding historical events from a first-person perspective. This customization enhances engagement and deepens understanding.

Social Impact and Awareness:

Promoting Diversity and Inclusion: By including a diverse range of figures, this technology promotes inclusivity and representation. It allows the stories of marginalized or lesser-known figures to be told, providing a more comprehensive view of history and promoting diversity and inclusion.
Fostering Global Connection: Learning about historical figures from around the world fosters a better appreciation for different cultures. It promotes a sense of global connection and mutual understanding, breaking down barriers and fostering collaboration on a global scale. As we gaze into the future, we envision a world where history isn't just a subject to be learned but an experience to be lived. At Revere XR, our vision extends beyond music history to encompass pivotal moments in sports, entertainment, cultural milestones, and beyond. Our goal is to broaden horizons and foster a deep reverence for the past, creating a world where the legacies of our heroes and legends educate and unify us all. The fusion of AI and AR with historical preservation represents a seismic shift in how we explore and engage with our collective heritage. Picture a dynamic cityscape where the past seamlessly intertwines with the present. Historic buildings, meticulously preserved, stand as silent witnesses to bygone eras. Yet, with the magic of augmented reality, these structures come alive as interactive canvases for immersive murals, transforming the city into a living museum.