Eventis

VR Conferences, Concerts, & Events powered by Streamspace

Comment

Inspiration

The COVID-19 pandemic highlighted the importance of virtual events, conferences, and even concerts. However, we all know the limitations of current virtual event platforms—they often feel… underwhelming.

They don’t come close to replicating the energy or excitement of in-person experiences. In addition, many platforms are centralized, making them prone to failure. All it takes is one power outage or a few too many people past server capacity (nwhacks moment), and it's all over.

That’s where Eventis comes in! Our decentralized platform, built in partnership with Streamspace, reimagines virtual conferences, events, and concerts. Eventis leverages immersive virtual reality to make events feel real again and includes a friendly, locally-run LLM assistant with Retrieval-Augmented Generation (RAG) capabilities to guide users and enhance their experience.

Our solution is globally accessible, supporting social and environmental sustainability efforts by reducing the need for travel and physical infrastructure. Plus, with its built-in monetization options, organizers can ensure their events are financially sustainable too.

What it does

Eventis is a decentralized platform that creates immersive and interactive 3D environments for virtual conferences, events, and concerts. It enables users to:

  • Stream and store event content securely using the Streamspace platform.
  • However, we do plan on moving past just video, and would like to incorporate image, 3D environment, and whatever your heart desires.
  • Explore beautifully rendered 3D event spaces, including stages, booths, and networking areas.
  • Interact and network with other people in the same space
  • Interact with a locally run LLM assistant that answers questions, provides session summaries, or connects participants.
  • Experience live concerts or conferences without needing to travel, as if they were the real thing.

How we built it

We made use of the following technologies in this project:

  • Unity: Built an immersive 3D event space, including a virtual auditorium, stage, audience area, and interactive zones.
  • Streamspace: Integrated decentralized video storage and streaming to ensure data ownership and seamless content delivery.
  • Locally Hosted Llama 3.2 LLM: Used a lightweight (3B params), locally hosted LLM to enable real-time Q&A and personalized interactions.
  • Meta Quest 3: Used a virtual reality headset, made to maximize immersion and engagement. VR headsets are getting cheaper and cheaper - best tap into the market as soon as possible!
  • Python RAG API with FAISS: We process and store information in a vector database on our decentralized servers. (our LLM is still local, however)

Challenges we ran into

  • Integrating Streamspace was certainly the most difficult part of this journey - it was quite challenging to use a brand new API! However, we cooperated with Streamspace to make it work.
    • Spending 7 hours on making a stream work is crazy though
  • Our solution had a lot of moving parts, making it difficult to ensure everything worked.
    • The host needs to read the input stream and store the information in a vector database
    • The client needs to call an API to get this information
    • The client needs to process the user’s speech and gather their query
    • The client needs to feed the database info and the user’s query into the LLM
    • The LLM needs to output a valid answer and display it properly
  • The sheer amount of Unity programs, streaming, and facilitating API calls was quite taxing on my computer. Oftentimes I would run out of memory, mistaking it for a bug.
  • As usual, the time limit of 24 hours forced us to manage our time effectively (and pull an all-nighter)
  • Not losing my sanity

Accomplishments that we're proud of

  • Getting streampace to work within Unity
  • Sharpening my skills with VR development in Unity
  • Improving my experience with LLM’s and using RAG for the first time
  • Not breaking down in stress or giving up
  • Came out with, more or less, a functioning prototype! (for now, we will see during judging)

What we learned

  • How to use Streamspace (and in general, the workflow of streaming APIs)
  • A further understanding of RAG and LLMs in general
  • Working solo in a hackathon is certainly one of the experiences of all time

What's next for Eventis

  • Cooperate with Streamspace to host a greater variety of media, such as images, 3D environments, etc…
  • Expand the functionality, such as breakout rooms and customizable stages
  • Make an interface where anyone can easily make custom events
  • Improve scalability and performance for larger audiences through better decentralized video streaming optimization
  • Introduce monetization features, such as NFT-based event tickets and advertising spaces
  • Collaborate with event organizers to pilot Eventis for real-world conferences and concerts.
  • Explore AR/VR integrations for an even more immersive experience.

Built With

Share this project:

Updates