Vektre Labs

Inspiration

I recently read Matthew Ball's book, The Metaverse, which was a massive inspiration for this hackathon's project. The book was released prior to the emergence and virality of popular AI chat bots like GPT-3 or Claude. Many of the roadblocks to VR's rapid commercialization of the technology outside of hardware constraints feel very solvable with AI agents embedded within VR, AI tools for improved developer experience, modernized libraries, a bridge to the traditional web via connectors, and tapping further into the social potential of immersive technologies like VR.

What it does

Within your very own, customizable VR workspace or workspace, you and your AI agent companion can collaborate on websites and applications through voice commands. With all the connectors software engineers and AI agents utilize on a daily basis being managed by the vocal AI agent, you can have a more entertaining and gamified development experience that allows you to build more rapidly because you are no longer constrained to one coding agent and the amount of monitors you have. Now, you can orchestrate multiple agents in parallel with their avatars without having to know anything about that infrastructure.

How we built it

I used WebXR, a vocal agent service called VocalBridge, and OpenRouter along with some other VR-related packages. There's also a general Next.JS website for handling things like account management.

Challenges we ran into

It was very difficult getting the AI model to place entities in the VR space and trigger actions or commands from my voice dialogue with the AI agent. It took a lot of orchestration across multiple APIs and services.

Accomplishments that we're proud of

I got an AI to be able to interpret a spatial environment, hold coherent conversations, make edits to a website, and render them in the environment.

What we learned

The VR ecosystem's libraries are fairly dated and have not caught up to the rest of the web in terms of libraries, support, and integration across multiple APIs and layers. So, a lot of things had to be built custom unfortunately, but perhaps this provides an opportunity to create some really foundational libraries for VR development in the age of AI.

What's next for Vektre Labs

Continuing to build this out to completion for an MVP, establishing pilots and mentorship through the VRARA. I want to attract the companies that used Meta Horizons Workspaces when it was a live service and see if I can bring them here. I firmly believe services like Meta Horizons failed because of timing primarily and lack of engineering focus. They were trying to do it all at once, control the entire stack. That is a recipe for failure. It would be like if Microsoft or Apple decided they would be the only ones allowed to make applications and software on their operating systems.

Built With

• firebase
• nextjs
• openrouter
• python
• typescript
• vocalbridge