APPEARA

The Spark

How Appeara initialized

DOOM SCROLLING.
SLEEPING.
MARATHONING.
MATCHA.

We've all become the same person.

The same habits.
The same life.

How boring.

As we spend more and more time consuming digital content, we noticed that many games are becoming increasingly repetitive and predictable. We wanted to create something that would challenge players to think creatively instead of simply following a predetermined path.

This is what created Appeara.

We wanted to bring back imagination and creativity into gaming by giving players the ability to create the weapons they actually imagine.

Powered On and Creative

What does Appeara do?

Appeara is a VR survival game where players use their imagination to create weapons and defend the heart of a space station.

Let's talk features

⭐ Weapon Creator

Players can describe and sketch almost any weapon they can imagine. OpenAI interprets their input and transforms it into a structured WeaponSpec, which Unity uses to create a balanced and playable weapon.

Players have a limited number of weapons they can create, meaning they have to choose their ideas wisely.

⭐ Adaptive Enemies

The enemies in Appeara adapt to the weapons players create.

After analyzing a player's weapon, the Mothership develops defensive and tactical countermeasures. Once an enemy has adapted, the player has to creatively modify their weapon to find a new way to defeat it.

If the player doesn't change their strategy enough, they lose.

⭐ Dynamic Audio

ElevenLabs gives every invention its own sound and personality.

When a player creates a weapon, ElevenLabs generates a custom firing sound based on its behavior. ARIA introduces new weapons, while the Mothership reacts with dynamically generated dialogue and taunts.

Nothing is prerecorded, meaning every battle can sound different depending on what the player creates.

YouTube tutorial, Google Search, Code, Repeat

How we built Appeara

The majority of the technology we used for Appeara required us to learn how to connect multiple systems together in a short amount of time.

We built Appeara in Unity, combining VR, natural language processing, generative AI, procedural gameplay, dynamic audio, and real-time enemy adaptation.

For the weapon creator, we used OpenAI to combine the player's speech, sketches, and battle context into a validated WeaponSpec. Unity then uses this information to create a playable weapon.

OpenAI also generates holographic blueprints and constrained parts lists, allowing Unity to reconstruct the weapon as an original 3D object.

For enemy adaptation, the alien Mothership analyzes the player's weapon and generates defensive and tactical countermeasures.

Codex became our development teammate throughout the hackathon. It helped us implement interconnected gameplay systems, debug rendering, scaling, animation, and targeting issues, and create unit tests. We also used Codex to implement the Hive Avatar rig and bring its movement, visuals, and gameplay behavior together.

ElevenLabs was used to generate custom weapon sounds and dynamic character dialogue based on what was happening in the game.

Sentry gave us visibility into the game's entire runtime pipeline through tracing, structured logs, error monitoring, and performance profiling.

It helped us pinpoint where microphone input stopped flowing through the voice-to-weapon pipeline and diagnose why enemies stopped spawning during certain waves.

Tracing also revealed that counter generation was finishing several seconds after the 20-second enemy counter timer. We fixed this by starting the LLM counter analysis immediately when a weapon was fabricated and holding the result until the timer expired.

Performance traces exposed frame-time spikes when enemies and projectiles were repeatedly created and destroyed, leading us to implement object pooling to reduce overhead.

The Urge to Break our Laptops

The challenges we ran into

The main challenge we ran into while building Appeara was getting all of our different systems to communicate with each other in real time.

We were working with VR, Unity, natural language processing, AI-generated weapons, dynamic audio, enemy adaptation, and multiple external tools. Connecting all of these systems together while working under a hackathon deadline was challenging.

One of our biggest challenges was making sure the weapons generated by OpenAI were not only creative, but also structured in a way that Unity could understand and turn into playable objects.

We also ran into issues where microphone input stopped flowing through our voice-to-weapon pipeline and where enemies stopped spawning during certain waves. Using Sentry, we were able to trace these issues and identify where our systems were breaking.

Another challenge was timing. Our LLM counter generation was sometimes finishing several seconds after the 20-second enemy timer. We solved this by starting the analysis as soon as the weapon was created and storing the result until it was needed.

We also experienced performance issues when enemies and projectiles were repeatedly created and destroyed. After investigating our performance traces, we implemented object pooling to reduce the overhead.

The Top of the Hill

What we're proud of!

Throughout our process of creating Appeara, we accomplished many things. We managed to combine VR, Unity, OpenAI, ElevenLabs, Codex, and Sentry into one playable experience within a limited hackathon timeframe.

One of our biggest accomplishments was creating a weapon system where players can use their own imagination to create something that actually becomes part of the game.

We're also proud of our adaptive enemy system. Instead of simply making enemies stronger, the enemies respond to the player's creations and force them to creatively adapt their weapons.

We struggled with many technical issues throughout development, but we were able to research, debug, and work together as a team to get our systems working.

In the end, we managed to connect all of our ideas into one experience that challenges players to think creatively while having fun.

Sunday Morning

What we learned

We learned how to work collaboratively while building a project that required many different technologies to communicate with each other.

We learned how to use OpenAI to turn natural language into structured game data, ElevenLabs to create dynamically generated audio, Codex to help develop and debug our systems, and Sentry to monitor and understand what was happening inside our game.

We also learned that building with AI is about more than simply prompting a model. We needed to think about structured outputs, validation, timing, reliability, and how AI-generated information would interact with a real-time game.

Most importantly, we learned that creativity can become a gameplay mechanic.

Instead of giving players a fixed list of weapons, we gave them the ability to create their own.

What's next for Appeara

Appeara's end goal is to continue developing our idea into a larger gaming experience.

We plan to add more weapons, enemies, environments, puzzles, and ways for players to interact with the world using their imagination.

We also want to explore multiplayer gameplay, where players can create their own weapons and use them against other players, as well as expand our voice and drawing mechanics.

Our future as hackers does not stop with Appeara. We want to continue experimenting with AI, VR, and game development to create experiences that encourage people to think creatively instead of simply consuming content.

We want players to stop scrolling and start creating.