ExtraHorizon
Rvqh-NTya-Ua8g-DXdg-Dwnu-2TdL - code for website
Inspiration
The original idea behind ExtraHorizon came from something we have been thinking about for a long time: modern AI systems are incredibly powerful, but most of them still feel emotionally distant.
They can understand words, generate code, analyze information, and solve complex problems, but they rarely feel like they actually understand the person they are talking to.
We had experimented with character-driven AI systems before, but most of those projects were focused mainly on entertainment. They were fun, expressive, and sometimes unpredictable, but they did not provide enough real-world utility.
For this hackathon, we wanted to change that.
We asked ourselves:
What if an AI assistant could be both useful and human-aware?
Instead of choosing between a serious productivity tool and an AI with personality, we wanted to combine both.
That idea became ExtraHorizon.
The name represents the core philosophy of the project: helping people see what exists beyond the horizon, information, connections, emotions, problems, and opportunities that may otherwise remain invisible.
And we built the first working version in essentially one night and one full day.
We have worked on large technical projects before, but ExtraHorizon pushed us into a completely different territory because it combines AI, computer vision, voice interaction, maps, external data, and real-time communication inside a single system.
What it does
ExtraHorizon is an AI-powered platform built around Rika, a context-aware assistant that can listen, speak, understand emotional signals, work with real-world data, and help users discover connections between problems that would otherwise be difficult to notice.
The goal is not simply to create another chatbot.
The goal is to create an AI system that understands more of the context surrounding a conversation.
Emotion-aware AI
One of the main features of ExtraHorizon is real-time facial emotion recognition.
When camera mode is enabled, the system analyzes the user's facial expression and estimates their current emotional state.
That information can then be translated into contextual metadata and injected directly into the AI prompt.
Instead of receiving only:
"Explain this problem to me."
Rika can receive additional contextual information indicating that the user may appear confused, happy, frustrated, surprised, or focused.
This gives the language model another signal it can use when deciding how to respond, not just what to respond.
The important distinction is that emotion recognition does not replace the user's words. It provides additional context to help the assistant adapt its communication style.
Meet Rika
The AI assistant inside ExtraHorizon is called Rika.
Originally, we planned to call her Azi, based on one of our previous AI experiments. During development, however, we decided that this project deserved its own identity.
That is how Rika was born.
The name was inspired by the word "річка" / "rechka", meaning river in our language.
We liked the idea because water and infrastructure became surprisingly important themes during development.
One of ExtraHorizon's major features analyzes infrastructure projects involving utilities such as water systems, sewer systems, road work, and other public construction projects, identifying projects that may be geographically or temporally close to each other.
Just like rivers connect locations, Rika connects information.
Infrastructure intelligence
ExtraHorizon is not limited to conversation.
One of the most practical parts of the project is its infrastructure analysis system.
Using publicly available information, ExtraHorizon can help compare planned technical and construction projects and identify situations where work may overlap.
For example, two utilities may be planning projects:
- close to the same location,
- on nearby streets,
- During approximately the same time period,
- or involving infrastructure that could potentially benefit from coordination.
The system can visualize this information directly on a map using OpenStreetMap.
This allows users to move beyond reading individual project records and instead understand how projects relate spatially.
The idea is simple:
Data becomes much more useful when you can see the relationships between it.
Potentially overlapping projects can then be surfaced to engineers, planners, or other users for further investigation.
ExtraHorizon does not decide whether two projects should be coordinated. Instead, it helps humans discover connections that may be worth examining.
Voice interaction
We wanted communication with Rika to feel natural rather than requiring constant typing.
ExtraHorizon, therefore, includes a complete voice interaction pipeline.
The system can listen to the user through speech recognition, convert their speech into text, process the request through the AI system, and generate a spoken response.
Rika's voice is generated through Fish Audio using the Drama-3 voice model.
Audio generation is integrated through a WebSocket-based architecture so that responses can move through the system with minimal interruption.
We also implemented a tag-based voice system.
These tags allow different information to be attached to generated text so the voice pipeline can better understand how a particular response should sound or when certain audio behaviors should be triggered.
This gave us much more control over Rika's personality than simply sending plain text into a TTS model.
AI architecture
At the center of ExtraHorizon is a streaming language-model pipeline.
The system processes conversations through a streaming GPT-based core, allowing generated tokens to begin moving through the rest of the pipeline before the entire response has been completed.
This was extremely important for voice interaction.
Without streaming, the system would need to:
- wait for the user to finish speaking,
- wait for the entire AI response,
- send the full response to TTS,
- wait for audio generation,
- finally begin playback.
That creates an interaction that feels slow and artificial.
Instead, ExtraHorizon was designed around streaming data wherever possible.
Rika also uses a dedicated system prompt that defines her personality and behavioral rules. This allows her to remain expressive while still functioning as a practical assistant.
The combination of the language model, system prompt, emotional context, voice system, and external information gives Rika her identity.
External knowledge and developer intelligence
ExtraHorizon can also interact with information from multiple public sources.
The project uses or works with data from sources including:
- GitHub
- Stack Overflow
- Miami-Dade public data
- OpenStreetMap
This makes Rika useful beyond normal conversation.
For example, developer-related questions can be supported with information from software-development ecosystems, while infrastructure questions can use public local datasets and geographic visualization.
One of our goals was to avoid creating isolated AI features.
Instead, we wanted all of these systems to become tools available to the same assistant.
How we built it
ExtraHorizon is built as several independent modules connected through a common AI pipeline.
The webcam system continuously extracts visual information when enabled.
Rather than sending raw camera footage into the primary language model, a smaller specialized emotion-recognition model processes the facial information locally and produces a simplified emotional state.
This result can then become part of the context sent to Rika.
We used an existing lightweight open-source model from GitHub as the foundation of the facial emotion recognition component and integrated it into our own pipeline.
For geographic visualization, we integrated OpenStreetMap.
For voice generation, we integrated Fish Audio.
The rest of the system connects these components into a single interface so that users do not need to think about which model or service is currently being used.
They simply interact with Rika.
The biggest challenge
Without question, one of the hardest parts of the entire hackathon was facial emotion calibration.
It sounded relatively straightforward at the beginning.
It was not.
Different lighting conditions, camera positions, facial angles, subtle expressions, and image quality could dramatically affect the output of the emotion model.
A system that correctly detected an expression in one situation could suddenly produce completely different results several minutes later because the user's head had moved or the room lighting had changed.
We spent hours adjusting and testing the camera pipeline. WE HATE THAT!!!
We repeatedly tested expressions, modified preprocessing, changed thresholds, experimented with calibration, and tried to make the recognition system stable enough to provide useful information to Rika.
Our goal was not simply to make emotion recognition work in ideal conditions.
We wanted it to remain usable under normal webcam conditions where lighting and positioning are far from perfect.
We eventually managed to make the system significantly more reliable under different conditions.
There is, however, one enemy we still have not defeated:
darkness.
If the camera cannot properly see your face, Rika cannot magically see it either.
And yes... debugging this throughout the night was a nightmare.
We are never forgetting that night.
Another challenge: making everything work together
Individually, many of the components behind ExtraHorizon already exist in some form:
speech recognition exists, language models exist, TTS exists, mapping systems exist, public datasets exist, and emotion-recognition models exist.
The difficult part was making all of them behave like one product.
A delay in one system affects everything after it.
A bad emotion classification can change the AI context.
A slow AI stream delays speech.
Incorrect parsing can prevent external data from appearing correctly.
A map can contain useful information but still be useless if the user cannot immediately understand what is being displayed.
The challenge became less about building individual features and more about creating a stable pipeline between them.
That was one of the most valuable parts of this project.
What we learned
ExtraHorizon taught us that building an AI product is very different from simply connecting an API to a text box.
The quality of the experience depends heavily on everything surrounding the model:
- latency,
- context management,
- streaming,
- UI design,
- voice generation,
- computer vision,
- external tools,
- error handling,
- and how information moves between components.
We also learned how important specialized models can be.
A large language model does not need to do everything itself.
A smaller computer-vision model can understand facial expressions. A speech model can handle audio. A mapping system can handle geographic information. External datasets can provide factual context.
The language model can then act as the layer that connects these capabilities together.
That architecture makes ExtraHorizon more than a chatbot.
It makes it a platform.
What we are proud of
The thing we are most proud of is how much of the system we managed to bring together during such a short period of time.
In roughly one night and one day, we built a project that combines:
- real-time AI conversations,
- AI personality,
- speech recognition,
- streaming responses,
- neural text-to-speech,
- webcam-based emotion recognition,
- dynamic prompt context,
- interactive geographic visualization,
- public infrastructure data,
- developer knowledge sources,
- and multiple external services.
More importantly, these features are not presented as disconnected demonstrations.
They are all connected through Rika.
What's next
ExtraHorizon started as a hackathon project, but the underlying concept can go much further.
We want to make Rika increasingly context-aware without making the interaction intrusive.
Future versions could integrate additional specialized tools for engineers, developers, planners, teams, and organizations while keeping the same simple interaction model.
You should not need to learn ten different interfaces to use ten different AI tools.
You should be able to explain what you need.
And your assistant should understand the rest.
Why ExtraHorizon?
For us, ExtraHorizon represents the idea that AI should not only generate answers.
It should help people notice things.
It should notice context.
It should notice connections.
It should notice when information from completely different sources suddenly becomes relevant to the same problem.
And, when possible, it should understand a little more about the human on the other side of the screen.
That is what exists beyond the horizon.
ExtraHorizon helps you see it.
Built With
- chatgpt
- claude
- cloudflare
- openai
Log in or sign up for Devpost to join the conversation.