Inspiration

Developers working with AI today need to make a difficult choice between using a light-weight model for fast response times ($\text{latency} < 1\text{s}$) or using a huge frontier model for complex reasoning but at the expense of speed and computational cost. Additionally, client-side web applications often leak private API keys in network traffic. This inspired us to develop KERNEX, an all-in-one high performance AI copilot utilising the NVIDIA Nemotron range of models through the Nebius Token Factory API. This removes the need to make this choice by incorporating a smart routing system which dynamically routes requests to the appropriate model complexity, whilst being secured through a zero-trust backend proxy.

What it does

KERNEX intelligently routes each user prompt through an intelligent three-tier system:Nano tier: ultra-fast classification, very fast C/C++ code generation, and fast querying.Ultra tier: deep logical processing, system restructuring, and complex optimisation of code.Super tier: complex multi-modal processing (image and diagram understanding), and high level reasoning. Generation speed is monitored in real-time ($\text{tok/s}$), and only clean code is generated with no comments. The code is written in standard ASCII format ($\text{!=}$ rather than $\neq$).

How we built it

KERNEX is built using a modern full stack architecture based round NVIDIA's leading AI models. The system works as follows: Routing Engine: Prompts are passed through Nemotron-3.5 Nano/Nemo ($\sim 30\text{B}$ MoE, 3B parameters) which calculates the complexity of the prompt, this acts as an inline classifier ($C(x)$). Dynamic Routing Based on the mathematical complexity of the prompt($$f(x) = \begin{cases} \text{Nemotron-3.5 Nano/Nemo}, & C(x) \le \theta_{\text{low}} & \text{(Simple Queries / Rapid Code)} \ \text{Nemotron-3 Ultra}, & \theta_{\text{low}} < C(x) \le \theta_{\text{high}} & \text{(System Logic / Refactoring)} \ \text{Nemotron-3.5 Super-VL}, & C(x) > \theta_{\text{high}} & \text{(Complex Vision / Multi-Step Reasoning)} \end{cases}$$) Frontend & Telemetry: Built using React, TypeScript and TailwindCSS. Live token throughput data is streamed from the frontend. Calculated inside the browser ($$T_{\text{rate}} = \frac{\Delta \text{Tokens}}{\Delta t} \quad (\text{measured in tok/s})$$). Security A Python/Flask backend proxies the NEBIUS_API_KEY in the backend environment variables. This ensures that there is no exposure of any credentials.

Challenges we ran into

Strict Code Output Rules: Enforcing that the LLM will only output compilable C/C++ code with the appropriate headers($\text{#include } \langle \text{stdio.h} \rangle$). Multimodal Data Handling: Passing Base64 encoded image data cleanly through our Flask security proxy to Nemotron-3.5 Super-VL. Cross-Origin Request Handling: Handling CORS requests when passing requests between React front-end and local/cloud based Flask proxies.

Accomplishments that we're proud of

Developed functional 3 tier model routing system on Nebius Token Factory Implemented comment free C code generation to clipboard Secure $100\%$ keys using zero trust proxy server architecture Integrated live token telemetry including speed of token generation ($\text{tok/s}$) and overall volume

What we learned

Optimizing compute resources by dynamically adjusting compute complexity $C(x)$ to model parameters rather than overprovisioning. Using MoE models such as NVIDIA Nemotron to generate responses in less than a second. Best practice for securing API proxies in AI web applications.