Inspiration

I’ve always been fascinated by how fast AI is changing developer workflows, but actually connecting local applications to modern open-weight models often feels surprisingly clunky. You either end up stuck behind regional access barriers or fighting with slow API setups. When I saw the Nebius x NVIDIA hackathon, I wanted to build something clean and practical: an AI assistant that developers can actually talk to for quick, reliable technical advice, powered by top-tier models without the usual setup headaches.

What it does

It’s an interactive AI technical assistant designed to help developers and data scientists answer complex coding questions on the fly.

You ask a technical question in a simple Streamlit interface—like how to set up async database sessions or write data pipelines.

The request gets passed to a FastAPI backend that formats the prompt and enforces proper structure.

The backend sends the query directly to the nvidia/Nemotron-Mini-4B-Instruct model, which stream-renders detailed explanations, code blocks, and best practices back to the screen.

How we built it

I kept the stack lean, asynchronous, and modular:

Frontend: Built with Streamlit so the interface stays clean, lightweight, and capable of rendering formatted Markdown and syntax-highlighted code.

Backend API: FastAPI handles incoming prompts asynchronously, using Pydantic models to validate the data before it ever hits the model logic.

Model Routing: Instead of running heavy local models or running into regional API blocks, the app routes requests to nvidia/Nemotron-Mini-4B-Instruct using the Hugging Face Serverless Router backed by Nebius infrastructure.

Challenges we ran into

The biggest hurdle was dealing with regional restrictions on direct model endpoints, which initially blocked API calls from my setup. Bypassing that required figuring out how to route requests through Hugging Face's serverless router while keeping the official NVIDIA Nemotron model at the core.

Another classic challenge was handling cold starts in shared GPU pools—when a serverless container sleeps, initial requests can lag or time out. I had to refine the backend error handling so the app stays resilient and doesn't crash when an endpoint takes a moment to wake up.

Accomplishments that we're proud of

Getting a full end-to-end pipeline running smoothly: Streamlit UI -> FastAPI backend -> Nebius/NVIDIA inference engine.

Structuring the FastAPI backend cleanly with Pydantic validation so it handles technical queries without breaking.

Building a reliable workaround for local API limitations without sacrificing access to official NVIDIA model architectures.

What we learned

Building this project really reinforced the importance of asynchronous design in web backends. I learned how to properly handle environment variables, structure clean API schemas, and navigate serverless GPU routing. It was also a great lesson in troubleshooting network/API constraints creatively rather than giving up when an endpoint rejects a direct connection.

What's next for Nebius x NVIDIA AI Agent Assistant

I’d love to take this beyond single-prompt Q&A. The immediate next step is adding multi-turn conversational memory, letting users upload full files (like .py scripts or SQL schemas) for automatic code review, and integrating custom search tools so the agent can fetch real-time documentation directly while generating answers.

Built With

  • fastapi
  • huggingface
  • nebius
  • nvidia-nemotron
  • pydantic
  • python
  • streamlit
  • uvicorn
Share this project:

Updates

Submission history