Helios

• 

Inspiration

Managing cloud infrastructure at scale often means juggling countless AWS services — EC2, S3, CloudWatch, IAM, EKS, and more — each with its own operational nuances. As an AI/ML Engineer and AWS Cloud Architect, I wanted a smart, self-serving agent capable of answering operational questions using existing runbooks and architecture documentation — instead of manually searching through Confluence or wiki pages. This vision inspired Helios, an agentic AI co-pilot that acts like an on-demand AWS SRE assistant powered by Retrieval-Augmented Generation (RAG) and NVIDIA NIM inference APIs.

What it does

Helios intelligently answers AWS operations and troubleshooting questions by combining:

A local FAISS vector index of internal technical runbooks.

NVIDIA Retrieval Embedding NIM (nv-embed-qa-e5-v1) to vectorize and retrieve the most relevant content.

Llama-3 1-Nemotron-Nano-8B-v1 NIM LLM to synthesize context-aware, human-like answers.

An agentic reasoning loop orchestrated via FastAPI and LangChain/LlamaIndex patterns.

For example, if you ask:

“How do I troubleshoot slow API responses in my AWS Lambda?” Helios retrieves related sections from your internal runbooks and provides a concise, contextualized answer — complete with AWS-specific diagnostic steps.

How we built it

The project architecture blends modern MLOps, cloud engineering, and agentic AI design principles:

Language model (LLM): NVIDIA NIM Llama-3 1-Nemotron-Nano-8B-v1

Embeddings model: NVIDIA Retrieval Embedding NIM (nv-embed-qa-e5-v1)

Vector store: FAISS-CPU (cosine similarity)

Frameworks: FastAPI, LangChain / LlamaIndex

Containerization: Multi-stage Docker build for lightweight images

Orchestration: Kubernetes manifests (Deployment, Service, Secret) deployed on Amazon EKS

CI/CD: (Optional) GitHub Actions for build & deploy

Challenges we ran into

Latency optimization: Initial inference through NIM endpoints required careful batching and async I/O management.

FAISS dimensionality alignment: Ensuring embedding vector sizes matched FAISS index configuration.

Token and context management: Balancing retrieval chunk sizes vs. prompt token limits.

Security & DevOps: Injecting NVIDIA API keys securely into Kubernetes secrets without exposing them in logs or code.

Agent reasoning loops: Designing deterministic yet adaptive reasoning paths to avoid LLM hallucinations.

Accomplishments that we're proud of

Built a fully containerized and production-deployable RAG system from scratch.

Achieved sub-2-second average query latency for most AWS queries with local FAISS retrieval.

Successfully integrated NVIDIA NIM services with FastAPI microservices and EKS infrastructure.

Created a reusable, open-source RAG boilerplate for future domain-specific AI co-pilots.

What we learned

The importance of hybrid intelligence: combining deterministic retrieval with generative reasoning.

Efficient model-in-the-loop architectures for cloud-native environments.

Practical use of vector databases and embedding scaling for enterprise documentation.

How to design agentic loops that reason transparently without losing control.

Deepened understanding of NVIDIA NIM inference patterns, FAISS optimization, and AWS EKS orchestration.

What's next for Helios

Integrate AWS SDK Agents for real-time cloud inspection (e.g., EC2 status, CloudWatch metrics).

Add multi-modal RAG for parsing configuration files and diagrams.

Fine-tune lightweight local models for offline inference.

Introduce Helios Dashboard — a web UI for monitoring query analytics and performance.

Extend to Azure and GCP to become a multi-cloud AI co-pilot.

Built With

• aiofiles==23.2.1
• faiss-cpu==1.7.4
• fastapi==0.99.0
• httpx==0.24.1
• numpy==1.26.2
• pydantic==1.10.12
• python-multipart==0.0.6
• uvicorn[standard]==0.22.0