NovaPilot

Your AI co-pilot for intelligent automation and multimodal understanding.

Comment

About NovaPilot

What Inspired Us

We wanted to build something that makes Amazon Nova on AWS approachable—whether you have an AWS account or not. Many hackathon and learning projects stall at “configure credentials” or “enable model access,” so we were inspired to:

  • Lower the barrier: A demo mode lets anyone run the app with zero AWS setup or cost. You can ingest documents, ask questions, and see retrieval in action before touching the cloud.
  • Showcase real Nova: When you’re ready, flipping one flag and adding credentials switches to real Amazon Nova Lite for chat and Nova embeddings for retrieval—the same stack you’d use in production.
  • Keep the scope focused: One clean flow—upload or paste text → chunk & embed → ask questions with grounded, cited answers—so the “agentic + multimodal” idea is easy to see and extend.

The result is NovaPilot: a small web app that demonstrates Multimodal Understanding (documents + Q&A with citations) and can be extended toward Agentic AI (tools, planning) on top of Nova.

What We Learned

  • Bedrock API shape: Nova text models use a messages-v1 schema with system and messages; response text lives under output.message.content[].text. Embedding models vary by provider; we handle common response keys (embeddings, embedding, vectors) so switching models is straightforward.
  • RAG without a vector DB: For a lightweight demo, we avoided external services. We chunk text with overlap, call a single embedding API (or a deterministic mock in demo mode), and do cosine similarity in memory with NumPy. Persistence is simple (e.g. JSONL). Scaling to millions of chunks would mean moving to a proper vector store—but the retrieval logic (embed query → score → top-(k)) is the same: [ \mathrm{sim}(\mathbf{q}, \mathbf{c}) = \frac{\mathbf{q} \cdot \mathbf{c}}{|\mathbf{q}| \,|\mathbf{c}|}. ]
  • Graceful degradation: By defaulting to demo mode when DEMO_MODE is not set (or set to 1), the app always runs. No credentials ⇒ no failed Bedrock calls ⇒ a smooth first-run experience. Turning on real Nova is an explicit opt-in.

How We Built It

  1. Stack: FastAPI (Python) for the backend, Jinja2 for the single-page UI, boto3 bedrock-runtime for Nova, pypdf for PDF text extraction, NumPy for vector similarity. No frontend framework—vanilla JS and a simple dark theme so the focus stays on the flow.

  2. Core pipeline:

    • Ingest: User uploads a PDF or pastes text → we split into overlapping chunks (e.g. 1200 chars, 200 overlap) → each chunk is embedded (Nova embed or mock) → chunks and vectors are stored (e.g. in data/chunks.jsonl).
    • Ask: User question is embedded → we retrieve top-(k) chunks by cosine similarity → we build a context string with [source=... id=...] and pass it plus the question to Nova Lite with a system prompt that says “ground answers in the context and cite sources.”
    • Demo mode: If DEMO_MODE=1 (or unset), we skip Bedrock entirely: mock text returns a short message that echoes the question; mock embeddings are deterministic (hash-based) so retrieval still behaves and the UI is unchanged.

  3. Configuration: A single .env (or env vars) drives region, model IDs, demo mode, and data directory. We load .env with a minimal parser so we don’t require python-dotenv for the core path.

  4. Extensibility: The same ingest/retrieve/answer pipeline can later drive agents (e.g. “summarize,” “extract action items”) or Nova Act-style automation by replacing the final “one-shot answer” with tool use and multi-step reasoning.

Challenges We Faced

  • Credentials and first-run experience: Early testers hit “Unable to locate credentials” or “InvalidClientTokenId” before they could see the app. We addressed this by defaulting to demo mode and adding clear error messages that point to aws configure or DEMO_MODE=0 when they want real Nova. We also documented the easiest path: create one access key in the console → aws configure → set NOVA_EMBED_MODEL_ID in .env.

  • Embedding API diversity: Bedrock embedding models don’t all use the same request/response shape. We implemented a small adapter that tries common keys (embeddings, embedding, vectors) and raised a clear error for unknown schemas so we could add new models without breaking the pipeline.

  • Port and environment: On some Windows setups, the default port was already in use or required elevation. We documented an alternate port (e.g. 8002) and made sure the app binds to 127.0.0.1 so it runs without opening the firewall.

  • Balancing “no AWS” vs “real Nova”: We wanted both a zero-setup demo and a production-like path. The solution was a single demo_mode flag that swaps Bedrock calls for mocks while keeping the rest of the code (chunking, storage, retrieval, UI) identical—so the story is “same app, turn on AWS when you’re ready.”

We hope NovaPilot makes it easy to see what Nova-powered document Q&A looks like and to extend it toward agentic or multimodal workflows on AWS.

Built With

Share this project:

Updates