Nova Auditor

Multimodal AI that monitors your inventory, flags shortages, and verifies every shipment in real time

Comment

Inspiration

Inventory mismanagement is one of the most costly and overlooked problems in modern operations. Parts run out without warning, shipments arrive with wrong quantities, and data entry errors go unnoticed until it's too late. I wanted to build something that could autonomously monitor inventory health and physically verify incoming shipments — turning a reactive process into a proactive one. Amazon Nova's multimodal reasoning capabilities made this possible in a way that wasn't feasible before.

What it does

Nova Auditor is an AI-powered inventory management agent that:

  • Monitors stock levels by querying a MySQL database and identifying parts below their safety threshold
  • Reasons about reorder quantities using Amazon Nova Lite, suggesting exactly how many units to order to reach the threshold
  • Lets users approve orders directly from the dashboard, saving approved orders to the database
  • Verifies incoming shipments by analyzing photos of physical shipping labels using Nova's vision capabilities
  • Cross-references shipments against approved orders, flagging mismatches in part name, quantity, or vendor

The result is a closed-loop agentic workflow: detect shortages → approve reorders → verify delivery.

How we built it

The stack consists of three main layers:

Backend:

  • Python + FastAPI as the API framework
  • AWS Lambda for serverless deployment via Mangum
  • Amazon RDS (MySQL) for inventory and orders storage
  • Amazon Bedrock with us.amazon.nova-lite-v1:0 for AI reasoning and vision

Frontend:

  • Next.js 16 with TypeScript
  • Tailwind CSS for styling
  • Custom markdown parser to render Nova's structured responses
  • File upload interface for shipment label verification

Agent Flow:

$$\text{Low Stock Query} \rightarrow \text{Nova Reasoning} \rightarrow \text{Human Approval} \rightarrow \text{Order Saved} \rightarrow \text{Shipment Image} \rightarrow \text{Nova Vision} \rightarrow \text{Accept / Reject}$$

The agent uses two of Nova's core capabilities in sequence: natural language reasoning over structured database data, and multimodal vision to read and interpret physical shipping labels.

Challenges we ran into

  • Lambda deployment size limits — packaging all dependencies exceeded the 70MB direct upload limit and later the 250MB unzipped limit. We solved this by removing packages already available in the Lambda runtime (boto3, botocore) and deploying via S3.
  • CORS configuration conflicts — having both FastAPI middleware and Lambda Function URL CORS settings active caused duplicate headers that browsers rejected. Disabling CORS at the Lambda URL level and letting FastAPI handle it exclusively resolved the issue.
  • Nova image format validation — Nova strictly validates image MIME types. Hardcoding jpeg caused failures for PNG uploads. We fixed this by dynamically detecting the file type from file.content_type at runtime.
  • Quantity extraction from AI responses — Nova's responses use markdown bold formatting (Suggested Reorder Quantity: 20) which broke our initial regex. We iteratively refined the pattern to handle markdown syntax correctly.
  • IAM key compromise — AWS automatically attached a quarantine policy after credentials were accidentally exposed in logs, blocking all deployments. We rotated the access keys and removed the quarantine policy to restore access.

Accomplishments that we're proud of

  • Built an agentic multi-step workflow where Nova reasons across both structured data and unstructured visual input
  • Successfully deployed a full-stack application on AWS Lambda + RDS with a live Next.js frontend
  • Nova correctly identifies shipment discrepancies by cross-referencing vision output against database records in real time
  • The approve order flow creates a human-in-the-loop checkpoint, making the agent trustworthy for real-world use

What we learned

  • How to integrate Amazon Bedrock and Nova's multimodal API including the correct message format for combining image and text inputs
  • How to structure a serverless FastAPI application with Mangum for Lambda deployment
  • The importance of always returning JSON from APIs — plain text error responses cause cascading failures in frontend JSON parsers
  • How AWS IAM policies interact, specifically that an explicit Deny overrides any Allow regardless of how many permissive policies are attached
  • How to write regex patterns that handle markdown formatting in AI-generated text

What's next for Nova Auditor

  • Vendor price scraping — automatically visit vendor URLs to pull live pricing and availability before the user approves an order
  • Automated reorder scheduling — use AWS EventBridge to run audits on a schedule and send email alerts via AWS SES when stock hits critical levels
  • Analytics dashboard — track stock level trends over time with charts showing the most frequently reordered parts and average lead times
  • ERP integration— connect Nova Auditor to existing enterprise systems like SAP or QuickBooks to automatically sync approved orders without manual data entry

Built With

Share this project:

Updates