Nova Sync-Master

Bridging legacy data to autonomous web actions with Nova Act's reliable reasoning and human-in-the-loop safety.

Comment

Inspiration:The "Inaccessible" Challenge

The digital world is divided into two halves: structured data trapped in legacy formats like Excel and dynamic actions locked behind web portals that lack modern APIs. In enterprise environments, teams often waste thousands of hours on manual data entry—a repetitive, error-prone process. I was inspired to create Nova Sync-Master to bridge this gap. My goal was to build an agent that could "read" an offline spreadsheet and "act" on the web with the same intuition as a human, solving the "programmatically inaccessible workflow" problem identified in the Amazon Nova Act User Guide.

What it does:Nova Sync-Master is a "Data-to-Action" agent designed to automate complex enterprise workflows by bridging the gap between static, offline data and dynamic web portals. It specifically targets "programmatically inaccessible" processes where traditional APIs are unavailable, and manual data entry is the only current solution.

Core Functionalities:

Intelligent Data Extraction: Automatically parses legacy Excel files to ingest pending order data, transforming unstructured spreadsheets into actionable JSON inputs.

Autonomous Web Action: Utilizes the Amazon Nova Act foundation model to navigate web interfaces naturally. Instead of relying on brittle UI selectors, it uses multi-step reasoning to find and interact with form elements to process orders.

Cross-System Synchronization: Once a web task is complete, the agent captures confirmation details and synchronizes them back to a local SQLite database, ensuring a closed-loop record-keeping system.

Human-in-the-Loop (HITL) Oversight: To maintain enterprise-grade security and accuracy, the agent identifies critical decision points and pauses for a human supervisor to approve the transaction before committing final updates.

How we built it:Phase 1: Data Architecture & ExtractionWe began by creating a synthetic data pipeline. Using pandas and openpyxl, we developed generate_mock_data.py to simulate a real-world enterprise environment where pending orders are stored in a local Excel file. This structured data serves as the "source of truth" for our agent.Phase 2: Local Tool IntegrationTo bridge the gap between the agent and the local system, we leveraged the Strands Agents framework. We built two primary local tools:Excel Tool: For reading and parsing tabular data from .xlsx files.Database Tool: A persistence layer using SQLite to record the agent's successes, including unique confirmation IDs extracted from the web.Phase 3: The Agentic OrchestratorThe core logic resides in main.py, where we initialized the Amazon Nova Act session. Unlike traditional automation, we didn't write CSS selectors. Instead:We defined a high-level instruction (e.g., "Process these orders through the portal").The agent entered an agentic loop, where it independently determined the necessary Steps to navigate complex UI.We used Builder Mode in the IDE extension to visually debug the agent's multi-step reasoning in real-time.Phase 4: Human-in-the-Loop (HITL) ImplementationTo ensure the project was production-grade, we manually integrated a Human Approval pattern. We used a terminal-based blocking prompt that pauses the workflow execution. This ensures that the agent completes the "heavy lifting" of web navigation but requires a human "supervisor" to verify the data before the final database commit.Phase 5: Efficiency ModelingTo measure our success, we calculated the Efficiency Multiplier ($M$) based on the time saved per order processing:$$M = \frac{T_{manual}}{T_{agent} + T_{approval}}$$Where $T_{manual}$ is the estimated time for a human to process the batch and $T_{agent}$ is the automated execution time. Our testing showed that even with the $T_{approval}$ delay, the agent increased throughput by over 400%.

Challenges we ran into:Building a bridge between offline data and legacy web portals presented several unique technical hurdles. Here are the primary challenges we overcame during the development of Nova Sync-Master:Navigating the "Chaos" of Legacy UI: Many target portals utilize nested IFRAMEs and non-semantic HTML, which are notorious for breaking traditional automation. We had to refine our instructions to leverage the Nova Act foundation model's multi-step reasoning, moving away from hardcoded selectors to let the agent "see" the page layout as a human would.State Persistence and Synchronization: Coordinating data across three different environments—an Excel file, a live browser session, and a local SQLite database—required precise timing. We solved this by creating a robust Workflow that uses the agent's success logs to trigger database updates only after a confirmed web action.Managing "Agentic Drift": In complex workflows, an agent can sometimes get "distracted" by irrelevant UI elements or site pop-ups. We addressed this by implementing Step-by-Step constraints and descriptive tool definitions, ensuring the agent remained focused on the primary task within its session.Secure Credential Handling: To protect sensitive enterprise data, we had to ensure that no API keys or credentials were leaked during the agentic loop. We implemented a strict environment variable strategy using .env files and boto3 sessions to adhere to the AWS Shared Responsibility Model.Quantifying Reliability: Measuring the performance of an autonomous agent is more complex than a standard script. we developed a custom metric called the Action Success Rate ($ASR$) to track reliability:$$ASR = \frac{S_{success}}{S_{total}} \times 100$$where $S_{success}$ represents the number of successfully completed acts and $S_{total}$ represents the total attempted acts. Optimizing this required multiple iterations of our prompt engineering strategy.

Accomplishments that we're proud of:For our hackathon submission of Nova Sync-Master, we are particularly proud of several key technical and strategic milestones that demonstrate the power of the Amazon Nova Act ecosystem:

Bridging the "Inaccessible" Gap: We successfully built a functional end-to-end bridge between static, offline enterprise data (Excel) and dynamic web portals that lack traditional APIs.

Implementing Agentic Reasoning: Rather than relying on "brittle" hardcoded UI selectors, our project leverages the agentic loop, allowing the Nova foundation model to observe the page state and determine its own Steps for high-reliability automation.

Safety-First Design with HITL: We successfully integrated a Human-in-the-Loop (HITL) pattern, ensuring that the agent remains an assistant by requiring a manual "supervisor" approval before any critical data is committed to the local database.

Sophisticated Tool Integration: We pushed the boundaries of the SDK's preview features by integrating local tools for both Data Extraction (via pandas) and Persistence (via SQLite), creating a complete "Data-to-Action" pipeline.

Enterprise-Ready Architecture: While we optimized for rapid prototyping using an API Key, our codebase is fully structured for production-scale deployment on the Amazon Bedrock AgentCore Runtime (ACR) with secure AWS IAM authentication.

What we learned:Building this project was a journey from traditional, script-based automation to Agentic Reasoning. Here is what we learned throughout the process:Reliability Over "Brittle" Scripts: We learned that traditional automation often fails due to minor UI changes. Nova Act achieves high reliability because it is a foundation model specifically trained for UI-forward applications. By using an agentic loop, the model observes the page and takes sequential actions, adapting naturally to layout variations.The Critical Role of Human-in-the-Loop (HITL): For enterprise-scale workflows, pure autonomy is a risk. We learned how to implement HITL patterns—specifically Human Approval—to provide supervisors with decision points. This ensures that critical actions, like updating a database, only occur after manual verification.Expanding the Agent's "Senses": We discovered that the most powerful AI agents operate beyond the browser. Nova Act’s ability to integrate external tools allowed us to connect our agent to local files and databases. We learned to use the @tool decorator to provide the model with the necessary context to decide when to use local logic versus web navigation.Security as a Shared Responsibility: Implementing the project taught us the importance of the AWS Shared Responsibility Model. While AWS protects the infrastructure, we learned to secure our agent by using environment variables for credentials and implementing state guardrails to restrict the agent to authorized domains.Performance ModelingTo quantify our success, we modeled the efficiency of the Nova Sync-Master against manual processing. If $T_{manual}$ represents the time taken by a human and $T_{agent}$ represents the agent's processing time, the Automation Efficiency Ratio ($E$) is:$$E = \frac{T_{manual}}{T_{agent} + T_{approval}}$$Our testing showed that even with the time taken for human approval ($T_{approval}$), the agentic approach achieved an efficiency ratio of over 4.0, effectively quadrupling the throughput of the "programmatically inaccessible" workflow.

What's next for Nova Sync-Master:The vision for Nova Sync-Master involves moving from a specialized local tool to a scalable, enterprise-grade autonomous fleet. Here is our roadmap for future development:

  1. Cloud-Native Scaling with Amazon Bedrock The current prototype runs locally for development, but the next step is to containerize the workflow for deployment on Amazon Bedrock AgentCore Runtime (ACR). This will allow the agent to scale automatically based on demand, moving from a single session to hundreds of concurrent Workflow Runs without local hardware constraints.

  2. Managed Human Intervention Service (HIS) We plan to upgrade our terminal-based HITL to the Managed Human Intervention Service. This will allow the agent to send professional approval notifications via Slack, email, or a custom dashboard. This transition ensures that human supervisors can approve or reject actions from any device, improving the speed of the Data-to-Action pipeline.

  3. Multi-Agent Orchestration We aim to integrate our agent with the Strands framework to participate in multi-agent systems. In this setup, a "manager" agent could delegate specialized web tasks to Nova Sync-Master, which would handle the UI interaction and report back, allowing for more complex, cross-departmental automation.

  4. Advanced Observability and Security To make the agent production-ready, we will implement:

Enhanced Tracing: Using the AgentCore Observability tools to monitor every Step, Act, and Session for detailed performance auditing.

Customer-Managed Keys (CMK): Implementing KMS encryption for all agent trajectory data to meet strict enterprise compliance standards.

  1. Multimodal Data Ingestion Finally, we plan to expand the agent's "eyes" beyond Excel. By using the multimodal capabilities of the Amazon Nova model family, the agent will be able to "read" scanned PDFs, hand-written invoices, and even screenshots to trigger web actions, truly fulfilling the vision of an all-purpose data bridge.

Built With

Share this project:

Updates