mova-contract-agent-demo

Humans speak in intent. Systems execute contracts.

Comment

Inspiration

Customer support teams process thousands of repetitive tickets every day. Many of these tickets describe the same problems: login errors, registration issues, interface confusion, or configuration mistakes.

Most existing solutions rely on either rigid workflows or free-form AI responses. Workflows are difficult to maintain and adapt when new scenarios appear, while AI-generated responses are often unpredictable and difficult to audit.

This project explores a different approach: using structured resolution contracts to combine the flexibility of AI with predictable and auditable execution logic.

What it does

The project demonstrates a multimodal support agent that processes customer tickets using both text and screenshots.

When a ticket is submitted, the system analyzes:

  • the message written by the user
  • the visual state of the interface from the screenshot

From this information the agent extracts structured signals that describe the problem context.

These signals are matched against a catalog of predefined resolution contracts.
Each contract describes how a specific class of issues should be handled, including:

  • required inputs
  • validation checks
  • allowed actions
  • possible outcomes

The system ranks available contracts and selects the most appropriate one.
The selected contract then executes validation checks and produces a resolution.

The output depends on the communication channel:

  • Live chat: the system produces a clear next action for the user.
  • Email: the system generates a ranked list of recommended solutions for the support team.

Every ticket execution is recorded as an episode, creating a full audit trail of how the issue was processed.

How we built it

The architecture consists of several stages:

  1. Multimodal signal extraction
    The system analyzes both the ticket text and the screenshot.

  2. Signal interpretation
    Structured signals are generated to describe the detected context.

  3. Contract selection
    Signals are evaluated against a catalog of predefined resolution contracts.

  4. Contract execution
    The selected contract runs validation checks and produces a resolution.

  5. Episode recording
    Each execution is stored as a structured record for auditing and analytics.

The application is deployed on Google Cloud Run, which allows the system to scale easily while maintaining a clear execution trace.

Challenges we ran into

One of the main challenges was designing a resolution model that is flexible enough for real support scenarios but still structured enough to be auditable.

Another challenge was combining multimodal input (text and screenshots) with deterministic execution logic. Extracting useful signals from screenshots while keeping the system predictable required careful design of the contract selection process.

Finally, it was important to ensure that the same architecture could support different communication channels such as live chat and email.

What we learned

This project shows that combining AI perception with structured execution logic can create more reliable support automation.

Instead of letting AI generate full responses directly, AI can be used to interpret context and select the appropriate structured resolution path.

This makes the system more predictable, easier to audit, and easier to extend with new contracts.

What's next

The next step is expanding the catalog of resolution contracts and applying the same approach to other business processes beyond customer support.

The contract-based model can potentially be used to describe many operational workflows in a structured and auditable way while still benefiting from AI-driven context understanding.## Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for mova-contract-agent-demo

Built With

Share this project:

Updates

Submission history