Fatoorah AI

Inspiration

Invoice intake is still surprisingly manual for small and mid-sized finance teams. A user receives a supplier invoice, takes a photo, checks totals, maps line items, opens an accounting system, fills fields, attaches the invoice, and saves a draft. The hard part is not only extraction; it is coordinating the messy human-in-the-loop process around extraction, review, mapping, posting, exceptions, and system-specific handoff.

We wanted to build something that felt practical for Saudi and regional businesses using tools like Qoyod and ERPNext. The original idea started around invoice capture and Qoyod automation, but as we explored the process we realized the real product was a dynamic finance Case. That made UiPath Maestro the natural center of the solution.

What it does

Fatoorah AI captures one or many invoice images/PDFs, extracts invoice data with LLMs, routes the result through finance review, and then creates draft invoices in connected systems.

Today it supports:

• Batch invoice capture from a local PWA that can be used from desktop or phone.
• Saudi FATOORA QR data as validation context when available.
• Intelligent AI Extraction: to parse multi-line items, handwritten details, and complex Arabic typography, instead of IXP when IXP is unavailable.
• Human review for supplier details, invoice metadata, totals, line items, mappings, and destination selection.
• ERPNext draft Purchase Invoice creation through the ERPNext API.
• Qoyod draft filling through a Chrome side panel extension using the user’s logged-in browser session.
• UiPath Maestro Case orchestration across capture, extraction, review, destination posting, Qoyod drafting, exception resolution, and closure.
• A Maestro dev log inside the app showing callbacks, stage movement, workflow details, and errors.

The system never stores Qoyod credentials and never approves/submits invoices. Everything ends as a draft unless a human decides the next step.

How we built it

We built Fatoorah AI as a backend-first, Maestro Case-ready application.

The main parts are:

• React/Vite PWA for batch upload, review, mapping, and status visibility.
• Express API for batch/job state, file handling, extraction orchestration, ERPNext posting, and Maestro callbacks.
• Intelligence Layer: Django receives the raw image bytes and coordinates with Google Gemini. Gemini extracts all individual line items, vendor details, and core totals..
• UiPath Maestro Case project with API Workflow tasks for stage orchestration.
• Orchestrator storage bucket and queue records for captured invoice artifacts.
• Chrome Manifest V3 side panel extension for Qoyod draft filling.
• Cloudflare temporary tunnels so UiPath Cloud and phone browsers can reach the local development app during demos.

We first explored a phone-only Chrome extension and UiPath IXP/RPA. Then reality shaped the architecture: no IXP access, no RPA runtime, and no Qoyod API access. Instead of blocking, we replaced IXP with LLM extraction, replaced RPA with a browser side panel, and kept Maestro as the real orchestration layer.

Challenges we ran into

The biggest challenge was keeping the system honest while the available platform pieces changed.

We initially planned for UiPath IXP and attended RPA, but did not have access to either. That forced us to redesign extraction and Qoyod filling while still preserving a path back to UiPath-native services later. Since we also did not have a commercial Document Understanding license, we completely bypassed traditional paid extraction engines and engineered precise, strict semantic prompts on Gemini instead. This ended up giving us better accuracy for local Arabic invoices than the generic extraction approach we originally expected to use.

We also had to decide whether this should be Maestro BPMN or Maestro Case. The process is full of exceptions: invalid QR data, OCR mismatches, missing mappings, duplicate invoices, ERPNext posting errors, Qoyod selector issues, attachment problems, and human corrections. That made Case Management a much better fit than a rigid process flow.

Validation created another challenge mid-build. At first, we compared line-item subtotals directly against the VAT-inclusive QR total, which created false mismatch errors. We fixed this by explicitly mapping tax weights before the final assertion phase, so the validation checks the correct relationship:

$$ \text{Computed with VAT} = \text{Subtotal} + \text{VAT Amount} $$

Local development with cloud orchestration was another real challenge. Maestro cannot call localhost, so we used temporary HTTPS tunnels. When a tunnel URL changed, live Maestro workers kept using the old URL that had been passed as the Case input. We fixed this by making the backend refresh tunnel configuration at runtime, adding tunnel sync scripts, and documenting that existing Case instances keep their original input URL.

We also had to make the extraction layer resilient to transient external API failures. Under heavy request loads, external AI endpoints can drop connections or return temporary 503 overload errors. We wrapped the extraction layer in an Exponential Backoff with Jitter retry mechanism, up to 5 attempts and capped at 30 seconds, so temporary overloads would not break the full invoice-processing flow.

We also hit frontend state issues. Maestro and the backend were moving forward correctly, but the web app stopped polling too early and appeared stuck in Capture Intake. We fixed the polling model so live Case activity keeps the UI current.

Finally, Qoyod filling was tricky without an API or RPA. We built a Chrome side panel, calibration flow, and content-script messaging model that uses the user’s authenticated browser session and keeps the user in control before saving a draft.

Accomplishments that we're proud of

We are proud that Maestro became a real part of the product, not just a diagram. The Case moves through stages, API Workflows call the backend, variables update, incidents are inspectable, and the web app reflects Maestro activity.

We are also proud of the pivots. When IXP and RPA were unavailable, we did not fake them. We built a working alternative with LLM extraction and a Chrome side panel, while keeping the architecture ready for future UiPath-native replacements.

Other wins:

• Batch intake instead of single-invoice-only flow.
• Human review before any ERP action.
• ERPNext draft creation working through live API calls.
• Qoyod draft filling without storing credentials.
• A visible Maestro dev log for demo transparency.
• Scripts for temporary tunnels, preflight checks, and live Maestro smoke tests.
• Clear failure states instead of silent automation failure.

What we learned

We learned that invoice automation is less about one perfect extractor and more about orchestration, state, and trust. Extraction can be wrong. Selectors can break. Mappings can be missing. External systems can reject records. The product becomes valuable when it can keep all of that visible and recoverable.

We also learned that Maestro Case is a strong fit for finance workflows where humans need to stay in control. It gives structure without pretending every invoice follows the same happy path.

On the technical side, we learned:

• Cloud Maestro needs public HTTPS callbacks, so local demos need disciplined tunnel management.
• Case start inputs matter; if a URL is passed into a Case, that Case keeps it.
• UI polling must follow orchestration state, not only local job status.
• Browser extension automation needs careful connection checks, calibration, and user confirmation.
• Building around real blockers produces a more credible demo than simulating access we do not have.

What's next for Fatoorah AI

Next we want to make Fatoorah AI more production-ready and more deeply UiPath-native.

Planned next steps:

• Add a UiPath Action App for finance review so Maestro owns the human task experience end to end.
• Add more robust Qoyod selector profiles and guided calibration.
• Support more ERP destinations and richer mapping rules.
• Add supplier/item/category suggestion agents.
• Improve duplicate detection across batches and accounting systems.
• Add audit-ready reporting for every Case stage, human decision, and draft creation.
• Deploy the backend to stable HTTPS infrastructure instead of temporary tunnels.
• Package the solution for repeatable UiPath deployment.

The long-term vision is a Maestro-powered finance intake layer where invoices can arrive from phone capture, email, WhatsApp, or folders, and every invoice becomes a governed Case that moves safely from capture to reviewed draft.

Built With

• django
• javascript
• llms
• maestro
• uipath