PactPulse AI

5
6
4
2
1
3

Inspiration

Large-scale commercial infrastructure overhauls—such as retrofitting the 415V 3-Phase electrical distribution and Dante AV grid at New Delhi's Bharat Mandapam (IECC)—are paralyzed by the Contractor Escrow Dilemma:

Owners & Clients are terrified of disbursing funds prematurely before technical compliance is certified.
Subcontractors & Specialist Trades operate on razor-thin margins and routinely face payment delays of 45 to 90 days, or risk unfair chargebacks after work is delivered.
Emergency Bottlenecks ("Break-to-Buy"): When an MCCB switch or earth pit lead fails during commissioning, the entire job site freezes for days waiting for bureaucratic purchasing approvals, triggering cascading critical-path delays.
Data Privacy & Insecure Clouds: Defense, government, and critical infrastructure facilities strictly forbid uploading sensitive high-voltage schematics, substation layouts, or internal jobsite photos to public commercial clouds.

We asked: What if payment didn't wait on bureaucracy, but instead autonomously flowed the instant physical, tamper-proof proof was verified by a completely private, on-premise AI?

That vision became PactPulse AI—an autonomous escrow coordinator and agentic purchasing engine that pairs PayPal's sandbox payment rails with 100% Private Local AI (Ollama), real-time IoT grid telemetry, and on-site cryptographic proof of liveness.

What it does

PactPulse AI coordinates multi-party commercial projects through an autonomous, proof-gated financial pipeline:

Cryptographic Escrow Authorization:
Before work begins, the project owner locks milestone capital (in INR ₹) into escrow via the PayPal Orders & Authorizations API (intent=AUTHORIZE). Funds remain securely locked in the client's account under a 72-hour honor window: $$ \text{Escrow Hold} \ge \text{Milestone Value} \quad (\text{e.g., } ₹2,80,000 \text{ INR}) $$
Tamper-Proof On-Site Inspection (Live Webcam + Nonce):
Subcontractors open the on-site inspection workstation. To eliminate pre-recorded video fraud and photo re-use, PactPulse issues an ephemeral cryptographic challenge code (e.g., PP-9104-ELEC) valid for 15 minutes. The technician holds up the challenge code inside the Live Device Webcam Feed (getUserMedia). The app stamps the frame with live millisecond clock sync and GPS coordinates $ (28.6189^\circ\text{ N}, 77.2425^\circ\text{ E}) $.
100% Private On-Premise AI Verification (Ollama):
Inspection frames are evaluated locally on port 11434 with zero data leaving the host machine:
- moondream (Local Multimodal Vision): Inspects the image to confirm the physical challenge card in frame, checks 3-phase RYB wiring color codes against Indian Standard IS 732, and verifies earth pit connections.
- llama3.2 (Local Structured Reasoning): Operates with native format: "json" enforcement to score acceptance criteria against IS 3043 (earthing resistance $ R_{\text{earth}} < 1.0 \, \Omega $) and executes a deterministic 3-tier release policy based on confidence score $ S_{\text{proof}} \in [0, 100] $:

$$ \text{Action}(S_{\text{proof}}) = \begin{cases} \text{Instant PayPal Capture & 25\% Payout Split}, & S_{\text{proof}} \ge 85 \ \text{Human Reviewer Queue Escalation}, & 60 \le S_{\text{proof}} < 85 \ \text{Actionable Punch-List Rejection}, & S_{\text{proof}} < 60 \end{cases} $$

Instant Release: Triggers programmatic PayPal Capture (/v2/payments/authorizations/{id}/capture) and automatically disburses a guaranteed 25% direct split payout (₹3,12,500) to the subcontractor via the PayPal Payouts API.
Human Review Queue: Diverts borderline submissions to the Owner Reviewer Queue with visual discrepancy notes.
Actionable Punch-List: Rejects non-compliant work with pinpoint instructions (e.g., "Phase B busbar torque seal missing").

"Break-to-Buy" Agentic Hardware Procurement:
When emergency hardware fails on-site, the contractor snaps a photo of the burnt component. Ollama diagnoses the part locally; Channel3 API locates distributor inventory; and PactPulse verifies the pre-approved corporate spending mandate: $$ \sum_{i=1}^{n} \text{Cost}(P_i) \le M_{\text{limit}} \quad (₹35,000 \text{ emergency ceiling}) $$ The replacement is auto-funded via PayPal, and the Bryntum Gantt schedule recalculates critical-path dependencies dynamically.
Dispute Defense & AG Grid Immutable Ledger:
Every transaction, cryptographic nonce, GPS coordinate, and Ollama reasoning token is recorded to an immutable AG Grid financial ledger. If an unfair chargeback is filed, PactPulse compiles a complete legal dispute defense dossier in one click.
Dedicated Role Dashboards:
Tailored workspaces for 🏛️ Owner (Client) (capital preservation, reviewer queue, dispute defense), ⚡ Lead Contractor (Bryntum schedule, Break-to-Buy purchasing, live IoT telemetry), and 🔧 Subcontractor (live camera proof workstation, direct PayPal split payouts).

How we built it

We built PactPulse AI as a single-port, full-stack application running on http://localhost:3000:

Frontend: React 18, TypeScript, and Vite styled with a unified Velvet Red Wine (#24040A) & Milky Porcelain (#FFFDF9) palette.
On-Site Proof Capture: HTML5 MediaDevices.getUserMedia() capturing 60 FPS video streams, canvas frame extraction, and browser Geolocation API.
Backend Architecture: Node.js, Express, TypeScript, and Multer for multipart image handling.
Financial Rails: PayPal REST SDK (v2 Checkout Orders, Authorizations, Captures, and Payouts) running against the PayPal Sandbox with PayPal-Request-Id idempotency headers.
Private AI Engine: Ollama running locally on port 11434 (moondream for multimodal vision inspection + llama3.2 for JSON-structured compliance evaluation). Zero cloud vision exfiltration.
Data Grid & Ledger: AG Grid Community for multi-party transaction tracking, column sorting, and dispute audit trails.
Commerce Engine: Channel3 API integration for industrial distributor SKU search and automated purchase orders.
Project Orchestration: Bryntum-inspired Reactive Gantt engine for dynamic schedule recalculations.
API Safety: APIMatic type adapters for compile-time PayPal payload schema validation.

Challenges we ran into

Eliminating Camera Roll Fraud & Replay Attacks:
Simple computer vision can be easily fooled by pointing a camera at an old photo on a tablet. We solved this by generating an ephemeral, single-use cryptographic challenge nonce that the technician must physically hold inside the camera frame. The local AI explicitly confirms the presence and text of the nonce before scoring technical criteria.
Guaranteeing Strict Jobsite Privacy:
Commercial engineering projects contain proprietary single-line diagrams, high-voltage panels, and secure facility layouts. Cloud AI APIs pose major confidentiality risks. We solved this by architecting the entire multimodal inspection pipeline to run locally via Ollama, decoupling intelligence from third-party cloud vendors.
Multi-Party Escrow State Machine:
Coordinating Client $ \rightarrow $ Lead Contractor $ \rightarrow $ Subcontractor disbursements required strict atomic guarantees. We implemented idempotent PayPal captures that prevent double-debits even if network interrupts occur during inspection processing.
Harmonizing High-Density UI with Industrial Telemetry:
Mixing a deep Red Wine and Milky Cream palette while maintaining high contrast across complex AG Grid financial tables, live camera streams, and Bryntum Gantt timelines required custom Tailwind CSS design tokens and theme variables.

Accomplishments that we're proud of

Zero-Dispute Speed: Built an end-to-end loop where payments release from live camera capture to confirmed PayPal capture in under 4 seconds ($ \Delta t < 4\text{s} $).
100% Private On-Premise Inspection: Successfully ran multimodal vision (moondream) and structured reasoning (llama3.2) on local hardware via Ollama with zero cloud leakage.
Subcontractor Protection: Built an automated 25% direct split payout via the PayPal Payouts API, ensuring trades receive guaranteed compensation the moment their work is certified.
Real Hardware Interactivity: Judges can switch to the Subcontractor dashboard, activate their actual laptop webcam, generate a challenge code, snap a live frame, and watch the local AI evaluate compliance in real time.
Autonomous Break-to-Buy: Demonstrated real agentic parts procurement where hardware diagnosis, mandate verification, and schedule adjustments happen autonomously.

What we learned

Private AI is mandatory for industrial infrastructure: Enterprise clients refuse to send critical facility photos to external cloud APIs. Local models like moondream and llama3.2 via Ollama provide production-level accuracy while guaranteeing complete data sovereignty.
Financial automation requires deterministic guardrails: Pairing AI confidence scores ($ S_{\text{proof}} \ge 85\% $) with PayPal's cryptographic authorization holds (intent=AUTHORIZE) eliminates financial risk while removing manual payment friction.

What's next for PactPulse AI

Smart Glasses & Bodycam Integration: Porting the inspection workstation to wearable displays (Apple Vision Pro, RealWear) and inspection drones for high-altitude stadium trussing and high-voltage substations.
Direct Enterprise ERP Connectors: Building native webhook synchronizers for SAP S/4HANA, Autodesk Construction Cloud, and Procore.
Dynamic Escrow Re-balancing: Machine-learning algorithms that adjust milestone authorization buffers dynamically based on historical contractor reliability and supply chain volatility.

Built With

ag-grid
agentic-workflows
autonomous
bryntum-gantt
canvas-api
channel3-api
computer-vision
escrow-infrastructure
express.js
fintech
gemini-3.8-flash
google-genai-sdk
indian-standards-(is-732)
iot-telemetry
multer
multimodal-ai
node.js
paypal
paypal-sandbox
react
rest-apis
tailwind-css
typescript
vite
webrtc-/-mediadevices

Updates

Somnath Pal started this project — Nov 04, 2026 10:57 AM EST

Leave feedback in the comments!