YardSight

• 

  Hero

• 

  ANPR Scanner

• 

  cctv matrix

Inspiration

Industrial freight depots, intermodal terminals, and manufacturing logistics centers handle thousands of heavy vehicle turnarounds daily under high operational stress. Inbound logistics gates suffer 45-minute bottlenecks due to manual paper manifest checks, perimeter CCTV cameras capture license plates at extreme 30°–50° angles where standard OCR fails completely, and hazardous chemical or diesel leaks on slick depot concrete trigger critical OSHA violations before anyone notices.

We built YardSight AI to transform ordinary perimeter and gate cameras into an autonomous logistics nerve center. By combining an Alexa+ Model Context Protocol (MCP) agent with Ring perimeter monitoring, yard managers and logistics dispatchers can converse naturally with the yard's camera matrix, automatically verify e-Way bills and electronic BOLs, and trigger immediate physical containment for hazardous spills.

What it does

• Alexa+ Spatial-Temporal MCP Agent: A self-hosted Model Context Protocol (MCP) server running on Streamable HTTP. Terminal supervisors can use conversational voice commands via Alexa+:
  • "Alexa, which Schneider rig is currently at Loading Bay 3, and are any chemical hazards detected?"
  • "Alexa, check if any unmanifested tankers entered through the South Gate in the last hour."
• Ring Perimeter & Access Control Intelligence: Ingests live perimeter feeds, tracking inbound truck approaches, gate clearance status, and driver credential verification.
• OpenCV 4.10 Perspective Homography ANPR: Normalizes severe 30°–50° oblique camera angles using a 4-point projective transformation matrix (H), producing orthogonal, crystal-clear character strips for dual-market recognition:
  • United States: USDOT commercial carrier formats and state plates (TX 942-WKY, CA 8ABC123).
  • India: High Security Registration Plates (HSRP) with state & RTO district codes (MH 12 RN 4589).
• Specular Concrete Hydrocarbon Sheen Classifier: Leverages thin-film optical dispersion modeling to distinguish dangerous diesel/hydraulic oil leaks from benign rainwater puddles, automatically sizing required granular absorbent boom poundage under OSHA 1910.120.
• Multi-Channel Emergency Dispatch & Audit Certification: Automatically notifies yard supervisors via WhatsApp Cloud API webhooks and generates cryptographically sealed jsPDF inspection reports.

How we built it

• Alexa+ / MCP Architecture: Built on the 2025-11-25 MCP specification exposing schema-validated endpoints (/api/mcp/execute) to allow autonomous AI agents to query live yard state and invoke physical actuators.
• Computer Vision Pipeline: OpenCV 4.10 homography normalization and Canvas 2D chromatic variance calculation.
• AWS Backend Integration: AWS Bedrock multi-modal vision reasoning for cross-camera temporal scene understanding.
• Frontend & Real-Time Video: React 19, TypeScript, Vite, Tailwind CSS v4, Lucide Icons, and WebRTC streaming.
• Automated Verification: 23/23 passing automated pytest test cases covering homography transformations, hydrocarbon sheen classifiers, and MCP tool execution.

Challenges we ran into

• Severe Camera Yaw & Pitch: Computing accurate 4-point homography matrices in real time without introducing character blur required precision edge detection and bilateral filtering.
• Thin-Film Specular Variance: Accurately distinguishing rainbow-sheen oil slicks from wet concrete reflections under shifting sunlight required fine-tuning chromatic dispersion thresholds.
• Latency Optimization: Maintaining smooth multi-camera WebRTC streaming while concurrently processing real-time MCP agent tool calls.

Accomplishments that we're proud of

Industrial freight depots, intermodal terminals, and manufacturing logistics centers handle thousands of heavy vehicle turnarounds daily under high operational stress. Inbound logistics gates suffer 45-minute bottlenecks due to manual paper manifest checks, perimeter CCTV cameras capture license plates at extreme 30°–50° angles where standard OCR fails completely, and hazardous chemical or diesel leaks on slick depot concrete trigger critical OSHA violations before anyone notices.

We built YardSight AI to transform ordinary perimeter and gate cameras into an autonomous logistics nerve center. By combining an Alexa+ Model Context Protocol (MCP) agent with Ring perimeter monitoring, yard managers and logistics dispatchers can converse naturally with the yard's camera matrix, automatically verify e-Way bills and electronic BOLs, and trigger immediate physical containment for hazardous spills.

What it does

• Alexa+ Spatial-Temporal MCP Agent: A self-hosted Model Context Protocol (MCP) server running on Streamable HTTP. Terminal supervisors can use conversational voice commands via Alexa+:

  • "Alexa, which Schneider rig is currently at Loading Bay 3, and are any chemical hazards detected?"
  • "Alexa, check if any unmanifested tankers entered through the South Gate in the last hour."

• Ring Perimeter & Access Control Intelligence: Ingests live perimeter feeds, tracking inbound truck approaches, gate clearance status, and driver credential verification.

• OpenCV 4.10 Perspective Homography ANPR: Normalizes severe 30°–50° oblique camera angles using a 4-point projective transformation matrix H, producing orthogonal, crystal-clear character strips for dual-market recognition: US USDOT commercial carrier formats and India High Security Registration Plates (HSRP).

• Specular Concrete Hydrocarbon Sheen Classifier: Leverages thin-film optical dispersion modeling to distinguish dangerous diesel/hydraulic oil leaks from benign rainwater puddles, automatically sizing required granular absorbent boom poundage under OSHA 1910.120.

• Multi-Channel Emergency Dispatch & Audit Certification: Automatically notifies yard supervisors via WhatsApp Cloud API webhooks and generates cryptographically sealed jsPDF inspection reports.

How we built it

• Alexa+ / MCP Architecture: Built on the 2025-11-25 MCP specification exposing schema-validated endpoints (/api/mcp/execute) to allow autonomous AI agents to query live yard state and invoke physical actuators.

• Computer Vision Pipeline: OpenCV 4.10 homography normalization and Canvas 2D chromatic variance calculation.

• AWS Backend Integration: AWS Bedrock multi-modal vision reasoning for cross-camera temporal scene understanding.

• Frontend & Real-Time Video: React 19, TypeScript, Vite, Tailwind CSS v4, Lucide Icons, and WebRTC streaming.

• Automated Verification: 23/23 passing automated pytest test cases covering homography transformations, hydrocarbon sheen classifiers, and MCP tool execution.

Challenges we ran into

• Severe Camera Yaw & Pitch: Computing accurate 4-point homography matrices in real time without introducing character blur required precision edge detection and bilateral filtering.

• Thin-Film Specular Variance: Accurately distinguishing rainbow-sheen oil slicks from wet concrete reflections under shifting sunlight required fine-tuning chromatic dispersion thresholds.

• Latency Optimization: Maintaining smooth multi-camera WebRTC streaming while concurrently processing real-time MCP agent tool calls.

Accomplishments that we're proud of

• Sub-second perspective rectification and plate recognition on edge camera streams.

• Zero-drift spatial tracking across multi-camera handoffs.

• Seamless conversational Alexa+ simulated agent experience executing complex yard operations.

What we learned

• How to design standardized Model Context Protocol (MCP) tool schemas tailored for conversational voice and agentic execution.

• Mathematical modeling of thin-film specular reflectance on industrial concrete surfaces.

What's next for YardSight

• Direct hardware integration with Ring Alarm & Ring Video Doorbell Pro SDKs for two-way audio gate intercoms.

• Autonomous drone perimeter patrol integration with Alexa+ emergency dispatching.

  What we learned

What's next for YardSight

Built With

• alexa+
• aws-bedrock
• computer-vision
• fastapi
• iot
• mcp
• opencv
• python
• react
• ring
• tailwindcss
• typescript
• vite
• webrtc