Inspiration

Fort Worth already has the ingredients for a real industrial tech ecosystem: manufacturers, suppliers, logistics, rail, and a growing technical workforce. What we kept seeing—and what HackFW’s flywheel framing made concrete—is that the talent pipeline and the plant floor are often disconnected. Students and early-career engineers learn in notebooks and toy datasets; plants still discover quality problems at the end of the line, after scrap is already made.

The brief’s self-checkout analogy stuck with us: a grocery store watches every station continuously, catches problems early, and keeps a human in the loop. We wanted the same for a manufacturing line—overhead and station cameras watching parts, tools, and people; in-process quality checks for wrong assembly, missing fasteners, surface defects, and cycle-time stalls; and an action a plant could actually run. That became ForgeSight: a thin, end-to-end “capture → decide → act” system for in-process manufacturing quality and cycle-time monitoring.

We were also inspired by the techno-industrialism focus. This is not a model demo. It is a system with an event schema, an edge deployment path, an operator workflow, and adapters that a plant engineer could clone, run, and maintain.

What it does

ForgeSight ingests overhead and station camera streams and turns them into live line health.

It detects parts, tools, and people; tracks them across frames; and applies station-level zone logic to check whether the expected assembly sequence is happening. It flags wrong or missing parts, missing fasteners in the prototype model, surface defects, unsafe motion near a station, and cycle-time stalls. Each detection becomes a structured event with station ID, timestamp, event type, severity, confidence, evidence crop, and a suggested action.

The decision layer is deliberately temporal, not frame-by-frame. The dashboard shows live station status, active alerts, and an auditable event ledger. A station tablet view gives the operator a clear “check this” or “hold” action. The operator can confirm or dismiss an event, and that feedback is stored for threshold tuning and future labeling. The demo runs end to end from a recorded or simulated line stream through the dashboard and station alert—no notebook hand-waving.

How we built it

We built ForgeSight as a small product, not a pile of scripts.

Capture: a Python/OpenCV ingestion service reads RTSP, video files, or a simulated camera stream, with configurable frame sampling and health checks.

Perception: Ultralytics YOLO models detect parts, tools, and people; ByteTrack keeps identities; polygon zones encode station geometry. A lightweight classifier handles prototype fastener/surface-defect checks.

Decision engine: a per-station state machine checks expected sequence, dwell time, and zone occupancy. It applies temporal voting, debounce, and severity rules before emitting an event.

Data layer: FastAPI plus Postgres stores events, station configs, operator feedback, and evidence metadata. Evidence crops are stored locally and linked to events.

Operator UI: a React dashboard shows live line health, event review, and station-level alerts. Operators can confirm/dismiss; the ledger is exportable.

Integration surface: MQTT and an OPC UA adapter stub publish events to an Andon/MES-style endpoint. All station logic and thresholds are YAML-configured.

Ergonomics: Docker Compose, seed data, a recorded demo stream, .env.example, health checks, and a README that a plant engineer can follow. The goal was: clone, run, see the full path, then swap in a real camera and adapter.

We kept the end-to-end path narrow: capture, decide, act. The demo boots with one command and shows a real event lifecycle from camera frame to operator action.

Challenges we ran into

The plant floor is hostile to computer vision. Metal reflects, lighting changes, people occlude parts, and motion blur is normal. Early frame-by-frame detections produced false positives. We addressed this with zone constraints, temporal voting, confidence thresholds, debounce, and human-in-the-loop confirmation rather than pretending a single model score was truth.

Label scarcity was another constraint. We had limited labeled line footage, so we leaned on pretrained detectors, a small task-specific dataset, and an active-learning queue built from operator dismissals and confirmations. We also had to be honest about what was production-ready versus prototype: the system is designed so models can be swapped without changing the event contract.

Latency and reliability mattered more than benchmark accuracy. We used frame sampling, async workers, and local buffering so the edge service can run offline and degrade gracefully. Integration realism was also hard: no real MES/ERP was available, so we built a clean adapter contract and simulator instead of overfitting to one vendor. Finally, alert fatigue is a real operator problem. We spent time on severity, debounce, and clear evidence crops so an alert is actionable, not noise.

Accomplishments that we're proud of

We are proud that ForgeSight is a running system with a complete path: camera capture, perception, temporal decision logic, structured event, dashboard, and station action. The event ledger and operator feedback loop make the system auditable and tunable.

We are also proud of the ergonomics. A teammate, judge, or plant engineer can clone the repo, run Docker Compose, open the demo, and see the system work end to end. The README, environment file, seed data, and adapter stubs are part of the submission, not an afterthought. We kept the architecture small enough to maintain but real enough to pilot.

Most importantly, it fits the track: AI for Manufacturing, focused on overhead/station cameras, in-process quality, cycle-time stalls, and a human-in-the-loop action a plant could actually run.

What we learned

We learned that in industrial AI, “accurate enough, explainable, and actionable” beats a better benchmark score that an operator cannot trust. The event schema and data infrastructure matter as much as the model. Temporal logic, zone constraints, and operator feedback are not polish—they are core to making vision usable on a line.

We also learned to ship the thin vertical slice first. A working capture-to-action path with clear contracts is more valuable than a pile of notebooks. Techno-industrialism is as much about integration, uptime, and ergonomics as it is about model architecture. And the HackFW flywheel idea is real: exposing technical talent to a plant-floor problem changes how you build software.

What's next for ForgeSight

Next is a pilot with a Fort Worth manufacturer or supplier. We want to calibrate cameras and cycle-time baselines on a real station, expand the missing-fastener and surface-defect models with active learning from operator feedback, and deploy on an industrial edge PC with offline buffering. On the integration side, we will harden the OPC UA/MES adapter and add a PLC/Andon action path so a confirmed event can trigger a real station hold.

Longer term, we want multi-station scaling, a safer mobile robot/drone VSLAM inspection mode, and an open adapter library so small plants can adopt the system without a large integration project. The goal is to keep the thin end-to-end path, keep the human in the loop, and turn ForgeSight into a system a plant engineer can clone, run, and maintain.