SEWAI — Smart Edge Workflow & Production AI

SEWAI transforms ordinary factory CCTV into edge-AI production intelligence, automatically counting workpieces, detecting workflow anomalies, and generating real-time manufacturing insights.

Comment

SEWAI: Smart Edge Workflow & Production AI

1. Short Description

SEWAI is an edge-AI system that transforms ordinary factory CCTV footage into production intelligence. It counts completed garment pieces, monitors workstation activity, detects workflow anomalies and potential bottlenecks, and uses a locally running multimodal AI model to generate industrial engineering insights — without requiring dedicated industrial sensors.

2. Selected AI Track

AI + Business — Manufacturing Operations & Productivity

3. Problem Statement

Small and medium-sized garment factories often depend on manual production counting and supervision.

Supervisors have to:

  • Track completed pieces
  • Monitor workstation activity
  • Identify idle periods
  • Detect workflow interruptions
  • Determine where production bottlenecks are forming

Dedicated industrial sensors and MES systems can be expensive for smaller manufacturers. Basic computer-vision counters can also produce inaccurate results because they may interpret every detected movement as a production event.

SEWAI addresses this by using existing camera footage and domain-specific workflow reasoning to identify and validate actual production cycles.

4. Target Users

  • Supervisors and line managers in small and medium-sized garment factories
  • Industrial engineers conducting time-and-motion and Kaizen studies
  • Factory owners seeking production visibility without costly additional infrastructure

5. Technical Approach

SEWAI processes video through a multi-stage edge-AI pipeline:

CCTV / Video
     ↓
YOLOv8-Pose
     ↓
ByteTrack
     ↓
Worker Re-ID
     ↓
Machine-Centric Workflow State Machine
     ↓
Dual-Evidence Payload Validation
     ↓
Production Telemetry
     ↓
Gemma 3 Multimodal Analysis
     ↓
Industrial Engineering Report

1. Detection and Tracking

YOLOv8-Pose extracts worker pose keypoints, while ByteTrack tracks workers across video frames.

2. Machine-Centric State Machine

Instead of tying production cycles directly to a worker's tracking ID, SEWAI anchors the workflow to the physical workstation.

A production cycle progresses through:

IDLE
 ↓
HAND IN SOURCE
 ↓
IN TRANSIT
 ↓
HAND IN DESTINATION
 ↓
PAYLOAD VALIDATION
 ↓
CONFIRMED

This approach helps the system remain stable during temporary tracker failures, worker handoffs, and short camera occlusions.

3. Dual-Evidence Fabric Payload Gate

A cycle is counted only when multiple forms of evidence agree:

  • Source-area pixel difference verifies that material physically moved.
  • Wrist temporal motion analysis verifies meaningful movement during the transfer.

This helps reject empty-hand movements, repeated detections, and other false production events.

4. Worker Re-ID Ghost Buffer

SEWAI temporarily preserves spatial and motion information so a worker's canonical identity can be recovered after short tracking interruptions.

5. Workstation Monitoring

The system can identify:

  • Extended inactivity
  • Crowding around workstations
  • Workflow interruptions
  • Production cycle time
  • Machine utilization
  • Potential bottlenecks

The system focuses on process and workstation conditions rather than individual worker scoring.

6. Local Multimodal AI

Production telemetry and selected audit frames are analyzed by Google Gemma 3 12B running locally through Ollama.

The model generates industrial engineering insights covering:

  • Total production
  • Cycle times
  • Workstation utilization
  • Idle periods
  • Potential bottlenecks
  • Possible causes of workflow interruptions
  • Suggested Kaizen improvement actions

This creates a complete pipeline:

Video → Computer Vision → Structured Events → Telemetry → Multimodal AI → Engineering Insights

6. Technologies & Components

  • Python
  • OpenCV
  • YOLOv8-Pose — pose and keypoint detection
  • ByteTrack — multi-object tracking
  • Custom machine-centric finite state machine
  • Optical / pixel-difference analysis
  • Google Gemma 3 12B — multimodal AI
  • Ollama — local model execution
  • CSV telemetry
  • Markdown report generation
  • Local video files
  • RTSP camera streams

7. Outputs

SEWAI produces:

  • 🎥 Annotated video with detection and workflow overlays
  • 📊 Piece-level production telemetry
  • 📸 Confirmed and rejected cycle audit snapshots
  • 📈 Workstation and production statistics
  • 🤖 AI-generated industrial productivity reports

8. Real-World Impact

SEWAI aims to make production intelligence more accessible to small and medium-sized manufacturers by using camera infrastructure they may already have.

Potential benefits include:

  • Automated and auditable production counting
  • Fewer false-positive production counts
  • Workstation analytics and bottleneck identification
  • Faster, data-backed Kaizen decisions
  • Reduced dependence on manual production tracking
  • On-site processing for environments where keeping factory footage within the deployment environment is important

The goal is to bring capabilities traditionally associated with expensive industrial monitoring systems to smaller manufacturers using an affordable, camera-based AI approach.

9. What Makes SEWAI Different?

SEWAI does not simply detect people or count objects.

It attempts to understand a production event.

A garment is counted only when the system observes a valid workflow and independently verifies material movement:

SOURCE
  ↓
MATERIAL MOVEMENT
  ↓
TRANSIT
  ↓
DESTINATION
  ↓
VALIDATION
  ↓
PRODUCTION CYCLE

This combination of computer vision, temporal reasoning, workstation-centric logic, physical payload validation, and local multimodal AI is the core of SEWAI.

10. Future Scope

Future versions of SEWAI could expand into:

  • Multi-camera factory-wide monitoring
  • Production forecasting
  • Automated quality-defect detection
  • Predictive machine maintenance
  • Automatic production line balancing
  • Live production dashboards
  • ERP/MES integration
  • Deeper local VLM-based root-cause analysis

11. Project Vision

Turn the cameras factories already have into intelligence they can actually use.

Built With

  • ai
  • bytetrack
  • computer
  • edge
  • estimation
  • gemma
  • industrial
  • learning
  • machine
  • multimodal
  • ollama
  • opencv
  • pose
  • python
  • rtsp
  • vision
  • yolov8
  • yolov8-pose
Share this project:

Updates

Submission history