SentinelGlove: Instinctive Edge AI for Industrial Safety

Project-Logo
Smart Glove connected to system
SmartGlove

Inspiration: Closing the "Hazardous Meter"

In high-stakes industrial environments, safety is often compromised in the "last meter" of interaction. We observed that workers frequently enter hazardous zones—exposed to extreme heat, toxic fumes, or high-voltage machinery—simply to toggle a switch or check a status. This inspired us to create SentinelGlove, a novel human interface designed to bridge this safety gap. By enabling instinctive movement and hands-free control, we aim to keep the human operator safe while maintaining full tactical authority over complex machinery.

What it does

SentinelGlove serves as an instinctive human interface for industrial operators, allowing them to control machinery from a safe distance using natural hand gestures. By analyzing eight simultaneous channels of data—five for finger flex and three for wrist acceleration—the system translates physical intent into digital commands.

Our current prototype demonstrates this "Safety Gap" closure by providing 1:1 real-time control over a high-torque motor and an RGB status signaling system. Through a high-speed 77Hz sampling rate and a scalable WebSocket communication protocol, the glove provides a seamless, lag-free experience that feels like a natural extension of the worker's own body.

How we built it

SentinelGlove is a multidisciplinary engineering project that combines custom embedded hardware, an Edge AI gesture-recognition pipeline, and a memory-safe Rust software architecture.

Hardware & Signal Processing
We used an ESP32-S3 (N8R8) as the central processing unit, interfacing with a sensor suite consisting of five flex sensors and a 3-axis accelerometer. Together, these provided eight channels of motion and finger-position data. The sensor streams were sampled into fixed-size windows and passed into our gesture-recognition model.
Edge AI Pipeline (1D-CNN)
Instead of relying on fragile threshold-based gesture logic, we built a 1D Convolutional Neural Network (CNN) that learns temporal patterns directly from sensor sequences. Each input window contained 154 samples. Before training, we standardized each sensor value using the formula z = (x − μ) / σ, where μ is the mean and σ is the standard deviation calculated from our custom dataset, which we collected and labeled over roughly three hours. The model was trained in TensorFlow, converted to ONNX, and then quantized into the optimized ESP-DL format for deployment on the ESP32-S3.
Software Architecture (Rust)
The firmware was built in Rust to support reliability, maintainability, and safer embedded development. Rust’s memory-safety guarantees helped us reduce risks from common low-level bugs while building firmware for BLE provisioning, Wi-Fi connectivity, HTTP registration, and WebSocket telemetry streaming. We used a client-server IoT architecture so SentinelGlove can scale beyond a single prototype toward larger industrial deployments.

Challenges we ran into

The journey from prototype to a functional industrial tool was not without hurdles:

• The Conversion Labyrinth: Transitioning from high-level TensorFlow models to the specialized ESPDL format presented significant rank and shape mismatches. We had to meticulously bridge the gap between 3D pooling outputs and 2D dense layers using custom flatten nodes.

• Data Hunger: 1D-CNNs are notoriously data-hungry. Collecting and labeling a high-quality, custom dataset in just 3 hours required a highly modular and efficient data pipeline.

• Bare-Metal Constraints: Implementing complex AI inference in a no_std Rust environment meant we couldn't rely on standard libraries, forcing us to optimize every byte of memory and CPU cycle.

Accomplishments that we're proud of

Accomplishments that we're proud of The Custom Dataset: We successfully built, labeled, and validated a high-quality custom dataset from scratch in only three hours, proving the modularity and efficiency of our data pipeline.

Rust-Powered Reliability: We achieved a "Zero Leak" architecture by implementing the entire firmware in memory-safe Rust. This ensures the level of crash-resistance required for safety-critical industrial applications.

State-of-the-Art Edge AI: We successfully navigated the "Conversion Labyrinth" to deploy a quantized 1D-CNN on the ESP32-S3. By leveraging the chip's vectorization instructions through ESPDL, we achieved high-fidelity inference with an extremely small memory footprint.

Industrial Scalability: We designed a robust Client-Server IoT architecture that is not limited to a single prototype; it is fully capable of scaling to support hundreds of synchronized "SmartWear" nodes across a single factory floor.

What we learned

This project pushed us to the boundaries of Edge AI and Embedded Systems:

• The Power of Custom Data: We learned that a model is only as stable as the data it’s fed. Building our own dataset allowed us to overcome the "Domain Gap" between generic data and our specific hardware.

• Rust for Industrial Safety: We discovered that Rust isn't just a performance tool; its compiler serves as a "safety engineer," preventing the runtime crashes common in C-based systems.

• Optimization is King: We mastered the art of quantization, learning how to compress state-of-the-art CNNs into 8-bit integers without losing the precision needed for safe industrial control.

What's next for SentinelGlove: Instinctive Edge AI for Industrial Safety

he current prototype is only the beginning of our vision for a safer, more intuitive industrial landscape. Our roadmap focuses on three pillars: technical refinement, system integration, and field deployment.

Advanced Signal Processing & DSP: We plan to implement more sophisticated Digital Signal Processing (DSP) techniques to handle the extreme electrical noise and high-frequency vibrations found on factory floors. By refining our multi-channel low-pass filters, we aim to reach a near-perfect signal-to-noise ratio, ensuring that only the operator's intentional movements are captured.

Dynamic On-the-Fly Retraining: While our current pipeline is modular, we want to fully automate the transition from data collection to model deployment. Our goal is to create a "Plug-and-Train" system where a worker can wear the glove, perform a new motion ten times, and have the 1D-CNN update its gesture vocabulary in real-time.

Haptic Feedback & Bi-Directional Control: To move beyond a one-way command system, we plan to integrate haptic actuators into the glove’s fingertips. This will provide tactile feedback to the worker, such as a vibration alert when a machine reaches a specific temperature or a "pulse" to confirm a command has been executed successfully.

Industrial Field Testing: We are looking to test SentinelGlove in real-world automotive or manufacturing environments, similar to the complex workflows found at companies like Cummins, to validate our 94% accuracy and zero-leak Rust architecture under true stress conditions.