Automated Vehicles Security System

"Real-Time Edge-AI Detection, Privacy-Preserving Blurring, and Cloud-Connected Collision Risk Analytics."

Comment

Inspiration

The project was inspired by the critical tension between autonomous vehicle safety requirements and public privacy concerns. While self-driving platforms must continuously capture surrounding video feeds to detect hazards, they also collect sensitive personal identifiers, such as pedestrians' faces and surrounding vehicle license plates, without consent. To resolve this conflict, the goal was to design a "Privacy-First" co-pilot at the edge that detects road safety threats in real-time but immediately redacts sensitive identifiers locally before any telemetry or imagery is stored or uploaded to the cloud.

What it does

The Automated Vehicles Security System is a real-time, privacy-preserving safety monitor that detects road hazards while safeguarding public privacy at the edge. The system processes a live video stream through a YOLOv8-Nano model, estimating physical distance $d = \frac{W_{\text{real}} \times f}{w_{\text{perceived}}}$ to objects and calculating the Time-to-Collision ($TTC$) based on relative frame velocity. To protect individual privacy, the system applies a Gaussian blur filter $G(x, y)$ to mask pedestrians' bodies and leverages EasyOCR to extract and blur vehicle license plates, logging event telemetry to Azure Application Insights and uploading hazard snapshots to Azure Blob Storage asynchronously.

How we built it

The system was built using a modular edge-to-cloud architecture consisting of a Python FastAPI backend and a Vite React dashboard. The vision pipeline runs a lightweight YOLOv8-Nano model to detect pedestrians and vehicles, estimating target distances $d$ using the pinhole camera geometry formula $d = \frac{W_{\text{real}} \times f}{w_{\text{perceived}}}$ and tracking Time-to-Collision ($TTC = \frac{d_{\text{current}}}{v_{\text{rel}}}$) to trigger hazard alert events. Sensitive regions are instantly anonymized locally using OpenCV's Gaussian blur kernel $G(x, y) = \frac{1}{2\pi\sigma^2} \exp\left(-\frac{x^2+y^2}{2\sigma^2}\right)$ and EasyOCR for selective plate recognition, while asynchronous threads log telemetry to Azure Application Insights and upload hazard frames to Azure Blob Storage.

Challenges we ran into

The primary challenges revolved around processing latency and mathematical stability at the edge. Integrating EasyOCR for license plate scanning introduced a severe bottleneck ($150\text{ms}$–$300\text{ms}$ delay per frame), which was resolved by restricting the OCR scan to the bottom $30%$ of the vehicle's bounding box. Additionally, slight frame-to-frame jitter in YOLO bounding boxes caused massive spikes in estimated relative velocity $v_{\text{rel}} = (d_{\text{prev}} - d_{\text{current}}) \times \text{FPS}$, which required implementing threshold filters, while network calls to Azure had to be decoupled into background threads to prevent dropping the stream framerate.

Accomplishments that we're proud of

We are incredibly proud of successfully achieving real-time processing and logging on edge hardware without introducing video stream lag. We engineered a multi-threaded asynchronous logging architecture that offloads network-bound Azure cloud requests to background threads, preventing frame drops and keeping dashboard streams running at a steady $\approx 15\text{ FPS}$. Additionally, we designed a highly responsive, modern React dashboard that translates complex vision metrics—such as bounding boxes, simulated Lidar sweeps, and critical warning signals—into clear, readable UI elements, making the safety monitoring process intuitive and user-friendly.

What we learned

Through this project, we learned that edge-based privacy and high-precision safety metrics can co-exist without compromising system utility. We discovered that decoupling high-frequency tasks (like object detection and distance tracking) from low-frequency, high-latency tasks (like cloud logging and OCR text reading) is essential to maintain steady real-time execution. Furthermore, building a visual dashboard with custom React gauges taught us that human-machine interface (HMI) feedback is just as critical to autonomous monitoring systems as the underlying neural network models themselves.

What's next for Automated Vehicles Security System

Looking ahead, the next phase of development focuses on incorporating sensor fusion and improving edge processing efficiency. We aim to integrate actual LiDAR depth data with our camera-based distance estimation using Extended Kalman Filters (EKF) to minimize measurement noise and box jitter. Furthermore, we plan to transition the vision pipeline to highly optimized TensorRT or ONNX runtimes and implement DeepSORT tracking models, allowing the safety system to track multiple moving obstacles across occlusion zones with greater accuracy and lower hardware requirements.

Built With

Share this project:

Updates