\section{Inspiration} During our hands-on work with Formula Student Electric Vehicle (EV) competitions (Formula Bharat Pi-EV and Formula Imperial), we experienced firsthand how demanding and error-prone pre-energization technical scrutineering is. Before a high-voltage (HV) battery pack or industrial power assembly can be safely energized, engineers must inspect every module for three safety-critical failure modes: \begin{enumerate} \item \textbf{Cell Thermal Runaway Precursors:} Localized surface heating where cell temperature $T_{\text{cell}} \ge 65.0^\circ\text{C}$. \item \textbf{Interconnect Busbar Micro-Cracks:} Structural fatigue fissures across aluminum busbars with length $L_{\text{crack}} \ge 3.5\text{ mm}$. \item \textbf{High-Voltage Harness Clearance Violations:} Minimum Euclidean clearance distance $d_{\min} < 6.0\text{ mm}$ between HV orange cables and grounded chassis rails. \end{enumerate} In real factory environments, static single-pass computer vision pipelines fail under shop-floor vibration blur and overhead specular glare, or flood operators with false alarms on benign surface grease smudges. We built VoltSight AI to solve this through a closed-loop Agentic Active-Perception workflow powered by OpenCV and Amazon Web Services (AWS).
\section{What it does} Rather than acting as a static chatbot wrapper, VoltSight AI operates a multi-step Model Context Protocol (MCP) perception-decision-action loop ($\text{PERCEIVE} \rightarrow \text{ACT_AND_REPERCEIVE} \rightarrow \text{PERCEIVE_ZOOM_ROI} \rightarrow \text{ACT_AND_GOVERN}$) where deterministic OpenCV visual measurements directly control which tool the agent executes next: \begin{itemize} \item \textbf{Pass 1 --- Frame Telemetry Diagnostics (\texttt{opencv_frame_diagnostics}):} Evaluates optical focus variance, illumination dynamic range, and specular glare ratio before running any defect detectors. \item \textbf{Conditional Branch A --- Active Visual Restoration (\texttt{opencv_active_restoration}):} Triggered automatically only when Pass 1 detects motion blur, glare, or low contrast. Executes Navier-Stokes specular inpainting, HSV chromatic recovery, LAB-space adaptive histogram equalization (CLAHE), and unsharp mask deblurring before re-scanning the frame. \item \textbf{Pass 2 --- Multi-Modal Defect Scrutineering (\texttt{opencv_scrutineer_scan}):} Concurrently segments thermal anomalies, extracts busbar fissures, and measures millimeter-accurate HV cable-to-chassis clearance. \item \textbf{Conditional Branch B --- Secondary $2.5\times$ ROI Active Zoom (\texttt{opencv_roi_highres_inspect}):} Triggered dynamically whenever Pass 2 flags an ambiguous detection ($0.55 \le C_0 < 0.85$). Crops and upscales the region of interest $2.5\times$ via Lanczos-4 interpolation and verifies 95th-percentile Scharr gradient energy to confirm true fissures while eliminating 100\% of surface-smudge false alarms. \item \textbf{Cloud Persistence & Safety Governance (\texttt{aws_dispatch_remediation_ticket}):} Uploads annotated OpenCV diagnostic evidence to Amazon S3, logs latency and accuracy metrics to Amazon CloudWatch, dispatches structured remediation tickets, and engages a hard High-Voltage Safety Interlock awaiting human sign-off on all CRITICAL faults. \end{itemize}
\section{How we built it} We implemented the computer vision core in Python (\texttt{voltsight/vision_engine.py}) using OpenCV (\texttt{cv2}) and NumPy, orchestrated by our MCP agent (\texttt{voltsight/agent_orchestrator.py}), exposed via a FastAPI web server (\texttt{app.py}), and integrated with AWS via boto3 (\texttt{voltsight/aws_cloud.py}):
\subsection{1. Optical Telemetry & Active Restoration Formulation} Given an input frame $I \in \mathbb{R}^{H \times W \times 3}$, we convert to grayscale $I_g$ and CIE-LAB luminance $L$, computing the Laplacian focus variance $\sigma^2_{\text{Lap}}$, dynamic range $\Delta L_{95,5}$, and specular glare ratio $R_{\text{glare}}$: $$\sigma^2_{\text{Lap}} = \text{Var}\left(\nabla^2 I_g\right) = \text{Var}\left(\frac{\partial^2 I_g}{\partial x^2} + \frac{\partial^2 I_g}{\partial y^2}\right)$$ $$\Delta L_{95,5} = P_{95}(L) - P_{5}(L), \qquad R_{\text{glare}} = \frac{1}{HW} \sum_{x,y} \mathbb{I}\left(S(x,y) \le 45 \;\wedge\; V(x,y) \ge 238\right)$$ The agent invokes \texttt{opencv_active_restoration} if and only if: $$\left(\sigma^2_{\text{Lap}} < 120.0\right) \;\lor\; \left(R_{\text{glare}} > 0.015\right) \;\lor\; \left(\Delta L_{95,5} < 115.0\right)$$ Restoration executes Navier-Stokes PDE inpainting (\texttt{cv2.inpaint} with \texttt{cv2.INPAINT_NS}), LAB contrast equalization via \texttt{cv2.createCLAHE(clipLimit=3.2, tileGridSize=(8,8))}, and bilateral unsharp deconvolution: $$I_{\text{sharp}} = \text{BilateralFilter}\left(2.25 \, I_{\text{CLAHE}} - 1.25 \, \left(G_{\sigma=2.4} * I_{\text{CLAHE}}\right)\right)$$
\subsection{2. Multi-Modal Defect Metrology} \begin{itemize} \item \textbf{Thermal Hotspot Segmentation:} HSV Dual-Band thresholding (\texttt{cv2.inRange}) and morphological opening (\texttt{cv2.MORPH_OPEN}) extract hotspot clusters via \texttt{cv2.connectedComponentsWithStats}, computing physical area $A_{\text{mm}^2} = N_{\text{px}} \cdot s^2$ ($s = 0.25\text{ mm/px}$) and peak temperature: $$\hat{T}{\text{cell}} = 48.0 + 0.42 \left(\bar{V}{\text{ROI}} - 165.0\right) \; [^\circ\text{C}$$ \item \textbf{Busbar Micro-Crack Profiling:} A $9 \times 9$ morphological black-hat transform (\texttt{cv2.MORPH_BLACKHAT}) extracts hairline fissures darker than local aluminum structures $K_{9\times 9}$: $$B(I_g) = \left(I_g \bullet K_{9\times 9}\right) - I_g$$ Extracted contours are scored by mean black-hat depth $\bar{B}{\text{cnt}}$ to set initial confidence $C_0 = \min\left(0.95, \, \max\left(0.58, \, 0.40 + \frac{\bar{B}{\text{cnt}}}{155.0}\right)\right)$. \item \textbf{High-Voltage Clearance via Euclidean Distance Transform:} Given masks for the HV orange cable $M_{\text{HV}}$ and chassis ground $M_{\text{GND}}$, we compute the Euclidean distance transform (\texttt{cv2.distanceTransform} with \texttt{cv2.DIST_L2}) over free space $\Omega = \neg M_{\text{GND}}$: $$D(p) = \min_{q \in M_{\text{GND}}} \Vert{}p - q\Vert{}2, \qquad d{\min} = s \cdot \min_{p \in M_{\text{HV}}} D(p) \; [\text{mm}]$$ \end{itemize}
\subsection{3. Secondary $2.5\times$ Lanczos-4 ROI Active Zoom} When candidate defects fall in the ambiguous band $0.55 \le C_0 < 0.85$, the agent crops and upscales the bounding box $2.5\times$ via \texttt{cv2.INTER_LANCZOS4}, computing Scharr directional derivatives (\texttt{cv2.Scharr}): $$G_{\text{Scharr}} = \sqrt{\left(S_x * I_{\text{ROI}}\right)^2 + \left(S_y * I_{\text{ROI}}\right)^2}, \qquad E_{95} = P_{95}\left(G_{\text{Scharr}}\right)$$ $$C_{\text{final}} = \begin{cases} \min(0.98, \; C_0 + 0.16) & \text{if } E_{95} > 920.0 \quad (\text{Confirmed Crack}) \ \max(0.22, \; C_0 - 0.38) & \text{if } E_{95} \le 920.0 \quad (\text{Rejected Surface Smudge}) \end{cases}$$
Log in or sign up for Devpost to join the conversation.