LumiProbe-X: Autonomous Active Photometric Reality Engine

Turn any laptop screen & webcam into a programmable 3D photometric stereo lab to defeat 2D counterfeits.

Comment

Inspiration

Almost every counterfeit-detection and anti-spoofing system today is completely passive: you take a single snapshot of an ID card, banknote, or face, and feed it into a deep neural network classifier.

The catch? Passive 2D vision is notoriously fragile. Ultra-high-resolution printers, tablet screens, and generative models can mimic 2D color patterns with ease. But in the physical world, humans never inspect a suspicious item passively. If you hand someone a banknote or credit card to verify, their immediate physical reaction is to tilt it under a light to see if the raised intaglio ink and paper embossing catch the illumination.

We asked a straightforward question: Why should anyone need a \$1,500 laser scanner or specialized LiDAR sensor to inspect physical depth? Why can't an ordinary laptop screen act as a programmable structured illuminator, and the webcam act as a calibrated sensor?

That idea became LumiProbe-X.

What it does

LumiProbe-X turns any standard laptop into an active 3D photometric stereo laboratory:

  • Active Structured Lighting: The computer screen functions as a multi-zone emitter, projecting complementary linear gradient ramps, grazing-angle bars, and steerable spots.
  • Mathematical Albedo Cancellation: By pairing complementary gradient illuminations, surface albedo ( \rho(x, y) ) cancels out algebraically: $$r_x = \frac{J_{LR} - J_{RL}}{J_{LR} + J_{RL}} \propto n_x(x, y)$$ Even high-contrast printed text and colored ink disappear, leaving behind pure, unadulterated 3D surface tilt.
  • Wald’s Sequential Probability Ratio Test (SPRT): Rather than blindly running a fixed sequence of frames, our autonomous agent tests statistical evidence dynamically at each step. If it observes zero relief energy (a flat 2D photocopy), it halts early and rejects it with 99% statistical confidence (( p < 0.01 )). If it detects genuine physical depth, it confirms it in ~6 light steps.
  • Fourier-Domain 3D Reconstruction: Using a Frankot-Chellappa 2D Fourier Poisson solver, surface normal gradients are integrated into metric 3D height maps: $$\hat{Z}(u, v) = \frac{-i u \hat{P}(u, v) - i v \hat{Q}(u, v)}{u^2 + v^2}$$
  • Interactive Mission Control Dashboard: Renders an interactive 3D WebGL mesh via Three.js that users can orbit with their mouse, alongside a synchronized fullscreen Lamp projector (/lamp) and instant Wavefront .OBJ 3D file export.
  • Sub-Millimeter Defect Segmentation: Morphological anomaly filters flag micro-cracks, scratches, and dents on manufactured parts.

How we built it

We designed LumiProbe-X around fundamental computer vision math rather than wrapping black-box third-party APIs:

  • OpenCV 5.0 Core Engine:
    • cv2.objdetect / aruco (ArucoDetector DICT_4X4_50) locks onto 4-corner desk window cards with sub-millimeter precision.
    • cv2.calib3d (findHomography, warpPerspective) unwarps perspective angles into clean 512x512 fronto-parallel metric space.
    • cv2.dnn (YuNet 2023 ONNX) provides real-time biometric face and landmark tracking.
    • cv2.imgproc manages inverse sRGB LUT linearization (IEC 61966-2-1), subpixel motion checks with phaseCorrelate, and morphological anomaly segmentation.
    • 2D Fourier Poisson integrator solves for physical height ( z(x, y) ) with zero boundary distortion.
  • AWS Graviton & COOL Acceleration:
    • Serverless AWS SAM infrastructure (infra/template.yaml) built for AWS Graviton3 (ARM64) Lambda compute. Offloading 2048x2048 multi-spectral FFTs to Graviton cuts cloud cost by 40% while doubling compute throughput.
    • Cryptographic evidence records and 3D meshes stream directly to Amazon S3 and Amazon DynamoDB, with zero-latency local fallback.
  • Agentic Vision & Model Context Protocol (MCP):
    • An autonomous Next-Best-Light planner tracks a 16x16 spatial uncertainty map, dynamically selecting illumination patterns that maximize information gain.
    • A native MCP server (src/lumiprobe/mcp/server.py) allows external LLM agents (Claude, Gemini, OpenAI) to invoke active photometric reality verification tools.

Challenges we ran into

  • UVC Webcam Auto-Exposure Wars: Laptop webcams constantly try to "help" by dynamically adjusting brightness and gain as screen patterns flash. On Windows, we locked the DirectShow hardware backend (CAP_DSHOW, CAP_PROP_AUTO_EXPOSURE = 0.25) to preserve strict radiometric linearity.
  • Near-Field Screen Curvature: Because a laptop screen sits only inches from the desk, it acts as an extended planar emitter rather than a point source. Even a flat card displays a slight gradient. We solved this by fitting a 2D polynomial reference plane to absorb the screen's curvature, leaving behind pure sensor noise on flat prints and coherent signal on real relief.
  • Clean Real-Time Telemetry: Coordinating the main dashboard with a separate fullscreen projector window over WebSockets without blocking Python's async event loop required designing a dedicated, low-bandwidth /ws/lamp synchronization channel.

Accomplishments that we're proud of

  • 100.0% Benchmark Accuracy: Tested over randomized trials across coins, Braille cards, and flat spoof prints, achieving 0.0% False Accept Rate (FAR) on counterfeit 2D prints.
  • 11/11 Automated Tests Passing: Complete automated test suite covering vision math, SPRT bounds, MCP server, and cloud fallbacks.
  • Zero Hardware Barrier: Democratized laboratory-grade photometric stereo so that anyone with a laptop and webcam can inspect physical reality immediately.

What we learned

Active perception fundamentally transforms computer vision. When you treat illumination as an active, programmable output rather than an uncontrolled environmental obstacle, mathematical principles (like albedo cancellation) can cleanly solve problems that even massive passive neural networks struggle with.

What's next for LumiProbe-X

  • Smartphone OLED Deployment: Porting the coded screen flashes to mobile OLED screens for instant point-of-sale passport and banknote verification.
  • Cross-Polarization Coded Flash: Adding simple cross-polarizing filters to eliminate specular glare on glossy credit cards and holographic security threads.

Built With

Share this project:

Updates

Submission history