Invisibility Cloak

Invisible effects in real-time that create the cloak of Harry Potter and make it appear like one made by children, without the use of any CGI.

Comment

Inspiration

In my childhood days I would have watched Harry Potter... I was fully enamored with the invisibility cloak of Harry Potter as I grew up watching Harry Potter. That military cloak that might cause a person to disappear seemed like the supreme superpower. This was my moment of what-if years later when studying computer vision - was it actually possible to make a real invisibility cloak by writing some code.

What it does

CloakVision transforms any colored cloth into a real-time invisibility cloak using computer vision technology.

Core Functionality: 1)Instant Invisibility: Hold up any red, green, blue, or yellow fabric in front of your webcam and watch it completely disappear, replaced seamlessly with the background. 2) Real-Time Processing: Live video feed with zero delay ,the invisibility effect happens instantly as you move. 3)Smart Face Protection: Face detection ensures your head stays visible for safety while everything else behind the cloth vanishes. 4)Multi-Color Support: Works with four different cloak colors, each optimized for different lighting conditions(mainly trained for RED,BLUE,GREEN colours).

Interactive Features: 1)Background Capture: One click background is captured when you are out of frame. 2)HSV Fine-Tuning: Adjustable color sensitivity sliders for perfect invisibility in any lighting(atomatically for RED colour). 3)Live Statistics: Real time display of cloak coverage percentage, face count, and processing FPS. 4)Mini Hide and Seek Game: Interactive mini game where you must stay hidden from a cartoon character using your invisibility cloak.

Technical Magic: 1)Dual HSV Detection: Handles tricky colors like red that wrap around the color spectrum. 2)Morphological Processing: Smooth, seamless edges with no jagged artifacts. 3)Background Subtraction: Intelligent replacement of cloak areas with pre-captured background. 4)Web-Based Interface: Modern, responsive design that works on desktop, tablet, and mobile devices too.

How we built it

Technical Architecture & Development Process: Backend Foundation: 1)Flask Web Framework: Built a lightweight Python server to handle real-time video streaming and API endpoints. 2)OpenCV Computer Vision: Core image processing engine for color detection, background subtraction, and face recognition. 3)NumPy Arrays: Efficient matrix operations for real-time pixel manipulation and mask processing.

Computer Vision Pipeline: 1)HSV Color Space Conversion: Converted RGB video frames to HSV for more reliable color detection under varying lighting conditions. 2)Dual Range Detection: Implemented special handling for red color detection (0-10° and 170-179° hue ranges) due to color wheel wraparound. 3)Morphological Operations: Applied opening, closing, and dilation filters to eliminate noise and create smooth mask edges. 4)Haar Cascade Integration: Used pre-trained face detection models to create protection zones around detected faces.

Real-Time Processing Optimization: 1)Threading Architecture: Separated video capture, processing, and streaming into independent threads for smooth performance.. 2)Memory Management: Efficient frame buffering and garbage collection to prevent memory leaks during extended use 3)Adaptive Processing: Dynamic quality adjustment based on system performance to maintain real-time speeds.

Frontend Development: 1)Responsive Web Design: HTML5/CSS3 interface with mobile-first approach and gradient animations. 2)JavaScript Integration: Real-time controls for color selection, HSV adjustment, and game mechanics. 3)WebRTC Integration: Browser-based camera access with fallback support for different devices.

Game Logic Implementation: 1)Random Timer System: Implemented unpredictable countdown intervals (5-15 seconds) for hide-and-seek gameplay. 2)State Management: Real-time tracking of player visibility, game score, and detection events. 3)WebSocket-like Updates: Continuous client-server communication for live statistics and game state.

Deployment Strategy: 1)Multi-Platform Support: Created both standalone Python application and web-deployable Flask version. 2)Docker Containerization: Packaged with all dependencies for consistent deployment across platforms. 3)Static Asset Optimization: Compressed images and minified code for faster loading times.

Challenges we ran into

• Red Color Detection Issues: Red cloth invisibility was extremely problem because red color wraps around the HSV color wheel at both 0° and 180°, causing flickering and inconsistent detection that required implementing dual range color detection algorithms not only in red colour all the colours have the same problemn. • Live Webcam Deployment Restrictions: Major hosting platforms (Render, Heroku, Netlify, Vercel) do not support live webcam streaming due to security policies and resource limitations, making direct deployment impossible. • Real-time Performance Bottlenecks: Initial version ran at only 2-3 FPS due to heavy OpenCV processing, requiring complete threading architecture redesign and optimization. • Face Detection Accuracy Problems: Haar cascade classifiers kept making faces disappear accidentally or detecting false positives on hands and clothing. • Cross-platform Compatibility Issues: Different webcam drivers, browser security policies, and hardware acceleration differences caused inconsistent behavior across devices. • Memory Management Challenges: Application crashed after extended use due to memory leaks in continuous video processing loops.

Accomplishments that we're proud of

• Creative Deployment Solution: Instead of direct hosting, we created a showcase website with embedded demo video and uploaded complete project files to GitHub, hosted on Netlify for judges to download and verify locally. • Successful Demo Documentation: Provided clear video demonstration of the invisibility effect working in real-time with detailed setup instructions for local testing. • Complete Project Accessibility: Made entire codebase available on GitHub with comprehensive README, requirements.txt, and step-by-step installation guide for easy verification in VS Code. • Real Invisibility Magic: Successfully created genuine real-time invisibility effects that work seamlessly with colored cloth and webcam. • Multi-color Support Implementation: Achieved working invisibility for red, green, blue, and yellow cloaks with optimized detection algorithms. • Interactive Gaming Features: Built a mini hide and seek game that transforms tech demo into engaging user experience. • Professional UI/UX Design: Created modern, responsive web interface with live statistics and intuitive controls.

What we learned

• Computer Vision Complexity: Real-time video processing requires deep understanding of color spaces, morphological operations, and performance optimization techniques. • Deployment Limitations: Live webcam applications face significant hosting restrictions, requiring alternative demonstration strategies for hackathon presentations. • Threading Architecture Importance: Separating video capture, processing, and display into independent threads is crucial for smooth real-time performance. • Color Detection Nuances: HSV color space is superior to RGB for robust color detection, especially handling edge cases like red color wraparound. • User Experience Design: Technical excellence must be combined with intuitive interfaces and engaging features to create memorable experiences. • Documentation Value: Clear setup instructions and demo videos are essential when direct deployment isn't possible.

What's next for Invisibility Cloak

• Mobile AR Integration: Implement ARCore/ARKit support for smartphone and tablet invisibility effects without webcam dependency issues. • Cloud Processing Architecture: Develop server-side processing with client-side video streaming to overcome hosting limitations. • Multi-person Detection: Expand algorithm to handle multiple people becoming invisible simultaneously. • Advanced Background Replacement: Add custom environment backgrounds (space, underwater, fantasy worlds) instead of static background capture. • Machine Learning Enhancement: Integrate ML models for improved color detection accuracy and automatic lighting adjustment. • WebRTC Implementation: Enable real-time multiplayer invisibility games for remote collaboration. • Social Media Integration: Create mobile app with direct TikTok/Instagram sharing capabilities for viral content creation. • Performance Optimization: Implement GPU acceleration and WebGL processing for higher resolution and smoother effects.

Share this project:

Updates