Inspiration
Dr. Alexy Bhowmick, a professor at Assam Don Bosco University, published two influential papers on FhVLAD:
- https://link.springer.com/article/10.1007/s11042-020-10491-7
- https://link.springer.com/chapter/10.1007/978-3-030-34869-4_61
He also had posted two repositories for FhVLAD and CNN-FV: https://github.com/lx-git/FhVLAD and https://github.com/lx-git/CNN-FV. I didn't fully grasp FhVLAD at first, but exploring further, I discovered Fisher Vector (FV)—a compact way to encode images as dense vectors capturing their essence. Fascinated by how FV could summarize complex visuals, I turned to Grok for ideas. It suggested a Chrome extension to detect duplicate images across tabs—a practical use for cluttered browsing. When the Google Chrome Built-in AI Challenge 2025 email arrived, it felt like perfect timing: Chrome lacks this feature, so I dived in as a tribute to Dr. Bhowmick's work, blending FV with the Prompt API.
What it does
DupliCheck scans open tabs, encodes screenshots to Fisher Vectors, and detects potential duplicates based on similarity scores. It uses the Prompt API to generate explanations (e.g., "Possible match on shapes/colors"). The prototype focuses on natural images (e.g., photos), with a simple popup showing results, including a "Close duplicate" button to remove one tab.
How we built it
- GMM Training: Used TensorFlow MobileNetV2 to extract 1280-dim features from a CIFAR-10 subset (20k images), fitted a 64-component GMM with scikit-learn, and exported params.json. Screenshot Capture: chrome.tabs.captureVisibleTab grabs tab views, cropped to content (canvas removes UI noise).
- FV Encoding: tfjs in worker.js loads GMM, computes FV gradients for 2560-dim vectors. Detection: Chi-squared distance <2.0 flags dupes in background.js.
- AI: Prompt API in ai-helper.js explains matches (with fallback for errors).
- UI: Popup.js handles async scan, progress bar, results with scores/tabs.
Built solo in VS Code with Copilot/Grok help—MV3-compliant, ~10MB.
Accomplishments that we're proud of
- Functional Prototype: From idea to scanning tabs with FV encoding in a week—solo, through burnout.
- Prof Tribute: Implemented FV inspired by Dr. Bhowmick's papers/repos—honors his scalable encoding research.
- Offline Privacy: 100% client-side—no cloud uploads, aligning with Chrome's ethos.
- Hackathon Fit: Integrates Prompt API for explanations, demonstrating built-in AI for UX.
What we learned
- FV Power: Compact GMM (64 components) encodes images robustly—better than hashing for variations, but sensitive to noise (e.g., UI).
- tfjs Realities: Browser ML is feasible but memory-hungry—tidy() and cropping saved the day.
- Hackathon Truth: Prototypes win on idea + demo—prioritize core (scan + score) over perfection; limitations are features (e.g., "tunable for products").
- Mental Health: Pushing solo teaches resilience, but boundaries matter—products evolve, but well-being doesn't.
What's next for DupliCheck
- Noise Reduction: Advanced cropping (AI-based content detection) for 95%+ accuracy on UI-heavy tabs.
- Web-Scale GMM: Retrain on 50k OpenImages subset for broader generalization (memes, e-comm).
- Functionalize and Deploy: A few issues remain, and it can be deployed as a Chrome Extension.
- Optimization and Study: Explore different algorithms to determine the best duplicate checker for Chromium browsers.
Built With
- pillow
- prompt-api
- scikit-learn
- tensorflow
- tensorflow-js
Log in or sign up for Devpost to join the conversation.