BillGuardAI

AI-powered system that automatically detects billboards and checks if they comply with government rules on size, placement, and content.

Comment

Inspiration

BillGuardAI was created to help civic officials, NGOs and concerned citizens quickly identify unauthorized or non-compliant billboards from street photography. The goal is to automate repetitive inspection work, surface potential violations, and provide actionable evidence (cropped images, measurements, text) while minimizing manual effort.

What it does

1)Detects billboards in photos using a custom YOLOv8 model. 2)Estimates real-world dimensions and area coverage to check compliance against regional rules. 3)Extracts and classifies billboard text (OCR → heuristic / Hugging Face classifiers) to flag prohibited or political content. 4)Runs lightweight NSFW/visual checks on the cropped billboard image. 5)Produces a compact JSON report and visualizations; offers both a PyQt desktop UI and a Streamlit web UI.

How we built it

1)Object detection: custom YOLOv8 model loaded from the Hugging Face Hub for robust billboard localization. 2)Sizing: pixel-to-meter approximation using a camera-width heuristic and detected bbox proportions. 3)OCR: Tesseract (with a Windows auto-path helper) and EasyOCR as a fallback for text extraction and preprocessing. 4)Content analysis: optional Hugging Face text and image classification pipelines with graceful degradation to heuristics when models are unavailable. 5)Interfaces: PyQt5 for a polished desktop app and Streamlit for a fast web/demo UI; both export JSON reports and cropped/visualized images.

Challenges we ran into

1)Inconsistent image quality and perspective distortions that affect OCR and size estimates. 2)Dependency variability: some users lack Tesseract/EasyOCR or HF model access — required safe fallbacks so the service still returns usable results. 3)Balancing detection sensitivity to avoid false negatives while keeping false positives low across diverse outdoor scenes.

Accomplishments that we're proud of

1)Integrated multi-stage analysis (detection → measurement → OCR → content classification → NSFW check) into a single, resilient pipeline. 2)Created two user-friendly frontends (desktop + web) that let non-technical users generate actionable reports quickly. 3)Implemented safe degradation: when optional models are missing the app still provides useful heuristic results instead of failing.

What we learned

1)Robust preprocessing (resizing, thresholding, denoising) materially improves OCR and downstream classification on real-world billboard photos. 2)Simple heuristics combined with optional ML models make the system practical across environments with constrained compute or internet access. 3)Clear, concise reports (cropped image + reason + metrics) are far more actionable for field teams than raw model outputs.

What's next for BillGuardAI

1)Improve geometric corrections (homography / perspective rectification) to produce more accurate real-world dimension estimates. 2)Add localization-specific rule sets and a small admin UI to maintain/update regional regulations. 3)Replace heuristic fallbacks with lightweight on-device models for offline scenarios. 4)Add secure, privacy-first cloud storage and multi-user dashboards with role-based access for municipal workflows.

Built With

  • can-produce-a-short-readme-snippet-listing-install-&-run-commands-for-windows-powershell.-sanjay1766-comma-seperated-please-github-copilot-python-3.x
  • easyocr
  • git-friendly-project-layout-if-you-want
  • gitgithub-copilot-built-with-languages:-python-3.x-ui-frameworks:-pyqt5-(gui-app.py)-for-desktop
  • github-copilot-built-with-languages:-python-3.x-ui-frameworks:-pyqt5-(gui-app.py)-for-desktop
  • hugging-face-models-(yolo12m.pt-+-optional-text/image-models)
  • huggingface-hub
  • images)-?-no-external-database-by-default-dev-/-tooling:-simple-local-scripts-(test.py)
  • json/file-based-storage
  • local-file-i/o-(json-export
  • numpy
  • numpy-ocr:-tesseract-(pytesseract)-with-easyocr-fallback-hugging-face:-huggingface-hub-for-model-download
  • opencv-(cv2)
  • optional-hf-text-&-image-models-platform:-cross-platform-python-(tested-on-windows);-desktop-app-packaged-as-pyqt-app
  • pandas
  • pillow-(pil)
  • plotly
  • plotly-(used-in-streamlit-dashboard)-networking-/-i/o:-requests
  • pyqt5
  • reports-folder)-packaging-/-deps:-requirements.txt-(project-dependency-manifest)-models-&-assets:-custom-yolo-model-hosted-on-hugging-face-hub-(yolo12m.pt)
  • requests
  • requirements.txt
  • streamlit
  • streamlit-(streamlit-app.py)-for-web-demo-cv-/-ml-libraries:-ultralytics-yolo-(yolov8)
  • tesseract-(pytesseract)
  • transformers-pipelines
  • transformers-pipelines-for-optional-text/image-classification-(nsfw-&-toxicity-models)-data-/-plotting:-pandas
  • ultralytics-yolo-(yolov8)
  • web-demo-via-streamlit-datastore:-file-based-exports-(json
  • windows-(tested)-/-cross?platform-python
Share this project:

Updates