The Problem
Checking packaged commodities for Legal Metrology compliance is a repetitive and detail-heavy process. Inspectors need to verify multiple declarations on a package, including product name, net quantity, MRP, manufacturer details, dates, country of origin, and consumer-care information.
A simple OCR system is not enough. Finding text does not mean that the declaration is correct, present in the required context, or properly positioned and readable.
We built LM-Check to turn this process into an AI-assisted digital inspection workflow.
What We Built
LM-Check combines computer vision, OCR, and a rule-driven compliance engine to analyze product packaging.
The workflow is:
Scan → Extract → Validate → Classify → Explain → Report
A package image is processed using PaddleOCR to extract text and its spatial information. The extracted information is then mapped to legally relevant declarations and evaluated against a structured set of Legal Metrology rules.
The system can identify required declarations such as:
- Product / generic name
- Net quantity
- MRP
- Manufacturer / packer details
- Manufacturing and expiry information
- Country of origin
- Consumer-care information
- Optional sector-specific declarations
Beyond text presence, the system also considers factors such as placement, readability, and measurable text characteristics where applicable.
The final result is classified as:
- PASS — no detected issue
- VIOLATION — a potential compliance issue was identified
- REVIEW — the system cannot confidently determine compliance and requires human verification
The system also produces supporting evidence, remediation guidance, and inspection reports.
How We Built It
Backend
The backend is built with Python and FastAPI.
It provides APIs for:
- Image scanning
- OCR processing
- Compliance analysis
- Rule retrieval
- Reports
- Evidence generation
- Inspection history
- Repeat-offender tracking
Computer Vision & OCR
We use PaddleOCR together with image-processing techniques to extract both textual content and spatial information from packaging images.
This allows the compliance engine to reason about more than just the raw OCR text.
Compliance Engine
Instead of asking an LLM to make unrestricted legal decisions, the core compliance logic is implemented as a structured rule engine backed by a versioned guardrails.json configuration.
This makes individual compliance checks explicit, inspectable, and easier to update.
Frontend
The inspection interface is built using Next.js, React, and TypeScript, providing a dashboard for scanning packages, viewing detected declarations, inspecting violations, and accessing generated evidence and reports.
Data & Evidence
Inspection information is persisted using SQLite, while generated evidence and reports provide a record of what the system detected and why a particular result was produced.
Deployment
The backend is containerized using Docker, with the application designed around a FastAPI service that can be deployed independently from the frontend.
What We Learned
The biggest lesson was that OCR accuracy alone does not solve compliance inspection.
A useful inspection system needs to connect several layers:
Image → OCR → Structured Information → Rules → Evidence → Human Decision
We also learned the importance of handling uncertainty. Real-world packaging contains unusual layouts, poor image quality, multilingual text, overlapping elements, and ambiguous declarations. Therefore, forcing every case into a binary compliant/non-compliant decision can be dangerous.
This led us to introduce the REVIEW state and keep the system focused on assisting inspectors rather than replacing regulatory judgment.
Challenges
1. Connecting OCR With Compliance
OCR returns text and coordinates, while regulations describe semantic declarations and requirements. Mapping the two required a separate interpretation and validation layer.
2. Spatial Validation
A declaration can exist on the package but still have problems with its placement or readability. This required combining OCR bounding boxes with image-processing logic.
3. Handling Uncertainty
Not every package can be reliably evaluated from a single image. Instead of hiding uncertainty, the system explicitly surfaces cases requiring human review.
4. Generating Useful Evidence
An inspection result is much more useful when an inspector can understand what was detected and trace the result back to the package image and extracted information.
Why It Matters
LM-Check demonstrates how AI can be applied to a real regulatory workflow, rather than simply using an LLM as a chatbot.
Our goal is to reduce repetitive manual verification, improve consistency, and give inspectors structured evidence and actionable information during package inspections.
The system is an inspection-assistance prototype and does not replace official regulatory or legal judgment.
Future Scope
Potential extensions include:
- Improved multilingual OCR
- More robust detection across packaging layouts
- Larger-scale inspection analytics
- Cloud-based evidence storage
- Mobile inspection workflows
- Integration with official enforcement systems
- Continuous evaluation against real inspection datasets
Log in or sign up for Devpost to join the conversation.