MISXMATCH — AI-Based Missing Person Search & Reunification Platform

Inspiration

Every year, thousands of missing-person cases remain unresolved because information is fragmented across different organizations, reports, and databases. Traditional identification processes often rely on manual searches, making investigations slow and resource-intensive. We were inspired to create a technology-driven solution that could help authorities, hospitals, NGOs, and communities collaborate more effectively to locate missing individuals and reunite families faster.

What It Does

MISXMATCH is an AI-powered missing person search and reunification platform that combines multiple sources of evidence to identify potential matches. Instead of relying solely on facial recognition, the system performs multimodal matching using facial similarity, person re-identification, text similarity, location analysis, and temporal information. The platform generates explainable match candidates while ensuring that final decisions remain under human supervision.

The system supports missing-person reports, found-person reports, hospital intake records, shelter registrations, citizen sightings, evidence uploads, and AI-assisted investigation workflows.

How We Built It

MISXMATCH was designed using a microservice architecture to ensure scalability, maintainability, and clear separation of responsibilities.

The platform consists of:

• Spring Boot microservices for authentication, case management, notifications, and API gateway operations.
• A FastAPI-based AI service for biometric matching, NLP similarity analysis, OCR processing, and risk scoring.
• React and Vite for a modern, responsive frontend experience.
• MySQL databases for secure data management.
• MinIO object storage for handling images and supporting evidence.
• Docker-based deployment for simplified setup and portability.

The AI engine combines facial embeddings, person re-identification models, sentence-transformer-based text similarity, OCR extraction, and multi-factor scoring to identify potential matches across different sources of information.

Challenges We Faced

One of the biggest challenges was balancing accuracy and responsibility. Missing-person identification is a sensitive domain where false positives can have serious consequences. To address this, we implemented a Human-in-the-Loop workflow that requires authorized officers to review and approve AI-generated matches before confirmation.

Additional challenges included:

• Handling low-quality and CCTV images.
• Combining multiple similarity signals into a meaningful confidence score.
• Designing explainable AI outputs instead of black-box decisions.
• Building secure role-based access control for multiple stakeholder groups.
• Maintaining auditability and transparency throughout the investigation process.

What We Learned

Through this project, we gained practical experience in:

• Microservice architecture and distributed systems.
• Computer vision and facial recognition.
• Person re-identification techniques.
• Natural language processing and semantic similarity.
• OCR and document intelligence.
• Secure authentication and authorization.
• Responsible AI and Human-in-the-Loop system design.

Future Improvements

Future versions of MISXMATCH will focus on:

• Real-time video stream analysis.
• Advanced multimodal fusion techniques.
• Improved large-scale matching performance.
• Mobile application support.
• Enhanced analytics and investigation dashboards.
• Broader integration with public safety and humanitarian organizations.

Impact

MISXMATCH demonstrates how AI can be applied responsibly to address a real-world humanitarian challenge. By combining advanced multimodal matching with human oversight, the platform aims to reduce investigation time, improve identification accuracy, and increase the likelihood of successful family reunification.

Built With

• ai
• computer-vision
• docker
• face-recognition
• fastapi
• java
• javascript
• jwt
• microservices
• minio
• mysql
• natural-language-processing
• ocr
• opencv
• person-reid
• python
• pytorch
• react
• rest-api
• role-based-access-control
• spring-boot
• spring-cloud-gateway
• spring-security
• vite