Inspiration
Every missing-person case represents more than a record in a database — it represents a family waiting for answers. In real-world situations, information about a missing person may be scattered across police reports, CCTV footage, public sightings, hospitals, shelters, NGOs, and citizen reports.
We wanted to build a system that could bring these fragmented sources together and help investigators identify potentially relevant connections faster.
This led to MisXMatch — an AI-Based Missing Person Search and Reunification Platform.
What it does
MisXMatch provides a centralized platform for managing and searching missing-person cases across multiple sources.
The platform supports:
Missing-person and found-person reports Public sighting submissions CCTV image records Hospital, shelter, and NGO records Location-based case information Role-based dashboards for different stakeholders Case tracking and status management AI-assisted matching between missing-person records and potential matches Image-based facial similarity analysis Text-based description matching Match scoring and priority-based case investigation Verified-authority workflows for sensitive operations
Instead of treating every report independently, the system is designed to connect related information and surface potentially relevant matches.
How we built it
The application follows a full-stack architecture.
The React frontend provides role-specific dashboards and interfaces for submitting, searching, tracking, and reviewing cases.
A Spring Boot backend handles business logic, authentication, authorization, case management, and REST APIs. JWT-based authentication and role-based access control are used to separate access between different types of users.
MySQL is used to store structured case, user, report, location, and status information.
For AI processing, the system uses a separate Python/FastAPI service. Computer-vision and language-processing components are designed to process different forms of information and generate similarity signals that can support case investigation.
The overall architecture allows the application layer and AI layer to remain modular, making it possible to improve the underlying models without redesigning the complete platform.
What we learned
Building MisXMatch required us to work across several areas rather than treating it as only an AI project.
We learned how to:
Design and integrate a full-stack application Build REST APIs with Spring Boot Implement authentication and role-based authorization Connect a React application with a relational database Design workflows for different types of users Integrate Python-based AI services with a Java backend Work with computer-vision and natural-language processing pipelines Handle real-world data challenges such as incomplete information and different image conditions Design systems where AI supports human decision-making rather than replacing it
One of the most important lessons was that building an AI system is not simply about achieving a high metric. The quality of the data, evaluation methodology, system architecture, and responsible interpretation of predictions are equally important.
Challenges
One of our biggest challenges was dealing with the diversity of information involved in missing-person investigations.
A person's appearance can change because of lighting, camera quality, pose, distance, occlusion, and image resolution. Similarly, the same physical description can be written using completely different words.
For example:
"black pants and white shirt"
and
"dark long trousers and a light-colored T-shirt"
may describe essentially the same appearance despite having very different wording.
This motivated us to explore both visual similarity and semantic text similarity rather than relying on a single matching technique.
Another challenge was designing the system so that AI-generated matches are treated as investigative assistance rather than automatic confirmation. A similarity score should help prioritize potentially relevant cases, while final decisions remain subject to verification.
Impact
MisXMatch is designed to reduce the fragmentation of missing-person information and provide investigators and authorized organizations with a unified way to search, connect, and prioritize relevant records.
The long-term vision is to develop the platform into a responsible AI-assisted ecosystem where information from multiple sources can be analyzed together while maintaining appropriate verification, access control, and transparency.
Future Work
Future improvements include stronger multimodal retrieval, improved handling of low-quality CCTV imagery, temporal analysis, more robust location-aware matching, larger-scale evaluation, and integration with verified institutional data sources.
The ultimate goal is not simply to build another missing-person database, but to create an AI-assisted investigation platform that can help turn fragmented information into meaningful leads while keeping humans in control of critical decisions.
Built With
- axios
- deep-learning
- fastapi
- framer-motion
- java-17
- javascript
- leaflet.js
- lottie
- lucide-react
- mysql
- opencv
- python
- react-19
- react-hook-form
- react-query
- recharts
- redux
- rest-apis
- shadcn-ui
- spring-boot
- spring-cloud-gateway
- spring-security
- tailwind-css
- vite
Log in or sign up for Devpost to join the conversation.