Two photos taken, one before maintenance, one after
FOD #1
FOD #2

Inspiration

Aviation safety depends on countless small details being checked correctly, even when those details are easy to overlook. During aircraft maintenance, a forgotten wrench, a loose bolt, or a piece of cloth left in an engine area can become a serious hazard. Something so small can have consequences far beyond its size.

What inspired us was the idea that computer vision could help people catch these details before they turn into problems. We wanted to explore a practical question: Can an AI-assisted system help maintenance teams notice what might otherwise be missed?

We did not want to build a project that only demonstrated an AI model. We wanted to build something connected to a real-world safety challenge, where reliability matters and where even a small improvement could be valuable.

That question became the starting point for our AI-Powered Foreign Object Debris Detection System.

What it does

Our system compares images of an engine area taken before and after maintenance. It looks for visual changes that may indicate the presence of foreign object debris, such as tools, loose hardware, or leftover materials.

The system has three main stages:

Image comparison: It aligns the two images and identifies regions that have changed.
AI-assisted assessment: It extracts candidate regions and uses Amazon Nova Pro through Amazon Bedrock to assess whether they may contain a new physical object.
Human review: It presents the candidate regions and assessment results in a dashboard, where a person can review the evidence and record a decision.

The aim is to help maintenance personnel focus their attention on areas that may need another look. It does not assume that every visual change is dangerous, or that every detected region contains an object.

Most importantly, the system is designed to support human judgment, not replace it. In a safety-critical setting, an AI-generated result must be treated as evidence to review, not as a final guarantee.

How we built it

We built the project around a simple idea: compare what an area looked like before maintenance with what it looks like afterward, then investigate the differences.

We used Python and OpenCV to build the computer vision pipeline. Since the two images may not line up perfectly, the system first attempts to align them. It then compares the images, processes the differences, and extracts candidate regions for further assessment.

However, identifying a difference is only the first step. A changed region could contain a foreign object, but it could also result from lighting, image alignment, background details, or other harmless changes. To investigate these regions, we integrated Amazon Nova Pro through Amazon Bedrock.

We then built a Streamlit dashboard to bring the complete process together. Users can upload images, run the analysis, inspect candidate regions, view AI assessments, and record human review decisions. We also deployed the application on AWS EC2 to make the prototype accessible outside our local development environment.

We approached the project as an iterative experiment. We tested it, observed where it failed, and used those results to understand what needed improvement. Each stage taught us something different about the gap between a promising demonstration and a dependable real-world system.

Challenges we ran into

One of our biggest challenges was learning that a visual difference is not necessarily a foreign object.

Images can change for many reasons. Small differences in camera position, lighting, shadows, and alignment can create regions that look significant to a computer vision algorithm but mean very little in practice. Reducing these false positives without missing actual objects became an important part of the challenge.

Small objects presented an even greater difficulty. In our simulated tests, the system could identify larger objects, including a wrench and a piece of cloth, but it did not reliably detect smaller objects such as a nut and a bolt. This was an important result because these small items are precisely the kind of objects a system like ours should not overlook.

We also had to work through the practical difficulties of connecting computer vision, a cloud AI model, a web dashboard, and an AWS deployment into one working application. Making each component function was one challenge. Making the complete workflow useful and understandable was another.

Perhaps the most valuable challenge was resisting the temptation to judge the project only by its successful results. It would have been easy to focus on the objects the system detected. Instead, we also paid attention to the objects it missed, the regions it misinterpreted, and the situations where its output needed human verification.

Those failures did not make the project meaningless. They showed us where the real engineering work begins.

Accomplishments that we're proud of

We are proud that we turned an idea into a working end-to-end prototype. The system can accept images, compare them, identify candidate regions, request AI-assisted assessments, and present the results through a dashboard with human review functionality.

We integrated OpenCV with Amazon Bedrock and Amazon Nova Pro, then deployed the application on AWS EC2. This gave us experience not only with individual technologies, but also with the work required to connect them into a complete application.

We are also proud of the testing we carried out. In one unchanged-scene test, the system correctly reported no new items. In a simulated foreign-object test, it identified some larger objects but missed smaller hardware. These results gave us a more realistic understanding of what the prototype can and cannot do.

For us, an accomplishment is not only a feature that works. It is also a limitation that we can identify, reproduce, and explain. By documenting the system's failures alongside its successes, we have a clearer starting point for the next iteration.

We set out to explore how AI could assist with an important safety problem. We now have a working prototype that demonstrates the approach, exposes its limitations, and gives us something concrete to improve.

What we learned

This project taught us that applying AI to a real-world problem involves much more than connecting a model to an application.

We learned that the quality of the input data matters, that computer vision algorithms can be sensitive to small changes, and that an AI model's assessment does not automatically make an earlier detection correct. If the vision pipeline misses an object, the AI assessment stage cannot reliably assess an object it was never given.

We also learned to treat confidence labels with caution. A model can describe a candidate with high confidence and still be wrong about what is present. This makes evidence, uncertainty, and human review important parts of the design.

Perhaps the most important lesson was about responsibility. When a project concerns aviation safety, a successful demonstration is not the same as proof of reliability. A system must be tested against representative conditions, evaluated using annotated data, and examined for both false alarms and missed hazards before anyone can make strong claims about its performance.

We began by asking whether computer vision and AI could help identify possible foreign objects. Along the way, we learned to ask a more demanding question: how do we know when the system is right, what happens when it is wrong, and how can a person make a well-informed decision from its output?

That shift in thinking was one of the most valuable outcomes of the project.

What's next for AI Powered Foreign Object Debris Detection System

Our current prototype is a starting point, not a finished aviation safety solution. There is still important work to do before the system could be considered dependable enough for operational use.

Our next steps are to improve the detection of small objects, reduce false positives, and combine duplicate candidate regions. We also want to test the system against a larger collection of images containing different objects, lighting conditions, camera positions, and backgrounds.

A key priority is to build a properly annotated evaluation dataset. This would allow us to measure how often the system detects real objects, how often it raises false alarms, and how often it misses objects entirely. We also want to improve candidate prioritization so that regions requiring attention are easier to review.

Beyond detection, we want to make the results more useful to maintenance personnel. The system should clearly explain what it detected, show the relevant visual evidence, communicate uncertainty, and preserve the distinction between an AI suggestion and a human decision.

Our long-term vision is not to replace the people responsible for aircraft safety. It is to explore whether a carefully designed AI-assisted tool can provide an additional layer of support, helping people direct their attention toward details that deserve another look.

We started with a simple question: could AI help us notice something that should not be there? Building this prototype helped us explore that possibility. The next challenge is to make the system more reliable, test it more rigorously, and understand exactly where it can help—and where it still needs to improve.

For us, that is what makes this project worth continuing.

Future Scope

The current prototype focuses on before-and-after image comparison using OpenCV to identify candidate changes, with Amazon Bedrock Nova Pro assisting in the assessment of selected regions.

Future development could extend the system in three directions:

Live Monitoring: Integrate camera feeds or periodic image capture to identify changes in maintenance areas.
AI-Assisted Analysis: Expand automated assessment and prioritization of potential foreign objects for human review.
AI-Assisted Inspection Proctoring: Introduce guided inspection checklists and help verify that required inspection steps have been completed.

These features are planned enhancements and are not implemented in the current version. The goal is to evolve the prototype into a more comprehensive inspection-support tool while keeping human verification central to the process.