Operational Manual Navigation Interface (OMNI)

OMNI open on the desktop, ready for input
OMNI giving directions, using data from a Shuttle User Manual

In aerospace, solving a problem quickly is just as important as solving it correctly. Spacecraft commonly have tens (some have hundreds!) of manuals, with some being thousands of pages long. Understandably, it takes even trained crew members many minutes to sift through these manuals and find the correct information and procedures.

Welcome to OMNI, the Operational Manual Navigation Interface! Trained on 11 different Space Shuttle technical manuals totaling over 6,000 pages, it can provide detailed information about any problem the user might experience. The result? The user can enter their problem in plain English and receive instructions in a matter of seconds!

OMNI utilizes AI models to their fullest to successfully return relevant data. First, we used an AI model to transform the complete text from over 6,000 pages of Space Shuttle technical manuals into a vertex database, easily and quickly searchable by a computer. Next, we used a Large-Language Model to convert the user's problem into a string that the computer can use to search our vector database, something that can be done far quicker than a standard keyword-finder. Finally, this LLM is given direct access to the text in a specific manual that outlines the solution to the problem the user is having.

Creating an embedding of each page of each technical manual that could be stored in a vector database took many hours, easily the most difficult part of the project!

We're definitely most proud of the fact that even queries as simple as "Engine go no no" result in detailed and appropriate steps to correct a main engine malfunction!

Before JSC Hack, neither of us had any experience with AI APIs, let alone using them in the way we did. We didn't even know what embeddings were! Now, they're the foundation of our entire project.

We really believe that this tool can help every astronaut and every pilot in the aerospace industry. While it's nowhere near perfect, we did only make it in 24 hours. With further development, we do think that this could become an invaluable tool!