The best time to capture what was done is when it's getting done.
Maintainers face multiple challenges to keep craft mission-capable. Legacy data entry systems are error prone and require extra effort on the part of maintainers or supplemental staff to even record data.
What it does
VADER is a voice activated data entry system that lets maintainers log activities hands-free in real time as easily as they can talk to a smartphone. Simply say "Hey Vader" to begin logging an action and say "Later, Vader" to stop.
As you talk, VADER transcribes in real-time to give you:
- feedback about what it understood in terms of procedures and entities mentioned
- links to relevant documentation, schematics, and job details to help get this craft FMC ASAP
- a continuous record of what activities were performed
- a pre-filled log entry to add to legacy systems
Maintainers can speak in "real talk" to keep their workflow steady. VADER maps this to relevant data for log entries and prompts the user for missing information when the JCN is closed.
How we built it
VADER is built by taking audio from a microphone and extracting entities from the text to link to a knowledge graph purpose-built to help acquire JCN log entries. VADER links to documentation and schematics too so that once a component or subsystem is identified, it can prompt you to see if any of the available references for that subsystem are useful for the task at hand and show them to you using an AR display.
By providing context-specific documentation to the maintainer in real-time, we get valuable feedback about the subsystem being maintained which can be used to improve the quality of the maintenance records generated. This can provide real-time feedback about the quality of the job control entry in progress.
Challenges we ran into
- Time and resource constraints
- Appreciating the nuances of a new domain
Accomplishments that we're proud of
- Working with the maintainers to improve our user-experience design and get towards an acceptable model for real-world conditions
What we learned
An enormous amount about the real-world constraints of the maintainers.
What's next for VADER