Family Memory Vault

Each family holds treasured memories in the form of videos, voice notes and pictures. When a family member passes or lives far away, it can be hard to find when they have recorded certain memories, like "the time grandpa told us how he met grandma." We wanted to be able to ask questions relating to certain memories and receive actual recorded responses. What it does: Family Memory Vault is a Skill for the UGREEN AI NAS. A family member will ask a question in a language that the NAS can understand, currently we plan to support English, Spanish, Arabic, Urdu and mixed speech. In response, the Skill will search through the user's authorised folders for the correct recording. It will return:

A link to the file and when the story started.
A transcript of the recording and when it happened.
A brief summary of what was said.
Photos from the same time. If there are no matches, it will return ‘No recording found for this question’. There will never be fake voices or responses. Example: If a user were to ask, "What did Grandpa say about how he met Grandma?" rather than giving them a general knowledge response, Family Memory Vault will search through all of the authorised recordings. It will find the correct conversation and provide the original recording with a timestamp, a transcript, a brief summary and any photos from the same time. This allows the family to watch the memory, rather than a computer-generated version. How we made it: This Skill was created using an orchestration workflow following the Uliya Skill Development Standard. It calls 4 different NAS experts and each stage will pass the actual results onto the next stage.
File-agent will look through the folders for recordings (ug-filesystem).
Knowledge-agent will create a transcript with timestamps and find the right passage (ug-knowledge).
Photo-agent will find any photos from the same time (ug-photo).
Document-agent will read any related notes (ug-document). All data is stored on the NAS, no internet is used and no data is uploaded. Everything is read-only. The only writing is creating a transcript note, however this will only happen if the user has approved. The Skill handles success, failure, cancellation, waiting_user and waiting_approval. Challenges:
Making sure we followed all the safety and privacy rules set out by the platform. We had to ensure we got permission to view data, we weren’t sharing any data, we weren’t creating any biometric identification and we weren’t overwriting any files.
Ensuring that all claims had the label of automatic and noted that the date of the file may not match the date of the event.
Sticking to the published capability list so each stage had a capability.
Not doing any extra work. We used existing transcripts and asked the user to be specific with their search. Proud Moments:
Creating a concept and user flow for a project that would help families store and find real memories.
Creating an orchestration workflow that would link 4 NAS experts to find recordings, transcripts, photos and notes.
Completing the SKILL.md and test cases.
Successfully running the package through the Skill Kit static checker. We had to fix an issue with the formatting in the capability section.
Including privacy, permission and approval throughout the workflow.
We’re very proud that we created a system that knows when it can’t find an answer and won’t make one up. Lessons learnt: We learnt how to create something that uses the capabilities of the platform. We learnt how to include no answer as a valid and trustable result. We ran our package through the Skill Kit static checker. Future: We’ve completed the concept, user flow, SKILL.md and test cases. Our package runs through the static checker. Our Skill isn’t fully created yet. If we become a Finalist we’ll test our workflow on the NAS. Testing:
Whether the transcription is accurate for all supported languages (English, Spanish, Arabic, Urdu and mixed).
Timestamps and summaries are accurate.
Consistently finding the correct recordings for the questions.
Finding related photos and notes accurately.
Permission and privacy are working as intended. Our aim long term is to make it easier for families to find, watch and store their memories whilst keeping them authentic and private.