With our project we are attempting the Sun Life Financial: Manage Elderly Care From A Distance challenge with Nuance: Nuance's Mix Tool Challenge.


We initially started with a simple prompt: to "create an application with a general theme of providing, supporting, and managing elderly care from a distance." From that basic idea, we decided to think of the most important aspects of an elderly individual's life which was simplicity and automation in mind. We wanted to create something that would start as a simple idea and a rough mock-up in this hackathon but could be expanded with limitless potential. The result is a ubiquitous service that can be used virtually anywhere, from browser to phone apps! With Nuance-Mix's versatile API, this program can have additional features added.

What it does

Due to the limited amount of time, we chose to only build a web-version accessible from all types of devices. This website uses the javascript version of Nuance's speech-to-text detection program to get replies from the client using this program. It offers a sleek log-in screen as to be client-specific and to reduce the amount of setup required by companies thinking of employing our program. The program can be set up to remind the user after a certain interval on a task they must complete which can be important for their health and wellbeing. Individuals that have issues remembering small details can benefit greatly from our program as it can send a reminder at the designated time for the task. The autonomy of our program removes the necessity to constantly set new date and times which can be a hassle. Keeping the elderly in mind, we also wanted a "safety system" to alert their caretakers during a potential fall or injury.

How we built it

We built the front-end website from scratch by creating the base and look purely by hand. Relying on Nuance's API, we generated a JavaScript file which initialized and managed the exception handling of the application.

Challenges we ran into

Unfortunately, we were not able to properly configure the JavaScript file to handle our intents, concepts and dialogues due to our limited knowledge of the language and issues with back-end integration on our website. It was a struggle dealing with new software, especially Nuance's Developer toolkit but we learned a lot about a cool service and got to learn a lot about its power from Vincent. Although our "dialogue" function did not work, which was supposed to interact with the user, we instead opted to simulate what a sample dialogue is supposed to look like. Attached are also photos of the example dialogue and our settings with Nuance's Mix toolkit.

Accomplishments that we're proud of / What we learned

We're very proud of the work put into making SPiel a somewhat working model of scattered bits and pieces. Although we couldn't unify it, we learned the importance of quick adaptability during a very short and limiting time-frame. We were also able to plan and follow a concept which initially started as a random pitch during lunch to something that could benefit many individuals. We learned teamwork and time management to maximize our productivity and output in a short timespan. Good leadership and planning is important or else you risk losing direction and focus which would lead to nothing getting done.

What's next for SPiel

Further development will allow SPiel to expand, connecting elderly individuals living by themselves with local caretakers. Each caretaker will be able to view a database of the individuals under their care displaying last activity (and generating a warning message if the user fails to be active for an extended period of time) as well as any requests for help (broken down into urgent and less so) made by each user. Eventually, the goal of the application is to incorporate every aspect of daily life of the elderly while maintaining a simple, easy to use interface that would allow users with little to no technical knowledge to use it effortlessly. These include daily weather updates, meal suggestions (tailored to fit dietary restrictions), and more thorough dialogue support.

The site is hosted on localhost. You may try and host the zip file. We have also added screenshots and sample uses.

Share this project: