Around 70 million people require wheelchairs worldwide. They suffer from physical barriers, social exclusion, health and safety concerns, all of which have psychological impacts. In addition to using a wheelchair, people with disabilities such as ALS, Multiple Sclerosis or Spinal Cord Injuries experience even more limited connection with their environments due to their lack of upper extremity and neck mobility. This leads to a significant decrease in people’s fields of vision, speech, or ability to point: these patients face significant challenges to communicate.
The lack of suitable assistive technologies to help them navigate safely inspired us to work to break down the barriers of social exclusion and help them increase their independence. We present the development of an augmentative system, Loro, for people with movement disabilities (mostly ALS, MS, SCI or CP people) to communicate with their loved ones and to enrich their interaction with the world around them, fundamental to successful social integration.
What it does
Loro is a platform for providing a smart companion robot for wheelchair users (ALS, Multiple Sclerosis, Muscular Dystrophy, Spinal Cord Injuries, Cerebral Palsy). Loro integrates several technologies including a universally mountable 360 camera, laser pointer, flashlight, alarm system, text-to-speech, speech-to-text, and user-friendly interface. Loro leverages artificial intelligence, cognitive services, and Amazon Alexa. Loro can help people with speech impairments to be recognized by Alexa and access the voice service. Loro will send the commands to Alexa by text-to-speech. Text queries are created on Loro's software which will be available for all platforms: Windows, iOS, and Android. Users can control Loro through a joystick, eye-tracking software, a head-mouse or a touchscreen app. Loro is compatible with multiple platforms including iOS, Android, and Windows. Also, Loro works in auto-mode to detect objects and identify people, the environment, and obstacles. Loro is designed to provide users with an intelligent camera, safe navigation, health monitoring, social interaction and smart home connectivity.
How we built it
To develop Loro, we have been using an engineer-and-iterate approach. Throughout our user-centered design process, we have been working closely with potential customers to design an ongoing user research plan to fully explore which features Loro should implement that fulfill their needs. The current Loro stage is a result of an extensive design process:
Prototype I: With 7 passionate individuals at MIT’s ALS Assistive Tech Hackathon, our team was formed. We spent the first half of the hackathon learning about daily challenges of people with ALS from members of the community. We identified that a big problem they faced was the limited field of vision and communication speed. We drew tens of prototype ideas and began building away until we won the hackathon: a GoPro mounted in a 360 motor controlled with a joystick and a rudimentary user interface.
Prototype II-IV: For the next 6 months, we prioritized learning from our users to understand how our technology could be molded to fit their lives. After each user visit, we would add a suggested feature to our camera system: an eye-gaze controlled system with a camera, laser pointer, text-to-speech, speech-to-text. The team begins by drawing out software and hardware ideas, and the engineers work with designers to find feasible starting points. After weeks of building, the team reconvenes and finds new features to add or modify.
Prototype V: The past 3 months, our engineers have conducted 30 user tests to design more friendly UI. Per demonstrated need, we’ve incorporated: health monitoring system, video recording, object recognition. We will continue to brainstorm, incorporate feedback, build.
Challenges we ran into
We are engaged to fulfill four main values: Vision, Communication, Memories, and Safety. These values represented the challenges we ran into:
Vision: by a universally mountable 360 camera with a flashlight to offer a visual coverage of the environment. During the earlier stages of development, we bought several cameras to understand which one fulfills all the requirements at a reasonable price.
Communication: by an interactive speech to text and text to speech with a laser pointer to enrich the users' interaction with the world around them. We faced several questions: How to incorporate this feature in our own app? How can we use this to improve smart home system? How to bring the Internet of Things (IoT) to people with physical challenges from home devices to connectivity to advanced analytics? Regarding this, the in-built speaker and microphone system were added as communication methods.
Safety: by building a tool that enhances outdoor navigation for wheelchair users. We promote and enable an accessible exploration to our users, by identifying the physical obstacles around them and suggesting the best routes to take. With the inclusion of this application, we expect Loro to provide users with an increased sense of safety, feeling of independence, and quality of life.
To address the mentioned challenges, an accessible UI designed in collaboration with end-users integrates with existing technologies: Microsoft Tobii, the artificial intelligence API by AWS, Alexa Skills Kit (ASK) for smart devices to provide cohesive and intuitive control.
Accomplishments that we're proud of
During the development of Loro, we have been pursuing all the opportunities that could help take Loro to the next level. These are some of the main accomplishments of Loro:
1. 1st winning team in Fit4start in luxumbeurge, Oct 2018
2. Nominated for: Microsoft Imagine World Cup, July 2018
3. 1st Place: MIT Assistive Tech Hackathon Final Round, June 2018
4. 3rd Place: Microsoft U.S. Imagine Cup 2018, May 2018
5. 2nd Place in USA: Livewell USA App Challenge, June 2018
6. Finalist: in TechCrunch disrupt SF, 2018
7. Finalist: University Startup World Cup 2018
Other measurable accomplishments are related with the community/user involvement and engagement. Although Loro is in its early stages, we are able to engage with several patients (thirty cases in Massachusetts area) that work with us on a regular basis. This is the most important to us because through their limitations, our user testing community is enthusiastic to help take Loro to the next level.
What we learned
Designing new technologies is all too often set in the framework of a technology-oriented mode of thinking about reality, separating the technical factor from the human factor. As a team of engineers, the engineering-driven product development could be very attractive. However the community involvement in Loro was fundamental. The patients helped us gain a better understanding of the deprivations imposed to persons with motor impairments. So we have learned to listen to end-users to understand their pain points. Nowadays, we already know challenges that cause frustration, emotional discomfort, irritation or hassle to our potential customers, however this is a work in progress.
Designing a tool for patients suffering from ALS and other movement disabilities amounts to trying to alleviate a severe deficit in possibilities for interaction by means of an equipment offering some practical comfort in their life.
Loro should be then always conceived on the basis of the user’s needs, in order to make it possible to recreate the forms of interaction which have been lost.
Additionally, we have learned and improved our software and hardware engineering knowledge. Every new feature that we added to Loro required an effort to understand how we should implement.
The need of pursuing several and different funding opportunities or accelerator programs is also pushing our team outside our comfort zone but we really improved in identifying the goals in order to reach more and better.
What's next for Loro
Next, our team is refining software for cross-platform use (Android, iOS, PC) and building a more compact robot body.
Design wise, Loro has more than 25 user tests lined up with ALS, Multiple Sclerosis, Muscular Dystrophy, Spinal Cord Injuries, Cerebral Palsy users.
We are optimizing existing technology and adding new software features to fulfill different patient population needs. We will continue to iterate to improve Loro through ongoing product testing.
We will be approaching wheelchair companies, disability support groups, and NGOs, to establish partnerships and continue advancing our development.