ariot

Inspiration

Our cities are getting smarter, but they are not getting more homely. This is because the focus of Smart City technologies are on functional design but not on creating a homely environment. For example, IoT device managers such as Google Home attempt to centralize the command of all IoT devices and make smart homes even more convenient and feasible, but do not help people be more emotionally attached to their homes. We believe that the potential for Smart Cities lies beyond the functional aspect, and can also provide people with homely environments they feel emotionally connected to.

What it does

Our virtual pet, ariot, is a smart virtual AR companion that pushes IoT away from the realm of pure functionality. It takes autonomous behaviour to the next level and has partial control over all your IoT devices, applying mood-sensing on its owner at times to decide what to do next. It gives a personality to your Home IoT devices

How we built it

The AR app was built in Unity utilising the Vuforia library, coupled with the Microsoft voice recognition system. The assets used in the app were designed and created by us: 3D assets using ZBrush and animations using Unity's mechanim system with the help of ClipStudio Paint. We also used an Arduino in our prototype to simulate IoT devices which was sent instructions by a back-end consisting of Python and Firebase.

Challenges we ran into

The main challenge was undoubtedly the incompatibilities: Unity versions, Python versions, Operating systems, library versions, etc. Certain OS-dependent libraries in Unity (such as the Microsoft voice recognition system) could also not be built into Android apps. As we were not experts in IoT, we faced problems when connecting to the Serial port in the Arduino to input values to it. We eventually used the pyserial library but it took some effort getting the right version among all the buggy ones. The IBM Watson speech-to-text API also had poor performance and we had to find other ways to conduct the speech recognition.

Accomplishments that we're proud of

We successfully completed our Minimum Viable Product that we aimed for, and even overcame several unforeseen problems along the way. Our pet is also very cute and love-able and we are proud of its design.

What we learned

Firstly, we learnt a lot from our research on Smart Cities and IoT. While we originally thought that it was a fast-rising trend, many articles and sources have instead discussed about the slow rate of IoT adoption. We also learnt from our brief experimentation the drawbacks and limitations of AR applications (as well as the plethora of compatibility issues that was associated to them).

What's next for ariot

We want to implement the full working app on a handphone. We would also like to add Machine learning techniques to make the the virtual pet as realistic and intimate as possible.