E-Kairos

Inspiration

E-Karios is a AR/VR solution which helps to solve issues with both cultural institutions and education and can be applied to multiple fields, from site heritage to immersive educational experiences, E-Karios offers a unique solution to revitalise cultural institution and education in the digital era. The project began on the 25th of July and has been improving since. All content has been created and designed by me within the weekend (25th and 26th).

Last year after a trip abroad, my family and I planned to visit many monuments and landmarks, but due to unforeseen circumstances we were not able to visit the institution. Inaccessibility, from unforeseen circumstances or even renovations, could cause a huge loss of tourism coming at a huge cost for these institutions.

The second issue which I came across was the damage caused by tourism in certain cultural/historic institutions, which caused negative effects to the institution physically and financially. Whilst many solutions for virtual museums and tours are available, there is market gap for immersive experiences which allows for user inputs (e.g. Q and A).

A solution which could help provide an immersive experience to act as a hub for information is needed for cultural institutions to provide a more accessible service. For many having the opportunity to visit these places can be amazing and possibly change the experience with education.

Ultimately these cultural institutions can make a huge effect on education worldwide, and despite 3D solutions on the market, very few truly offer an immersive feature.

Today, we have seen two problems caused by COVID-19 which has negatively affected these institiutions, which find it hard to make money and survive without any customers. Moreover, COVID-19 has impacted immersive education more than every before, and many look to these cultural institutions for inspiration to help enhance their education. A solution to help students to achieved immersive education at home is required.

What it does

1) AI-Powered Chatbot Host for the Tour

2) 3D Visualisation

3)Q/A with the virtual host

How I built it

Chatbot Chatbots were created using Amazon Lex, with around 2-3 questions for each intent, to ensure the chatbot could understand them with a high success rate. I then added responses for each of the instances, providing answers to the different questions. Currently, there are a few available, but this can be added later on.

Greek Temple For the demo, I created the model using blender where I added a few materials, manipulating planes, cubes and spheres. This took around an hours to make ! The pillars were made using cylinders and edge loops, and extruded inwards.

Sumerian I brought it all together using Amazon Sumerian, which allowed for the linking of the amazon lex bot to a demo host model. In this case, I had to simply connect the dialogue from Lex through the bot, and create a simple flow between asking the question and responses.

Challenges I ran into

Chatbot : Parts of the section of Lex included debug errors in the host's speech, to change this I had to change the order of the responses and reduce certain aspects, eventually solving the issue. Additionally, certain phrases were not being picked up hence I was forced to make more questions to train the model. By adding more data, the bot was more responsive and achieved higher fidelity in understanding responses. More can be added !

Importing the Blender model in Sumerian: The imported model caused huge amounts of errors, with textures not importing properly along with incorrect lighting. As a result, I spent time altering the textures and lighting until the model was similar to the blender model.

Controlling Movement: Originally, for class-platform use, I added the ability to touch to call the host. However, this made it impossible for the user on a mobile to travel around the model without triggering the chatbot ! To alter this I change the input.

Accomplishments that I'm proud of

First AWS VR/AR Project: This was my first AWS Sumerian project, and I am still quite new to AWS. Overall, I learned a range of new skills within this hackathon period ! Normally, my speciality is hardware (e.g. arduino projects) but I wanted to move out of my comfort zone, using new skills !

Blender Project: Normally, I also model using software such as Fusion or Solidworks, for project such as robotics and hardware builds ! However, this project led me to use blender, due to versatility ! Hopefully, I can improve these newfound skills in the future !

Educational Understanding: Within this hackathon,I learnt about educational immersion and the effects of this form of learning as opposed to reading textbooks and simple images etc. When I tested it out it was clear the impact the immersion made on usability and retention !

What I learned

AWS Lex: Through trial and error, I was able to learn how to develop an AWS lex chatbot. This led me to learn not only about lex, but understanding how it works (artificial learning) and why it sometimes fails ! This taught me the basic of machine learning, which I would like to explore in future hackathons.

AWS Sumerian: This was my first project using AWS Sumerian, and the first time I used a software similar to sumerian. Through this I learn a lot about lighting, animations which is really useful for future AWS projects, and also learning software such as Unity (on my to-do list !)

Education Communication: Communication methods for effective teaching and interactions to boost the impact on learning. First type of project, which looked at education and immersive experiences.

Time Management: I only had one to two days (even less really) for this submission, and to ensure I made the deadline I practically applied time management skills for an independent project.

Blender: How to use blender for designing and rendering visuals (i.e/ Greek Temple) and how to export and use in external software (i.e. summerian). I learnt a lot about the features as well as lighting !

What's next for E-Kairos

Further data to streamline AI Chatbot, looking at features involving lambda, external APIs to collect learning info and integrating tests for students !

Possible application using Unity for increased immersion, possibly using different chatbot software (even perhaps making my own from scratch) !