Stellies ASUS

Inspiration

Hi! We are ASUS (the first letters of our names) and we would like to introduce you to our amazing assistant V.A.L.S.E.A. (pronounced val•see•ah). The idea of VALSEA was brought about when one of the ASUS member's mothers, an electrical engineer, confided about the difficulties new engineers faced when going onsite to high voltage power stations in South Africa. Although there are many measures put in place for the safety of the engineers and technicians, in such an environment loss of focus and lack of knowledge can and has led to the deaths of engineers and technicians. These deaths are highly avoidable, and this is the mission of ASUS with VALSEA.

What it does

VALSEA is envisioned to be a multipurpose, voice-activated assistant that is able to readily provide reliable, context-specific information to line service engineers that assists them with undertaking the task they are currently completing. Where VALSEA provides the most value is in how users interact with her. The interaction is done completely via voice. The app is installed on your smartphone and initiated by a simple pre-assigned voice command (like "hey VALSEA") followed by the enquiry. VALSEA will process the enquiry and provide context specific feedback which is then relayed back via voice. This complete voice interaction prevents line service engineers/technicians from taking their focus off the job at hand while also providing them with the information they need. It's similar to having a conversation with a knowledgeable peer right next to you.

How we built it

VALSEA is powered by a Retrieval-Augmented Generation (RAG) model built on the Databricks Lakehouse Platform. We created a vector index within Databricks to enable efficient semantic search through our context-specific data. This indexed data is then used to supplement pre-trained models hosted on Databricks. The combined power of these models and the context-specific data allows VALSEA to generate highly relevant responses. To facilitate natural voice interaction, we developed a user-friendly front-end using the Flutter framework.

Challenges we ran into

Building VALSEA presented a unique learning curve. Our initial hurdle was navigating the Databricks Lakehouse Platform, particularly with limited experience in data lakehouses, APIs, serverless compute, and cloud compute. This knowledge gap made it challenging to determine the most efficient approach that effectively utilised Databricks' functionalities. While tutorials were available, the cost associated with running compute resources on Databricks (utilizing AWS) became a significant barrier. This limited our ability to freely experiment with Python notebooks, hindering the learning process and slowing down development. I.e. There was always a fear that costs were being incurred.

Accomplishments that we're proud of

Achieving success with the RAG model was immensely gratifying.

What we learned

VALSEA's development journey was a rich learning experience on both technical and collaborative fronts. Technically, the project equipped us with proficiency in Databricks, Flutter, and Langchain, among other tools. For some ASUS members entirely new to deep learning models, incorporating them into the application presented a significant learning curve. However, we effectively navigated this challenge by leveraging each other's strengths. Recognising our limitations in combined programming knowledge, we adopted a team approach. Those less familiar with coding became the vital business team, focusing on use-case development, data acquisition, and sales strategy. This approach ensured all skillsets were utilised effectively to bring VALSEA to life.

What's next for Stellies ASUS

The ASUS team is incredibly passionate about VALSEA's potential to empower South Africans. We're committed to building upon this foundation. Our roadmap includes implementing features like voice-activated wake words for a seamless user experience. We envision VALSEA becoming a scalable platform where users can contribute and request specific learning materials. Additionally, we plan to enhance VALSEA's conversational capabilities by incorporating context awareness through conversation history. This, along with further exploration of machine learning techniques, will significantly improve the reliability and natural flow of voice interactions.

Built With

• amazon-web-services
• dart
• databricks
• flutter
• langchain
• ml-flow
• openai
• python
• vector-search
• vscode