Introduction

There’s a very apparent buzz around AI agents interacting with the world right now, and companies are investing billions in their development, we are living in an era of AI Boom. Yet there’s a rudimentary hitch: most of the web as we know it was not designed with AI in mind. We want to take a step to better this with Contex: where developers can create customized components tailored for AI traversal or use one from our extensive library, making the web more accessible and inclusive through AI Agents.

We have tags that give context to the AI. It explains what the element is and a prompt that explains what happens when you use it. This allows the AI to navigate the web page easily as it is in a format it understands. This allows us to link in other tools such as a cursor which will click on the buttons the AI finds. This allows the likes of blind people or amputees to navigate the web more easily than before.

Technologies

Contex is built using TypeScript, React, Python, Django, NextJS, OpenAI, Whisper, and ElevenLabs. The front end is built completely with TypeScript and React. We allow users to create custom AI Native components through a chatbot interface. The responses are optimized using prompts augmented using RAG, we use FAISS as a vector storage that we built from libraries of code that were used to create these AI-friendly components. Whisper is used for speech input transcription and ElevenLabs for Text-To-Speech (TTS) Output (converting messages from the LLM) and Zustand for state management. We used Clerk for authentication allowing people to let us sign up through their GitHub, Google, and Microsoft accounts.

Journey of Development

We faced a lot of challenges while developing Contex. We had to re-design the entire site to allow for the stable use of AI navigation systems around Contex. We used components that Contex generated to design the site. The site was quite difficult to design too as we were not very familiar with React. Using NextJS was a challenge to some of our team who never used NextJS before. Prompt engineering was exceptionally difficult too as we tried prompting the Open AI Assistant to create more specialized responses for us. Testing proved difficult too as some components were harder to debug and test than others.

We’re proud of achieving the goals that we set out at the start of the hackathon. We worked well as a team and rapidly prototyped and integrated our codebase.

We learned new technologies such as NextJS and languages like TypeScript. We gained new problem-solving skills which were necessary for the development of the project. We learned to think bigger when asking others for help and consulted our network internationally when it was late in Ireland.

Future of Contex

We will be continuing with Contex in the future. There will be a better RAG system for the generator chatbot. We are building more of these AI-native components to further assist our models in generating better native components for developers. There will be more sound effects and fluid animations for our cursor and its actions.