-
-
2
-
1
Inspiration ##
The iPhones were revolutionary not because they directly changed how phones worked; rather, they innovated upon how we interacted with the screen itself with the more intuitive gesture of swiping. We aim to bring a similar change to the interaction with AI agents where we innovate upon the medium in which a user interacts with an AI in order to make it more efficient, save time, and primarily bridge the gap in understanding that is so often experienced by users. This gap in understanding is not only caused by the potential lack of understanding of coding concepts but also the output of long texts, which in general gets skipped by users in the output or to the last few lines.
How we built it
Our idea for this project was to help make communicating with any AI agent of your choice more natural and efficient. We did this by focusing on 2 central ideas which would be synced and work in parallel to create a seamless experience: 1, using speech to communicate with the AI agent rather than text in order to create a more conversational and natural environment, and 2, creating a UI centered around widgets that can be called upon dynamically by the agent you are interacting with. The widgets would be called from a vast library as needed based on the context of the conversation, such as notes, checklists, images, graphs, code blocks, and more, in order to show information in a more digestible manner and allow the agent to control the focus of the user through positioning and re-sizing of these widgets as needed.
The end goal would be an open-source framework that can be connected to any agent of your choice, such as OpenClaw or Gemini, and this agent would be able to pull from a library of widgets that the open-source developers have contributed to in order to present, discuss, and teach information to you in a more intuitive and natural manner.
When working on this project, we found that the best way to make progress was to tackle one issue at a time rather than trying to solve as many as possible in one go. We further optimized this process via Claude by prompting it to tier the issues we were facing in level of priority using its understanding of the codebase as well as user priority context.
This priority system in conjunction with our vibe coding method of, 1) Speak into ChatGPT and use its transcription to transcribe all of our thoughts 2) Paste that transcribed text into Claude and ask it to - Simplify out thoughts and remove filler - Reformat the wording of our text to be more structured and direct - Create a prompt using the reformatted text specifically optimized to get the most out of Claude Code 3) Run the code and test functionalities 4) Identify areas of improvement and add to priority list in Claude 5) Repeat cycle
Challenges we ran into
Due to the dynamic nature of the project, we had many issues from the widget UI to the core logistics of how the agent processed information.
1) Syncing the output of the agent with its voice was an issue of the highest priority, as seamlessness was key for the agent to effectively communicate the requested prompt in accordance with the widgets 2) The agent would often get caught in a loop cycle where after presenting a code block, answering a user request, or presenting recursion 3) The system would have a delay when starting the process but then return to minimal delay 4) The system would struggle with framing the code or text properly 5) The system was hyper sensitive to noise detection and would often get over stimulated and disconnect
All of these problems were ultimately solved through our 5 step routine and Claude optimized problem solving process.
Accomplishments that we're proud of
1) We are super proud of coming up with a truly original and most importantly an ambitious idea. 2) We are proud of overcoming numerous issues on this project. Not only were the issues complicated but they were also repetitive. This would have led most to quit or maybe even pivot to a different idea; however, we did not let it stop us. Rather, it was a learning experience from which we not only solved the problem but also gained valuable experience that we can apply elsewhere. 3) We were proud of creating a tangible prototype, which may not have done everything we wanted it to, but helped us convey the potential of our project accurately to the judges.
What we learned
- We learned that it helps to be surgical while debugging using Claude; it makes it easier to debug if we explain the problem and make it add all the possible causes to a list and later make it tackle only one of them at a time.
- We also learned to use a few very helpful tools like Clade Code, CLI interfaces, Vertex, Gemini, etc.
- Using Claude to create custom prompts designed for Claude Code was super powerful.
What's next for Synaps
The possibilities are truly endless. The beauty of our project is our plan to make it an open-source framework. This means that you can use any agent to integrate with the framework and call upon an endless potential of widgets, which can be developed by open-source developers or even OpenClaw itself. This has the potential to change how we interact with AI on a fundamental level, saving time and bridging gaps that were difficult communicate before.
