Inspiration
Our inspiration for this project is Jarvis and the VisionQuest TV show, as both got our team thinking about AI agents and their functionalities as personal assistants.
What it does
Jarvis is a personal assistant that listens to you and responds appropriately based off the user's request. It can have a conversation with the user, generate text files/excel sheets/code, store events into your calendar, tell you the weather/current news, and take down notes. It can also research and find sources on various topics, providing the user with the sources found and relays the information gathered.
How we built it
We built Jarvis using Swarms API, Elevenlabs Text-to-speech/Speech-to-text API, OpenAI, and Firecrawl API. It was through these various APIs that we were able to set up the agents/functions that make Jarvis work, followed by giving it the voice it has to speak to the user.
Challenges we ran into
A very notable challenge we ran into was the swarms agents getting caught up on key phrases, misinterpreting what functions were being called/needed by Jarvis. An example of this was having it so Jarvis would confusingly try to make an excel sheet on data it hadn't yet researched through, simply because it jumped the researching part of a prompt and skipped to the excel sheet generation.
Accomplishments that we're proud of
:)
What we learned
We got to learn about Elevenlabs and the features they give to users, like its TTS/STT services.
What's next for Jarvis
Taking feedback on what future additions Jarvis could take on, while also improving on it's efficiency/speed.
Built With
- css
- elevenlabs
- firecrawl
- javascript
- json
- jsx
- node.js
- openai
- react
- swarm
- vite
