Chronomind AI

Inspiration

I've had the idea for a while now to create a fully localized AI secretary that can plan out my daily tasks and classes, because I didn't like the idea of sending crucial data online to an LLM. The security risks from scraping for data everywhere for training LLMs, potential breaches, among other possible issues always put me off from sending any pivotal information to a chatbot. I always generalized any prompts or scenarios that I gave to Gemini, the chatbot I use regularly, but that always meant that the replies lacked a sort of personal touch. Having a chatbot to which I could give every bit of information about myself, letting it plan and organize so much more effectively to my preferences while keeping the data locally on the device was the dream for this project in the long term. In this hackathon, the project I built using AWS and CockroachDB will act as a stepping stone in my bigger goal of building a purely offline secretary.

What it does

The project features 3 elements: a checklist, a calendar, and a chat page. The checklist is quite self explanatory, it simply is a to do list where you can add items and tick them off when you are done. The cool part is when the checklist works together with the calendar. When selecting a particular day, the calendar only pulls up the tasks you have scheduled that day, and it lets you add tasks to be done on that day. The chat page is the other key feature. While the initial goal was to allow a conversation to lead to an agent recording the task for the user, that seemed quite out of my league when I came to the stage of building the feature. However, the chat feature is still quite useful. Through an engineered prompt that provides a hybrid chat history–containing recent messages and messages relevant to the current prompt–and a list of upcoming tasks, the LLM from AWS Bedrock provides a response considering travel restrictions and how much time it might take for completing other tasks before you finalize a meeting or task on a given day and provides alternatives if it doesn't seem feasible.

How it was built

Having a strong background in Python, I knew I wanted to build the backend logic in it. I've worked with the Streamlit library before but I knew it had quite a lot of design restrictions, and I knew I wanted to try something new. Then I discovered the Reflex framework, that let me program in Python but ultimately compiled into a React app. Seeing how I could explore a new framework from a confines of something I'm more experienced in, I decided to build the app purely in Python. Connecting to AWS and CockroachDB was quite straightforward, using API keys and Python libraries like 'psycopg2' and 'boto3' to establish a link and use the services for this project.

Challenges I ran into

This project had quite a lot of challenges, but it was fun to learn to leverage Google Gemini that helped me analyze the situation, uncover the issue behind many lines of code, and put me on the path to fixing it. There were two big challenges that I had to tackle during the development of this project: 1 - At first, I wanted to a fully visual calendar and the day that the user clicks is what is shown in the checklist, so I tried using the reflex-calendar framework, but I wasn't able to get the day that was selected to be sent from the calendar, so all the tasks added were only added to the current day. Ultimately, I ended up having to go with a typical dropdown input calendar, but it got me the desired functionality. 2 - My initial idea was to include a way for the user to talk to the AI, and I was planning to use Amazon Transcribe, but that feature wasn't available in the free tier, and I didn't realize that for quite a while. When I got to implementing this feature, and found out it wasn't possible, I had to make a quick pivot. Then after brainstorming for a while, I simply reversed the feature; instead of converting the user's voice to text for a model, I could convert the model's text to a voice. So I ended up using AWS Polly to give the LLM a voice. These challenges forced me to make changes to my project in ways I wouldn't have done before, and that was a crucial learning that I'll carry with me for future endeavors.

Biggest accomplishments

The biggest accomplishment by far is actually getting the entire project to work. The breakthroughs when I got the calendar to cooperate with the checklist and when I got the voice audio recording to actually play when I pressed the button had me jumping up and down after the hours I spent before trying to debug them. In the end, I'm very proud to have gotten this project working, and it will be a great foundation to work from for the next stages of this project.

What I learned

The biggest learnings I have from this project are in learning to apply new AI skills. From brainstorming to troubleshooting, crafting precise prompts and getting the right kind of help was crucial in keeping this developer workflow moving throughout the project. After completing this project, I'm also more prepared to tackle long stretches of debugging without desired results and pivot when an aspect of the project isn't possible given the constraints. All these learnings will prove useful as I take this project to it's next stage.

What's next for Chronomind AI

The ultimate goal for Chronomind AI is to build a digitally secure and offline secretary. The internet shouldn't come into the picture for anything other than specific tasks like researching or emailing. Privatization of personal data is crucial in the age where AI can find anything anywhere if it's on the internet, and this project had provided the foundations for me to take the next step in this long term project.

Built With

• amazon-web-services
• bedrock
• claude
• cockroachdb
• gemini
• polly
• python
• reflex