Kora

Kora, an app that continuously records your browser so you can access any memory in seconds.

Comment

Inspiration

Introducing Kora, the AI screen recorder that makes it impossible to forget. Have you ever been deep in work, only to forget how you did something just moments later? Maybe you saw an important chart, wrote an insightful line of code, or followed a workflow you couldn’t quite replicate later. We wanted to build a “second brain” for your digital life — a tool that remembers everything you see, and lets you instantly return to the exact moment it happened.

What it does

Kora records your screen and makes it possible to find any object, text, or element in the recording just by describing it. The user simply types a prompt like “graph about CO₂ emissions” or “error message from Python”, and Kora instantly returns the exact video frame containing that object. From there, users can scroll forward or backward through the clip to recall context.

Users can start and stop recording using intuitive on-screen overlay buttons, and type in prompts directly into a small floating input field. Once entered, Kora automatically searches the ongoing recording to pinpoint and replay the relevant moments.

How we built it

We used Python's CV library to record the screen and split the recording into 900 frames of 30 seconds(30fps). We stored the spliced 30-second clips in Firebase and each frame in the vector database Pinecone. The prompt is fed into a vector-embedding model which takes the text input and translates it into video frames that contain said input. The frames are pulled from Pinecone and the 30 second clip containing the frame is also pulled and returned to the user. Finally, the Gemini API spits out a description of video frame in the case that the user had forgotten the context.

Challenges we ran into

Our original idea was to use the Snap AR spectacles, however our relative inexperience with the software combined with the wifi struggles made it near impossible to work with the snapchat representatives at the venue and do productive work. Schedule and logistical conflicts made us pivot the night before the hackathon was due, and the software that we developed for SnapAR was reworked to fit the new idea. Kora still came with its own challenges, like optimizing the record processing to reduce memory usage and the best minimalist UI for the overlay. However, we were able to power through these issues that felt miniscule compared to the SnapAR debacle.

Accomplishments that we're proud of

We were able to launch and deploy a fully functioning app despite our original idea falling through almost 2/3 through the hackathon. We hope to keep continuing to build upon this idea after the hackathon and improve it to its highest standard.

What we learned

With all the problems that arose during the challenge, we quickly learned to adapt to adversity by thinking on the fly. Our willingness to embrace the change rather than shun it led us to a successful new idea rather than a dud. We also learned that hard work wouldn't go to waste in a very literal sense. The vector embedding model that we built for SnapAR got retooled for Kora.

What's next for Untitled

The next steps for Kora are focused on expanding its intelligence, interactivity, and scalability. We plan to integrate a chat interface directly with the returned frames, allowing users to have natural conversations with Kora about their digital memories. For example, users could ask “What was I doing before this frame?” or “Summarize everything that happened in this session.”

Built With

Share this project:

Updates