What it does
An agent that attends work meetings on your behalf. If the meeting requires your participation or is valuable for you to join, you will be notified and given a short brief with visuals that describes what was discussed thus far. This allows you to jump into the meeting, or get a detailed summary in your inbox after the meeting ends. The intended outcome is to reduce meeting fatigue and improve time management for professionals.
How we built it
The frontend joins the meeting as an agent and transmits data to a GPU on the backend with business logic to transcribe audio and run a vision model (LLaVa) on the zoom meeting visual inputs (presentation and attendee images).
Challenges we ran into
Getting access to Zoom meetings programmatically is difficult. Real time multimodal inference also required tinkering.
Built With
- javascript
- openai
- python
Log in or sign up for Devpost to join the conversation.