Workvibe

Multi-Modal AI Agent for Contextual Content Entertainment

Comment

Inspiration

Work can be an exciting place sometimes, but also a bit of an annoying experience at times, where nobody really has the burning passion to start working. Those cringe messages on my team's group chat from Monday morning that include '👍' still haunt me, and resorting to getting stuck in a loop of irrelevant reels that get you hooked isn't any better.

I wanted a way to unify periods of entertainment with work culture and create an overall better work experience through intelligent, context-aware content delivery.

What it does

WorkVibe is a multi-modal AI agent that takes as input a selfie (for emotion analysis), a description of the current mood, and fetches Teams meeting calendar to recommend appropriate entertainment content through curated YouTube videos and AI-generated memes to entertain yourself and other coworkers.

The system analyzes 8 different facial emotions using AWS Rekognition, combines this with your calendar context, and performs vector similarity search on a database of embedded content. It gets better with every use, thanks to its like feature, you can target exactly what you like through ingesting more similar and relevant content using cosine similarity matching.

How I built it

Tech Stack & Architecture

  • Frontend: React.js with Vite build system for optimized performance
  • Backend: Node.js/Express.js with modular service architecture
  • Database: TiDB Serverless with native VECTOR(1536) support for similarity search
  • AI Services:
    • OpenAI GPT-4o-mini for context understanding
    • OpenAI text-embedding-3-small for 1536-dimensional vectors
    • AWS Rekognition for computer vision
  • APIs: Microsoft Graph (OAuth 2.0), YouTube Data v3, ImgFlip

Multi-Step Agentic Workflows

  • Ingest & Index: Videos with 1536-dimensional embeddings
  • Search & Match: Vector similarity search on TiDB
  • Chain LLM Calls: Context analysis and meme generation
  • Invoke Tools: AWS emotion detection, Teams calendar
  • Automated Flow: Input → Analysis → Search → Content delivery

Challenges I ran into

Vector not supported syntax in Node.js TiDB driver. The VECTOR(1536) column type threw SQL parsing errors. Had to manually create tables in TiDB console and comment out auto-migration code.

Creating an effective relevance pipeline . Videos weren't matching user context well initially. Solution was embedding YouTube comments along with title and description, capturing viewer sentiment and creating richer contextual understanding.

Multi-modal data synchronization. Coordinating real-time emotion analysis, calendar fetching, and LLM processing in a single workflow required careful orchestration and error handling across distributed services.

Accomplishments that I am proud of

I unironically find it effective. Building projects for hackathons has always been a mission of self-discovery and asking the question: "What do I want to build to make an impact in my life and in others'?"

But with WorkVibe, I truly feel like this app is more than a hackathon project. It's an app that I genuinely enjoy using and has a soft spot in my heart. The vector search accuracy is remarkable, and seeing the AI understand my work context to deliver perfectly timed content feels magical.

Successfully implemented true multi-modal AI that processes visual, textual, and temporal data in real-time, creating an agentic system that chains multiple AI services seamlessly.

What I learned

  • Multi-modal AI orchestration - synchronizing AWS Rekognition, OpenAI, and Microsoft Graph in real-time pipelines
  • Prompt engineering for consistent AI responses and preventing emoji generation in memes
  • Microservices design for scalable workflows with proper error boundaries
  • Embedding strategies - learned that including user comments dramatically improves content relevance

What's next for WorkVibe

Further improvements in its algorithm for video refresh and ingestion, including:

  • Reinforcement learning from user feedback to improve recommendation accuracy
  • Extended integrations with Slack, Discord, Zoom for broader workplace coverage
  • Team collaboration features for shared entertainment experiences and mood analytics
  • Mobile application with React Native for cross-device experience

TiDB Cloud Account:

eduard.jitareanu@stud.ubbcluj.ro

Built With

Share this project:

Updates