Every team I’ve worked with — whether in college projects, internships, or real companies — struggled with the same issue: information was everywhere and nowhere at the same time. Searching through Drive, Slack messages, email chains, Jira tickets, PDFs, screenshots, and old meeting notes became a daily battle. The bigger the team, the worse it got.
This is what inspired EKOS, a unified, multimodal knowledge engine that makes every piece of organizational knowledge instantly searchable and actionable.
What inspired us
We realized that teams spend a huge amount of time just looking for things: documents buried in long email threads, screenshots saved in random folders, Jira tickets scattered across sprints, and meeting recordings nobody revisits.
At the same time, new AI models like Gemini Multimodal made it possible to understand not only text — but also images, screenshots, PDFs, tables, and audio in a single flow. That sparked the idea: What if we could build a “Google for the entire company”? Something that understands everything — not just keywords.
How we built it
We designed EKOS as a full stack project built around:
React + Vite + shadcn/ui for a fast, clean UI
FastAPI (Python) for backend services
Gemini APIs for text + vision + audio based reasoning
Vector DB (Pinecone) for semantic search
n8n workflows for connectors like Drive/Gmail/Slack/Jira
OCR & STT to process PDFs, screenshots, and meeting recordings
A Knowledge Graph layer to connect people, documents, decisions, and tickets
RAG pipeline to synthesize answers with citations
We started by building ingestion pipelines, then chunking and embedding documents, and finally a clean UI where users could:
search with natural language,
upload screenshots to find related tickets,
view timelines of decisions,
and generate summaries or action items using AI.
What we learned
This project taught us a lot about:
designing scalable ingestion systems
vector search and RAG evaluation
working with multimodal AI (images + text + audio)
building clean UI layouts that feel “enterprise-ready”
connecting multiple third-party services under a single identity
handling permissions, RBAC, and data governance
creating meaningful AI outputs with structured prompting
We also learned how difficult it is to orchestrate multiple modalities and multiple sources of truth — and how much AI can help streamline this.
Challenges we faced
Integrating many different APIs (Drive, Gmail, Slack, Jira) consistently
Making multimodal search feel fast and intuitive
Handling different content formats like PDFs, spreadsheets, screenshots, and audio
Designing a RAG pipeline that returns accurate answers with citations
Keeping the UI simple despite the complexity behind the scenes
Learning how to build a knowledge graph that updates continuously
Managing embedding costs and optimizing caching
Ensuring security, RBAC, and isolation between tenants
Despite these challenges, building EKOS has been extremely rewarding. It showed us how AI can transform workplace productivity and how multimodal search can unlock knowledge that was previously lost in silos.
Built With
- docker
- fastapi
- gemini-multimodal-apis
- github
- google-ai-studio
- n8n-workflows
- ocr/stt-pipeline
- pinecone-vector-db
- python
- react-(vite)
- redis-workers
- shadcn/ui
- tailwindcss
- typescript
