AegisVision: AI Doorbell Video Intelligence

Home Page
Video in Queue page
Full Video Analysis Page
Analyzed Video Page
Analysis Starting
Deeply Analysis
RAG for User page
Seprate RAG for each Videos
Daily Summary in offline condition page
Report and Event Intelligence Dashboard

Inspiration

Security cameras generate hours of video, but finding the important moments often requires manually reviewing entire recordings. Identifying package deliveries, unusual activity, visitor interactions, and potential security risks can be time-consuming. We wanted to make security footage easier to understand by turning raw video into searchable, timestamped information that people can investigate through natural-language questions.

This inspired us to build WorkMate AI: Intelligent Video Security Assistant, an AI-powered system that helps users understand video events instead of simply watching recordings from beginning to end.

What it does

WorkMate AI transforms security videos into structured event information, summaries, and searchable evidence.

Key capabilities include:

AI-Powered Video Analysis: Extracts relevant visual information and identifies observable activities, objects, and event categories from video frames.
Detailed Event Extraction: Generates descriptions of events, their sequence, relevant timestamps, and supporting evidence.
Natural-Language Question Answering: Allows users to ask questions about analyzed clips and retrieve answers grounded in stored analysis records.
Searchable Security Events: Helps users find relevant clips using event descriptions and concepts, rather than relying only on filenames.
Interactive Doorbell Feed: Presents video clips in a simulated front-door camera experience, with event notifications and playback-based navigation.
Daily Event Summary: Organizes stored records into a readable overview, including event-type counts, high-risk clips, and a chronological timeline.
Risk-Oriented Insights: Helps users prioritize events based on available risk classifications and their supporting evidence.

The goal is to make security footage more accessible, searchable, and useful for reviewing important events.

How we built it

We developed WorkMate AI using a web application architecture that connects a user interface, video-processing pipeline, AI analysis, and structured event records.

The workflow begins with a video clip or an existing dataset record. The system extracts selected frames, processes relevant visual information, and converts the results into structured event descriptions and timestamped records. These records power the event feed, search experience, daily summaries, and question-answering interface.

We designed the analysis workflow to reuse existing results wherever possible instead of repeatedly processing the same footage. For longer videos, the intended approach is iterative frame sampling, with additional frames selected when more detail is needed. This helps balance useful event coverage with processing cost.

The application also uses retrieval-based question answering so users can ask questions about recorded events and receive answers based on relevant stored information. The interface brings these capabilities together through clip playback, event details, search, summaries, and evidence references.

The implementation uses the project's existing web stack and configured AI/video-processing integrations. External services and provider fallbacks are used only where supported by the actual configuration.

Challenges we ran into

One of our main challenges was converting video frames into detailed, temporally grounded event descriptions rather than broad categories. A useful security assistant needs to explain what was observed, when it happened, and what evidence supports the result.

Another challenge was processing longer videos efficiently. Extracting too many frames increases processing cost, while extracting too few can miss important transitions. We therefore focused on iterative sampling, targeted refinement, reuse of completed results, and avoiding unnecessary repeated analysis.

We also encountered challenges with retrieval and response generation. Correct citations alone are not sufficient if the answer does not address the user's question. The system needs to connect the actual question to relevant records and generate a context-specific response.

Finally, maintaining consistent event records across the video feed, search, summaries, and AI assistant required careful handling of identifiers, timestamps, loading states, and stored analysis results.

Accomplishments that we're proud of

We brought together video analysis, structured event information, searchable descriptions, conversational investigation, and a simulated doorbell feed into one security-focused experience.

We are particularly proud of the focus on turning video into information that users can investigate. Instead of requiring users to repeatedly watch entire clips, the system is designed to help them locate relevant moments, review event evidence, and understand recorded activity through natural-language questions.

The project also emphasizes reuse of stored analysis results, timestamp-aware event navigation, and risk-oriented summaries. These design choices help make the experience more practical than a video viewer that only displays generated labels.

What we learned

We learned that useful AI video analysis requires more than identifying objects or assigning categories. Temporal context, evidence quality, uncertainty, and reliable retrieval are essential to producing trustworthy answers.

We also learned the importance of separating deterministic data processing from generative AI. Event counts, chronological ordering, and summaries of stored records can often be produced directly from structured data, while AI is better suited to interpreting relevant evidence and answering natural-language questions.

Finally, we gained experience integrating frontend interactions with video-processing workflows, managing analysis results, designing efficient frame-sampling strategies, and handling failures across multiple system components.

What's next for WorkMate AI: Intelligent Video Security Assistant

Our next steps are to improve temporal accuracy, expand support for authorized video sources, and make long-video analysis more efficient.

We plan to strengthen event-boundary detection, refine frame selection around ambiguous activities, improve timestamp-based retrieval, and expand on-demand visual evidence such as detection boxes when supported by the processing pipeline.

We also aim to improve the reliability of conversational answers, add more flexible filtering and reporting, and evaluate the system against a larger range of video scenarios using measurable quality metrics.

Our long-term vision is to make security footage easier to search, understand, and investigate while keeping results grounded in the available visual evidence.