Inspiration
Video is the biggest pile of data nobody can query. BI teams know SQL, not computer vision. We wanted video in, SQL out.
What it does
Video to SQL (vid2sql) runs an OpenCV 5 perception agent over video and writes segments, moving objects, events and per-minute counts into relational tables. Ask a plain question or run read-only SQL. Every row links to the second of footage it came from.
How we built it
OpenCV 5 does the vision. An agent loop reads its results and decides what happens next: accept the segment, retry with different settings, or flag it for human review. Those decisions are stored as rows too, so they can be audited with SQL. The AWS part is S3, a container Lambda on arm64 and an HTTP API, defined in deploy/template.yaml.
Challenges we ran into
Keeping answers honest. A question the tables cannot answer is refused, not guessed, and the only SQL allowed is read-only.
Accomplishments that we're proud of
Three public Creative Commons clips processed end to end, with every agent decision recorded and auditable, and a live endpoint judges can query.
What we learned
It counts moving objects, not people, and track counts are inflated. There is no labelled ground truth, so we claim no accuracy figure. The technical report lists these limits in full.
What's next for Video to SQL
Vision and agent rules are separate, so new models can be added as they arrive and thresholds re-tuned from measured runs as the footage grows.
Built With
- amazon-web-services
- aws-lambda
- opencv
- python
- sqlite
Log in or sign up for Devpost to join the conversation.