Inspiration

We were inspired by this opportunity to use highly technical tools related to backend ML models and frontend development in such a visibly impactful and meaningful way.

What it does

Our video intelligence dashboard uses open-source AI models ran locally to detect and categorize faces, then performs activity and sentiment analysis which are used to streamline a human-centered review process for locating impactful moments.

How we built it

We used the ADAFACE, X-Clip, and Hugging Face ViT models for face detection, activity classification, and sentiment analysis respectively. For the frontend, we used React, TypeScript, and FastAPI with a SQLite database.

Challenges we ran into

Determining how to process the frames in each video
Feasibility and time constraints for some of the AI features
Having to adjust our plan after examining the types of videos in the dataset and balancing this with project expectations ## Accomplishments that we're proud of
3 functioning ML models
User-friendly dashboard
Accessible data-ingestion display ## What we learned
Balancing data privacy and ethical AI use concerns with product functionality
Effective delegation of tasks from a large project into manageable chunks for each team member ## What's next for TPA Video Intelligence Platform Some future steps include:
More nuanced engagement scores that consider external factors such as body language, peer context, and activity type
Deeper temporal analysis with footage from throughout the multiple-week camps
Investigating which activities had the highest impact on different groups of students