Journal Vision

A video search tool so you can search for the clip you're looking for in seconds!

Comment

Inspiration

It's impossible to search through so many videos - on my camera roll, desktop etc .

  • Organizations, journalists, market researchers gather TONS of video data with no way to efficiently search through them (no CTRL+F :”( )
  • Let’s not waste time on finding the right video and timestamp for hours…
  • let’s go to it right away - served in seconds!

What it does

  • Created for individuals and organizations with large video data needs
  • Download the app, upload your gallery, and never worry about searching through your data ever again!
  • Linked to your personal or workspace account
  • Search all your videos with just one simply query
  • Journal Vision plays it - at the best matched video and timestamp
  • Backend processes audio and visual data (transcripts, video labelling, etc.)
  • Cosine similarities between the prompt and pre-processed data (saved in database) returns the best match video and timestamp

How I built it

React (TypeScript) frontend Flask (Python) backend Google cloud! Speech-to-text API for transcription Video Intelligence API for video labelling Scikit learn library + pandas used for data handling, vectorization, and running cosine similarities Auth0 library for user authentication

Challenges I ran into

  • Matching data formats and processing across different sources
  • Video player was tricky to set up and play as the source is served from the backend
  • Processing data appropriately without errors to extract transcript + labels from google cloud APIs and combine them

What's next for Journal Vision

  • Add semantic search for better matching the context of prompts to videos
  • Easy to upload video gallery to Google Cloud and display it
  • Google cloud APIs already in use - switch from local saving to cloud
  • I wanted to add more user account features, and make it collaborative based on the user accounts
  • For example, adding annotations to video segments that you can see for everyone in your workspace
  • Need to improve the UI
  • Delete and manual play features
  • Context summary for video clips
  • Serve on web
Share this project:

Updates