video-synopsis

print screen demo

Inspiration

Think about a long video clip, from a security camera. Most of the time nothing happens in the video. Now you are interested to see all the people that pass by the camera and the time they did. Instead of going over hours of video, you get a few seconds where all people showed in the same couple of frames.

What it does

The app takes a video and process it with multi object tracking network on a hailo8 chip. cutting all the frames where no persons were detected. Than cutting each person and putting it with its original time in to a set of combined frames in to a summery video.

How we built it

we used an online long video, and a video that we took in the office. run the multi object tracking network. we cut-off all the frames that no objects (persons) were detected. saved in a new dataset all the relevant info regarding each object: for each object we saved all the frames it was detected in, there original time, and box (coordinates for the object inside the frame). than we crop the object in to the same coordinates from the first frame. result was a short video includes frames with all the objects combined in the original video with the original time they appear.

Challenges we ran into

finding all the objects' frames and sorting it. cropping the object from the original frame in the video and putting it from the starting frame. multi object tracking net had a wrong threshold, not only persons were detected some of the time.

Accomplishments that we're proud of

we were able to detect only persons, by changing the network detection threshold, we were able to detect all the same persons frames and cropping them in to the same frame. we were able to make the synopsis video including all objects together.

What we learned

how to work and process the frames that came back from the hailo8 chip. how to use the detections ability in to a quick app. how to work with data frame and opencv.