Orzille

MADDANNON

Project Description

Video conferencing artificial intelligence utilities, integrated in your favorite web-communication application, including the following features:

Cartoon-style transfer
Deep-fake-like face swapper
Background face anonymizer (only background faces are anonymized)

Technical details

this section uses the face-swap feature as an example, but any other face-manipulation can be applied to the app

Orzille input: User's "wannabe" image Orzille output: Webcam video's feed cartoonized, with user's face swapped by it's wannabe's one.

Architecture includes 4 main video-processing components:

Face detection NN (on Hailo8)
Landmarks extraction NN (on Hailo8)
Face Swap (on Host)
Style Transfer NN (on Hailo8) Those are combined to a one App-flow with the Tappas platform.

Orzille implementation: The 3 NNs are allocated together on the Hailo8 device. Orzille holds 2 inter-process queues on host: One holds the Face detection outputs, and the other holds the Landmarks outputs On each frame Orzille works in a "snake" flow:

Preprocess puts the feed-frame into the Face detection input
Preprocess gets the popped-box from the Landmarks queue, applies the Face-Swap on it puts it in the Style Transfer inputs
Postprocess gets the output from the Face detection, computes NMS and crops the resulted box, and puts it in the Face detection outputs
Postprocess returns the Style Transfer output

Integration with Teams Orzille allows the user to choose the Hailo Virtual Camera on Teams settings (or any other application!, such Zoom, etc). Hailo Virtual Camera uses the Camera feed, applies the full process flow, and outputs the results back to Teams.

Usefulness

Cartoon-style transfer - Entertainment
Deep-fake-like face swapper Entertainment
Background face anonymizer Aids keeping family and friends anonymous in a home environment, where video communication is integrated into daily life (WFH, studying from home)

User adoption

Users benefit real-time entertainment and enhanced privacy with a zero-cost integration to user's daily routines. The applications are useful when video chatting in multiple environments, whether it be work, school or socializing over the internet. Now more than ever, it is a necessity.

Industry

Orzille project can be used in mobile devices, laptops, video conferencing camera, and other multitude of consumer products. Of course it will be also used as the coolest Hailo Demo, and as an example of a complex multi-step application for customers to be used as a reference.