That face on Mars but on a black plane in a blue sky world of IC....,.

What it does and How to Use It is the first Unity OpenCV with dlib face tracking and AI metaverse character app on the IC blockchain. We show the Dlib 68 feature point .dat ML model converted into a retro 8 bit look using Imgproc as a stylized retro face outline on a black ground plane in a blue sky world. This is a fully 3D environment - use WASD keys to move around and your mouse to rotate your view.

Connect with your metaverse NPCs. You can also click on any of the AI wandering characters to see from his or her perspective, on the top viewfinder window. opens blue skies for anyone using Unity OpenCV to build the future of HCI on the IC blockchain.

Simply grab the Unity webgl project from github and deploy it! Or check it out live on IC on the link below (please let it load, tested on a gigabit network).

How we built it

  • dfx dfinity
  • OpenCVForUnity (latest)
  • Dlib (latest)
  • Some Unity C#
  • Unity 2020.3.27f1 WebGL

Challenges we ran into

  • Apologies for the rough demo... AFAIK, this is the only entry that has actual AR / computer vision working on IC. Others seem to be pointing to external AR viewers.

  • Creating a new canister for StreamingAssets and then streaming the classic (but largeish) dlib 68 feature point .dat file into Unity WebGL

  • differences between local and main net

  • adding in the subdomain to load the larger dlib ML model file

  • It turns out that Unity 2021 triggers some ICP incompatibilities, so reverting to Unity 2020 fixed things

  • ... all while recovering from covid!

What's next for

Very excited to deploy WebGL AR and computer vision apps on IC, and this serves as a jumping point.

I've built many award winning apps using OpenCV dlib - for example, the makeup and AR try-on cross-platform WebGL and iOS/Android app faceStylr, which won TechCrunch Disrupt Grand Prize 2018 - which could all run on IC!

Built With

Share this project: