Had a great time building Omni and diving into the LiveKit and Overshoot SDKs at NexHacks.
LiveKit forms the backbone of our unified passive perception and 3D digtal twin reconstruction system; We send the poses and images from iOS ARKit to the GPU server continuously in real time via LiveKit byte stream and both load them into Depth-Anything3, as well as send the images to Overshoot for real time perception of the user's interactions with the world.
Take a look at our iOS integration of the LiveKit SDK in the main branch, and the NodeJS SDK integration in branch overshoot. You can find our usage of Depth-Anything3 for point cloud creation in dap3_web on the main branch; Image files received through LiveKit can easily be fed into the da3_streaming and our dap3.py scripts.
We loved how simple LiveKit makes it to connect devices that are not on the same network, and how easy Overshoot makes it to do VLM inference with huge models like Qwen3VL 30B with sub-second inference times, and no infrastructure provisioning. We are excited about the possibilities for vision on wearable devices.
Log in or sign up for Devpost to join the conversation.