Better Backgrounds

Inspiration

That blurry background may hide your messy room, but it also makes you look like you're in a different dimension.

What it does

Better Backgrounds is an exploration into the latest and greatest techniques to improve composited webcam footage in any way possible. It takes things even further with traversable Gaussian Splats and physically simulated Depth of Field.

How we built it

Using GPT 5.6 + Codex to crunch dozens of research papers, assess feasibility, run benchmarks, and carry out the implementation.

Written in Python + QT app container hybrid with JS frontend.

Challenges we ran into

Multiple road blocks and pivots including:

Phone based room scanning (not feasible): quality of produced gaussian splats was not sufficient, process was too convoluted for a user.
Tackling FPS performance issues: a lot of implementations were dropped due to performance, required a lot of trial and error.
Lack of licensing: Codex was used to identify research projects that were appropriately licensed. There were a lot of projects that could not be used because of a lack of one.

Accomplishments that we're proud of

It actually works and is a significant improvement over the daily webcam with a few extra features to boot. The only caveat is that it requires more powerful hardware acceleration than usual (CUDA/Apple Silicon).

What we learned

I have a new found appreciation for the complexity of working with webcam footage and the difficulty of the problems being solved. I also now have a rough understanding of about 20+ research articles on a topic I was previous unfamiliar with. All thanks to Codex digesting and explaining everything to me.