Ephemera

Inspiration

I wanted to make something that blends code and art in a way that feels alive. I spend so much time staring at static screens, so I wondered what it would look like if a digital canvas actively fought against being ignored. The idea was to create an interactive experience where your physical presence is the only thing keeping the art from rotting away into static and glitches.

What it does

Ephemera is an interactive app that takes your webcam feed, uploaded photos, or MP3 audio and constantly decays them in real time. If you stand completely still, your silhouette dissolves into eight different glitch styles like melting liquid, fire embers, or falling sand. The only way to restore the image is to physically move and sweep the decay away. For audio, the waveforms physically break down into pure static until you hover over them to bring the music back. It also uses AI body segmentation to automatically protect your outline from the surrounding chaos if you want to stand still.

How we built it

I built the core desktop application using Electron to keep it fast and native. The heavy lifting for the visuals was done using p5js, which handles the pixel rendering and the complex math for the eight different decay engines. To make the AI shield work, I integrated TensorFlow to run body segmentation models directly on the camera feed. For the MP3 mode, I used the Web Audio API to split the frequencies into four bands and extract their raw waveform data to draw custom oscilloscopes that distort alongside the music.

Challenges we ran into

Performance was a massive hurdle. Initially, doing distance calculations for every single pixel on a high resolution image caused huge lag spikes and dropped the framerate to a crawl. I had to completely rewrite the rendering engine to scale a tiny background canvas natively and only execute the math on the specific pixels that were actively glitching. Getting the audio API to properly split frequencies without blowing out the speakers was also a huge learning curve.

Accomplishments that we're proud of

I am really proud of the eight custom pixel shaders I wrote from scratch. Seeing the liquid style melt the pixels and the dust style blow them away based on actual math formulas was incredibly rewarding. For example, getting the motion distance calculated efficiently required abandoning heavy loops and relying on focused radius formulas like ( d = \sqrt{x^2 + y^2} ) only around the active cursor. I am also proud that I managed to get TensorFlow running smoothly alongside the renderer without melting my laptop.

What we learned

I learned so much about hardware acceleration, pixel array manipulation, and how to optimize JavaScript loops to run 60 frames per second. I also learned a ton about the Web Audio API and how to translate raw frequency data into visual movement.

What's next for Ephemera

I would love to port this into a web app so people can use it directly in their browser without downloading anything. I also want to add more interactive AI features, like hand tracking, so you can grab the glitching pixels and physically throw them across the screen.

Built With

• css
• electron
• html
• javascript
• node.js
• p5js
• tensorflow
• webaudio