Doom Scrolling

Doom Scrolling is a re-shader for DOOM (1993) which replaces every pixel with looping TikTok videos. Players navigate hell by literally “scrolling” through content.

Comment

Doom Scrolling has two components. Firstly it is a shader for DOOM (1993) where we have replaced every pixel with looping short-form videos. Secondly, the game itself is controlled using the TikTok interface remotely.

The shader

Demo timestamps:

At its core, the shader simply maps each pixel colour to a TikTok given that the average colour of the TikTok matches the pixel. However, a standard pixel in sRGB colour space has 16,777,216 different colour values - that's a lot of TikTok's we need. Instead, we apply a few tricks in order to reduce the number of colours we need to represent.

Firstly, DOOM (1993) uses a 16x16 colour palette which means we only actually need to represent 256 colours:

DOOM (1993) colour palette

Thus when we take the screen capture, we reduce its colour space to the DOOM colour palette. This works especially well for DOOM, and decently well for generic content whilst keeping the number of colours needed low.

Now we just need to find TikTok's that match each of these colours. If we were to randomly sample a large amount of TikTok's we could then find the best TikTok for each colour in our palette, but this is rather inefficient. Instead we use targeted queries to bias our search around the colours we want.

The queries were made up of all permutations of the following:

  • Tag => Theme of the video. Typically a generic object as we want the video to be mostly static to prevent too much flicker in the final render.
  • Colour => The colour we want.
  • Modifier => Simple modifier to get more variation.

The tags, colours and modifiers we used are as follows:

tags = ["aesthetic", "theme", "art", "room", "sign", "nature", "photography", "vibe"]
colours = ["black", "red", "brown", "white", "grey", "green", "blue", "orange", "pink", "yellow"]
modifiers = ["light", "dark", ""]

An example query would then be "light red art".

With a collection of TikTok's, we then matched each TikTok to a given colour on DOOM's colour palette by choosing the TikTok with the highest cosine similarity to the target colour. Since the frame being rendered will often be changing, we opted to only analyse the first 1-5 frames of each TikTok instead of the entire video as on average, each TikTok will only be visible for a short amount of time. This cut down on the processing time dramatically.

From this we could then generate a colour palette made from TikTok's:

DOOM (1993) colour palette

However, TikTok's are more than just single images, they are videos. This means that in order to play each TikTok we need multiple palette's to represent each video frame:

Full Texture used to render TikTok's

The above texture is a 6x6 grid of colour palettes which gives us 36 frames of each TikTok we can show. We utilize 5 of these textures to get a total of 180 frames. Since we play each TikTok at 30fps, this gives us 6 seconds of footage.

We additionally generate multiple of these texture sets to increase the variety of TikTok's we can show for each colour, using a white noise texture to indicate which pixel should use which texture set. These variants can also be swapped between using a scroll animation to give the illusion of scrolling TikToks.

The shader runs in real time (75fps) on webcam captured footage on a laptop RTX 3060. We use OBS virtual webcam to capture the screen.

TikTok Controller

In addition to the shader, we recreated the TikTok interface in the browser which could be used to control the game DOOM (1993). This was a website hosted from a python webserver which converted typical motions such as scrolling, liking and sharing to keyboard and mouse inputs. For example, scrolling up or down would move the player forwards or backwards.

TikTok Interface

Share this project:

Updates