rAIndeers

Inspiration

With rAIndeer, we want to offer a fun and sustainable alternative to boring Christmas cards. Our goal was to enable people to quickly and easily generate personalized, creative greetings that truly spark joy.

We believe, standard Christmas cards are

boring: Most cards we received in the last years were uninspired and failed to enhance the relationship between sender and receiver.
wasteful: one billion christmas cards end up in the bin each year! Most physical Christmas cards are not recyclable and end up in landfills (here).

On the other hand, creating great personalized cards today is

time-intensive & expensive: For example, people may have to spend hours creating the content themselves or pay others to do it for them (e.g. on fiverr)

We offer an alternative with AI: No more generic cards that get forgotten, no more hours of tedious card-making - just share your creations with friends and family online and bring a smile to their faces!

What it does

Our WebApp has a simple and easily navigable user interface that creates online Christmas greetings containing a personalized and funny Christmas poem, and AI enhanced pictures. The user can share the card with family & friends via a shareable link.

The WebApp works in the following steps, together taking <3 minutes for the user to complete:

Landing Page: the user learns about rAindeer and our card creation process
Personalize card: the user can (a) input information about the person they want to create a card for (e.g. interests, hobbies, connection to the person) and (b) upload images that they would like to include.

(BE: AI creates creative content for the card)

Edit: the user can select AI-generated content for their Christmas greetings. The user chooses:
- 1 of 3 different personalized Christmas poems (which can be adjusted manually as well)
- 1 AI created image from a selection

(BE: finalizes the card)

Review: the user reviews the card
Publish: the user can publish the card and share it via a link.

How we built it

The frontend is built with React/Typescript, the backend with python fastapi. Images are saved on a dedicated AWS S3 bucket and we use firebase NoSQL database to store user IDs, poems and image urls.
Poem generation
- We use user input about the card receiver to create prompts with are sent to GPT-3 to return 3-paragraph poems in different styles (e.g. “personal”, “ghetto”). We engineered prompts to make the suggested output as varied, positive, funny and consistent with the description of the person, as time allowed.
Image generation
- We used convolutional neural network (currently hosted by huggingface) to mask the faces on the input image.
- This mask is sent together with the image to a Stable Diffusion API (beta.dreamstudio.ai) to be used with the stable inpainting v1.0 model to keep the faces unchanged but change its surroundings
- We came up with 6 different prompts to send to Stable Diffusion, which add Christmas context (e.g. reindeer horns or christmas trees) to the picture in different styles (e.g. photorealistic, oil painting, pixel art).

Challenges we ran into

Prompt engineering: generating prompts that work based on varying user input (both text and different types of images) and create funny poems proved difficult and required more effort than expected. Currently, the output quality for pictures works best for selfies with one person.
Database & storage -set up: Setting up a database and file storage was unexpectedly painful: Firebase had internal issues in creating storage; Supabase had a non-existing documentation for python; and AWS RDS configuration is cumbersome in general.
Hosting & Time: We use a huggingface free model to segment the faces which can take up to 10s. For a production environment, we will need to host the model ourselves, but didn’t have time.

Accomplishments that we're proud of

Fully running web app built in 2 days with acceptable UX (w/o designer on board) which allows everyone to create a Christmas card without any additional explanation
We managed to get decently looking stable diffusion images without going through the pain (i.e. computation time/costs) of fine-tuning a model with dreambooth
Product we will use ourselves for all of our Christmas cards this year & saving the planet one christmas card at a time ;)

What we learned

Prompt engineering is difficult and we could’ve spent much more time on this alone
Image masking: If masked regions become too small, stable diffusion messes them up (even though it shouldn’t touch masked regions)
Team hacking: It’s great to have four people dedicated to build something in a small amount of time

What's next for rAIndeers

We will of course send more Christmas greetings this year than ever before (after improving the prompts even a bit more and optimizing for multiple-people shots as well)! Besides, we dream of a side hustle: an online personalized text & image webshop, not only for Christmas cards but any personalized messages from seasons greetings to best men speeches and more. Long term vision is to automatize as many services provided in Fiverr as possible - an AI driven Fiverr.