Promosaic: Remix anything, anytime.

"good artists copy, great artists steal". Using Meta's SAM2 and PyTesseract, we built an editor that makes it effortless to transform “designs in the wild" into reusable creative material.

Comment

Inspiration

In any creative process, tangible cultural artifacts inspire and guide iteration. Furthermore, this heritage of shared representation strongly influences the digital sphere through skeuomorphic representations. We see the problem posed by the hackathon -- and the Living Culture Track in particular -- is how we can enable new forms of creative expression, at both practical and artistic levels. Our answer is Promosaic, which takes advantage of state-of-the-art vision models to produce structured representations of graphic objects.

What it does

Promosaic turns everyday spaces into creative playgrounds. Our app allows you to take in your surroundings—whether street art, flyers, signs—as raw materials for visual expression. Rather than passively consuming imagery, tools like Promosaic enable the community to be curators, designers, and artists of the visual culture we hope to inhabit. Stickers, murals, posters are no longer static in their callouts and associations; instead, they are remixable building blocks. With our responsive and mobile-compatible web application, we take the best parts of Tumblr and Arena and reinvent curation for a highly mobile generation.

We take a different approach of graphics editing than existing forms of media creation, through using structured object representations. Under the hood, the app uses state-of-the-art techniques to produce well-formed and flexible structures. Promosaic uses Segment Anything 2 -- a large vision transformer model -- to detect and isolate objects, while we use opencv's edge detection and the Tesseract library for grouping. The result is a model of images with meaningful subunits, including individual characters, lines of text, visual icons, and subcompositions. Each object becomes an independent layer on a canvas: you can move, rotate, reorder, or swap backgrounds. Once you’ve composed something you like, you can export the full design as a polished PDF, or download each mask separately for use in other projects.

How we built it

We structured the app with a Flask backend and a Svelte frontend. The segmentation is powered by Meta’s AI model, and the canvas interface is crafted using vanilla Svelte plus the Fabric library, which handles layering, transformations, and rendering on canvas. The backend uses opencv2 kernel and edge detection, the Segment Anything Model 2, and PyTesseract OCR.

Challenges we ran into

One early obstacle was selecting a segmentation model that could robustly separate objects, then overlay those masks precisely and extract the underlying image data. Especially tricky was grouping: for example, treating text as one object or breaking it into letters on demand. The segmentation output lacked this hierarchy, making it hard to subdivide further. Building the canvas had its own difficulties—handling user interactions, layer stacking, edge cases, and smooth redraws proved nontrivial.

Accomplishments that we're proud of

Despite the tight timeline, we got a working prototype that can segment, recompose, and export. Testing Meta’s new segmentation AI in a creative tool context has been super exciting especially because capabilities like this simply weren’t accessible a few years ago, so we feel super privileged to be able to work with them.

By bridging machine learning, computer vision, and design, our tool empowers anyone to engage with their cultural environment as both a consumer and creator. Flyers on a street corner or stickers on a laptop no longer just advertise—they become living artifacts, remixable and generative in shaping cultural expression. With tools like Promosaic, we hope to usher in healthier and happier relationships at the intersection of creativity and media.

What we learned

We were genuinely surprised by how alive our tool felt once the collage engine clicked into place. As we played, we discovered that remixing visual fragments grows more addictive as you unlock more assets—much like how language expands when your vocabulary widens.

Another insight we arrived at is that building creative tools is more than coding features, and can be highly creative on its own: As you contextualize applications with data from your own surroundings (All kinds of pleasing visual errata an early morning walk through Cambridge, and then a long run across Allston and MIT), you start to notice how you see, remix, and express living culture. Ultimately we would have loved to do tests with other users, and we hope to extend our creation into an easily accessible and shareable form.

What's next for Promosaic

We’re excited to expand into typographic tools—automatically detecting fonts in an image, allowing users to swap them, and adjusting kerning, weight, and style. Since typography has such a strong emotional and aesthetic impact, giving users control over it will make their designs even more expressive.

We also plan to integrate a diffusion‑based inpainting model for background completion. This will let users preserve the original atmosphere of a scene while filling in blank gaps more seamlessly. The result: remixing becomes not just about objects, but about mood, texture, and composition.

Built With

Share this project:

Updates