OpenBoard

Inspiration

Designers often spend more time searching for assets than actually creating them. Whether it's trying to identify a specific font used in a flattened image or hunting through a massive brand toolkit for a specific icon, the workflow is often interrupted. We wanted to build a "bridge" that allows designers to use their most natural input—a quick sketch or a hand-drawn character—to instantly retrieve the right professional assets.

What it does

OpenBoard is an Adobe Express add-on that bridges the gap between a designer’s imagination and their brand assets. It offers:

Font Identification: Identify font families within your brand toolkit by simply drawing or uploading a character, powered by Vision Transformers (ViT).

Multi-Modal Asset Search: Find the perfect brand image using a sketch + text query. You can draw a rough shape and add a text prompt like "but make it more vintage" to refine the search results instantly.

Intelligent Region Generation: Mark specific areas on your canvas for the Nano Banana API to automatically generate and fill with high-fidelity imagery that matches your project’s context.

Doodle-to-Asset Matching: Uses rough sketches to find exact or near-match images from a pre-uploaded local brand library.

How we built it

The project leverages a modern, cost-efficient AI stack:

Backend: A Python FastAPI server that manages model inference and communication.

Embeddings & Search: We utilized CLIP (Contrastive Language-Image Pre-training) to handle the multi-modal queries. By projecting both the user's sketch and their text prompt into the same latent space, we can perform a joint search against the brand toolkit's embeddings.

Typography Analysis: Vision Transformers were implemented to capture high-resolution stroke data, ensuring that the font search remains accurate even with rough hand-drawn inputs.

Cost Efficiency: By running these models locally, we eliminate the per-query API costs associated with image and font searching, making it a sustainable tool for independent designers.

Challenges we ran into

The biggest hurdle was optimization. Running Vision Transformers and CLIP models locally while maintaining the snappy performance expected in a design environment required significant model quantization and efficient memory management. Additionally, mapping a "rough" user doodle to a "polished" brand asset required fine-tuning the similarity thresholds to ensure relevant results without being too restrictive.

Accomplishments that we're proud of

Zero-Cost Search: Successfully implementing local models means designers can search their libraries thousands of times without incurring a single penny in API fees.

Typography Precision: The ViT-based font search is remarkably accurate, even with stylized or slightly distorted hand-drawn characters.

Seamless Integration: Creating a fluid UI within Adobe Express that feels like a native part of the design workspace.

What we learned

We gained deep insights into Vector Embeddings and how different models "see" images. We learned that while CLIP is great for general concepts, fine-grained tasks like font matching benefit significantly from the attention mechanisms in Vision Transformers. We also learned the importance of "Human-in-the-loop" design—giving the user the ability to mark regions specifically rather than relying on full-canvas automation.

What's next for OpenBoard

Since the multi-modal search is already live, our roadmap now focuses on deepening the integration between AI and professional design workflows:

Vectorization Pipeline: Allowing users to not just find an asset, but automatically convert their rough sketches into clean, brand-compliant SVG vectors.

Style-Guided Layouts: Implementing a feature that suggests layouts based on the visual weight and hierarchy of the assets found via search.

Collaborative Brand Hubs: Syncing local models across a team so that every designer’s "OpenBoard" stays updated with the latest brand kit changes in real-time.