Precious Pixels

Inspiration

Precious Pixels is an Image Retrieval-Augmented Generation (RAG) system that combines the capabilities of text prompts, image retrieval (finding relevant images from a database based on a query) and generative models (creating or synthesizing content based on input data) for real world applications. It is capable of transforming a user's vision into realistic product designs. It is a product leveraging AI to generate personalized jewelry and craft designs based on user preferences, such as rings and necklaces.

What it does

• Interactive Search Functionality: Users begin by searching for products, receiving a list of items to choose from.
• User Feedback Integration: Opportunity for users to select liked items and provide input on desired product aspects.
• Iterative Selection Process: Repeated refinement process until the user identifies a product they like.
• Final Product Details: Delivery of an estimated price and manufacturing instructions upon final product selection.
• Enhanced Search with Personal Uploads: Users can upload their own files for more specific searches, saved and utilized in the vector database.

How we built it

Precious Pixels leverages multi-modal embeddings to store images in a vector database, utilizing AstraDB for efficient data handling. This approach allows for the rapid retrieval of images that match specific queries, enabling users to find relevant visuals with ease. Once a set of images has been identified, the project offers the unique capability to select several of these images and combine them with additional custom requests. This process results in the creation of new, composite images that are tailored to the users' needs. By integrating advanced search capabilities with creative image synthesis, Precious Pixels provides a versatile platform for generating highly customized visual content.

Challenges we ran into

• Stable diffusion is not so stable
• Lack of support for a batch of images to image generation model, had to rely on image to text and text to image gen

Accomplishments that we're proud of

1. End-to-end working Image RAG system
2. Successfully implemented SigLIP as our multi-modal embedder, enabling direct embedding from images, which significantly streamlined our data processing pipeline.
3. Explored various image generation models, including Stable Diffusion, DALLE-3, and Midjourney, allowing us to compare and select the best option in terms of quality and speed.

What we learned

• Allow sufficient time for fine tuning the models
• New tools/model we explored : DataStax-AstraDB, SigLIP, BLIP, GPT-4 Vision Preview

What's next for Precious Pixels

E-commerce customization

Leveraging the innovative Precious Pixel, a key application is its integration with e-commerce giants like Etsy or Amazon, transforming how we shop online. When shoppers can't find the right item, Precious Pixel steps in, using advanced AI to generate a realistic image of the desired product. This capability paves the way for a new e-commerce model, centered around artists and manufacturers crafting these AI-conceptualized items. It's a platform that marries bespoke, personalized shopping with a collaborative space where creators actualize user-generated designs. This melding of AI and artisanal skill not only broadens the scope of custom-made products but also creates a unique market where users don't just search for products, they bring them into existence.

Photos and Memories

The Image Retrieval-Augmented Generation (RAG) system is designed to transform everyday photos into automated, thematic storyboards, creating a unique storytelling experience. This system intelligently selects and sequences users' photos into coherent narratives, adding contextually relevant captions. It will produce 'Automated Reels' - dynamic presentations of personal memories, tailored to individual experiences and preferences. The technology organizes photos not just by time, but also by themes, capturing significant events, places, and emotions. This results in a personalized recap of one's life, highlighting moments of joy and adventure in a vivid, engaging format. The Image RAG system redefines photo storage, offering emotive storytelling and an effortless way to relieve cherished memories.

Built With

• astradb
• bentoml
• gpt-4
• javascript
• openai
• python
• react
• sdxl