Campaign Alchemist

• 

  Generate short english post with english audio

• 

  Translate the short english post to spanish and get generated spanish audio through single workflow

• 

  Long english, spanish, chinese, arabic post generated through single ClarifAI workflow

• 

  Long english, spanish, chinese, arabic post generated through single ClarifAI workflow

• 

  Long english, spanish, chinese, arabic post generated through single ClarifAI workflow

• 

  Workflow to transalte in spanish and generate spanish audio. Other languages also have same workflow:- translation followed by audio

• 

  ClarifAI Workflow to generate long english post and then generate spanish, chinese and arabic translated versions of the long post

• 

  Cool QR codes generated by the 'cool and quirky' QR code generator

• 

  Qr code for https://campaign-alchemist.streamlit.app/

• 

  We use text moderation models to intervene in case someone enters a description which violates policy. We keep a very high bar of acceptance

Inspiration

• With the increasing importance of online presence and marketing, there is a growing demand for high-quality content that captures the attention of audiences. I wanted to create an app that simplifies and enhances content creation, making it easier to market products, features, and companies.

• The ability to reach a larger global audience, using short audio clips in their language and textual content in their specific languages, can help firms cross global boundaries and market in multiple markets simultaneously.

  What it does

• Generate short posts with audio for platforms which are based on short posts based marketing and audios generated can be used for audio based marketing platforms.

• Generate long, detailed posts for platforms which have support for longer posts, blogs.

• Generate a aesthetic custom image embedded QR codes to share product links, feature links, marketing posts

How we built it

• Used ClarifAI workflows to generate short english posts and audios.
• Used ClarifAI workflows to translate short english posts to spanish, chinese and arabic and generated their audios using ClarifAI speech synthesis model integrated in the workflow itself.
• Used ClarifAI workflow to generate long english, arabic, chinese, spanish post in a single flow
• Used Modal labs to deploy a Stable Diffusion(controlnet) based Image embedded QR code generator.

Challenges we ran into

• Speech Synthesis model does not support text longer than a certain characters, truncated short posts keeping them till the last complete sentence possible.
• Faced challenges while coding and deploying controlnet based QR code generator.
• ClarifAI workflows timeout a lot if the number of step increases, tried to seperate workflows but the frequent timeouts still remain a problem, have tried to handle the timeout errors so that they don't get displayed. Can try to retry workflows but they already seem to take a decent amount of time.

Accomplishments that we're proud of

• Was able to deploy QR code generator on modal labs and understood a lot of the recent advancements in the Diffusion models and how ControlNet helps drive them.
• Was able to learn about ClarifAI workflows and able to create multimodal workflows using ClarifAI to streamline multiple processes.

What we learned

• Learned about Diffusion models and controlnets and how to use them to control image generation
• Learned about ClarifAI workflows and multilingual speech synthesis models and their limitations

What's next for Campaign Alchemist

• Will add support in future to support lip syncing given a video and an audio to generate multilingual videos from a single english video.
• Will add product placement support to generate multiple product placement images from a single product image.

Built With

• clarifai
• modal
• streamlit