Team94_9I_Contentify.ai

Inspiration

We found that many youths today dream of becoming content creators, whether it be on TikTok, Youtube or Instagram. However, we found 2 major problems:

1. The barriers to garnering attention as an amateur content creator are high due to the unique skills involved, such as creating unique music and video content
2. Even for professional content creators, it takes much work and time to create truly unique content that will allow them to stand on the same level as established content creators.

Thus, how do we allow everyone to become the content creator they wish to be?

What it does

Contentify.ai is an all-in-one digital platform powered by a robust AI engine developed specifically for social media content creation. The user journey comprises 3 steps:

1. Users may interact with Storiation, a chatbot powered by OpenAI and LlamaIndex. Storiation inspires users by aiding in the journey to create a unique story for their content, from initial conception to the final storyline. Storiation even curates a list of 3 theme-appropriate songs to accompany the storyline.
2. Our flagship AI music cover generator, Melodiation, allows the creation of any music cover (with the previously curated songs or anything of your own choice) with your own voice based on your own streaming content! Powered by Replicant, AI-generated voice models and covers can be created in a matter of minutes.
3. To bring their visions to life, content creators can create their own unique posts using Visionation where both the covers and the captions will be created using OpenAI models, ChatGPT and Dall-E.

How we built it

1. Storiation: Engaging conversations are powered by OpenAI's ChatGPT-4 models and we used a RAG (Retrieval Augmented Generation)-based approach using a vector store powered by LlamaIndex.
2. AI Music Covers: Using Replicate to generate RVC (Retrieval-based Voice Conversion) models, we devised a pipeline to deploy these models into custom training to create unique music covers tailored to one's own voice.
3. Full Post Generation: Employing a multimodal approach, we used ChatGPT-4 and DALL-E 3's models provided by OpenAI to generate a full post, from the post cover to the caption with simple text prompts.
4. User Interface: Our UI is powered by React.js and the backend and its seamless functionalities were created using FastAPI.

Challenges we ran into

1. We had difficulty integrating our proprietary RVC model pipeline into our backend server.

Accomplishments that we're proud of

1. High Level of Personaliation: We managed to achieve a high level of customisation by creating unique custom chatbots powered by OpenAI. We also managed to develop fully personalised music covers based on voice models created with one's own vocals. Hence, creating a fully personalised web solution is something our team is deeply proud of.
2. Integration of Advanced Features to build a full AI Engine: Despite the difficulty of creating a full Replicate-powered pipeline to generate custom RVC models, we managed to successfully integrate this into our web app. This was done in conjunction with other key features such as our story creation assistant and post generator assistant.

What we learned

We learnt how to use less common but complex APIs such as Replicate to create our AI engine.

What's next for Contentify.ai

We hope to deploy and optimise our AI cover solution fully such that it can be supported by various OSs and different types of GPUs as it can only be currently run on Nvidia GPU due to the high processing power required. We also hope to further improve our engine's features to make it more user-friendly and optimised to rival that of state-of-the-art AI engines.

Built With

• chatgpt
• dall-e
• fastapi
• llamaindex
• openai
• react
• replicant
• streamlit