Inspiration
Due to the turmoil and tension within Israel and Palestine, we were inspired to fight misinformation by building a tool that allows a user to visualize the distribution of viewpoints among the general public, and listen to summaries and videos from the people involved. The goal is to increase empathy to those experiencing the conflict.
What it does
Our project scrapes Youtube and Reddit for videos and comments around the conflict, and then runs an algorithm similar to T-SNE to visualize them. We built a tool that can scan over a set of viewpoints and generate an audio summary of it. We also did this for the videos, allowing the user to view videos with a range of opinions ranging from pro/anti violence to pro/anti Israel.
How we built it
We used 11labs to generate audio summaries, as well as OpenAI's API to clean up and interpret the reddit comments to generate intelligent embeddings. We used 12labs to index and embed and videos we scraped.
Challenges we ran into
One challenge we ran into is that the embeddings for the videos and comments were so different that it didn't make sense to display them on the same plot.
Accomplishments that we're proud of
We are proud of the tool which generates the audio summaries across a section of viewpoints!
What we learned
We learned about how to interpret multimodal AI tools to create a useful interpretation of otherwise disparate data.
What's next for Israel Palestine Video Understanding
The next steps would be to gather more data and provide some user-validation for the credibility of the videos and comments involved. Additionally, we would like to built a nice frontend to make the project accessible to those curious about the conflict.
Log in or sign up for Devpost to join the conversation.