Problem Statement
AI has become extremely powerful and useful for programming and data mining. However, current LLMs and AI agents are still not very effective in financial decision-making. In the financial system, humans continue to play a critical role, especially in areas such as investment decisions, risk assessment, and strategic judgment.
The challenge is not simply to automate financial decisions, but to help humans better understand complex problems before making those decisions.
Solution
My automated decision-making agent aims to support better human decision-making by simulating real-world discussion and reasoning processes. Instead of relying on a single AI response, the system breaks down a complex problem into multiple perspectives, with each perspective represented by a different agent.
These agents discuss, challenge, and debate the issue. Each agent provides a distinct viewpoint to the user who proposed the problem, allowing the user to see the issue from multiple angles.
The goal of this project is not to replace humans with AI agents. Instead, it is designed to help humans understand complex problems more clearly before making important decisions. By presenting multiple perspectives, arguments, and counterarguments, the system allows users to evaluate an issue more deeply and make more informed choices.
Each agent can perform web searches and access dedicated memory for storing relevant information, previous opinions, and insights from other agents. This allows the system to provide up-to-date analysis while maintaining context throughout the discussion.
AI Streaming Platform
To make the agent discussion more engaging and accessible, I also built a streaming platform for AI agents using FFmpeg. The idea is similar to Twitch, but instead of human streamers, the streamers are AI agents.
Agent discussion is one key use case, but the platform can also support live news, debates, analysis shows, educational content, and many other formats. The entire agent conversation can be produced as a podcast-style video or live stream, where users can watch the discussion in real time, raise questions, challenge an agent’s opinion, or participate in the debate.
Through this interactive process, users can gain a clearer and more complete understanding of the problem.
AI Director and Production Workflow
I designed a director agent to simulate a real TV production workflow using AI. The director coordinates multiple generation models during the live stream.
It uses Lyria, Google’s music generation model, to generate background music, and Gemini image generation to create real-time visuals based on the current speaker’s content. The director agent then switches visuals at the right moment, making the viewing experience feel more natural, dynamic, and closer to a real broadcast production.
Technical Challenges
One major challenge is subtitle synchronization. Real-time subtitles are important for a podcast-style experience, but they are difficult to align accurately with the generated audio.
Another challenge is generation latency. Music, images, and text all take time to generate, which can easily interrupt the flow of the stream.
To solve this, I allow agents to generate their content while other agents are speaking. I also use pre-generation whenever possible, so upcoming audio, visuals, and discussion points are prepared before they are needed.
This creates a fast first response and nearly zero waiting time between speakers, making the entire experience feel much more streamlined and natural.
Result
The final system combines multi-agent reasoning, real-time debate, web search, memory, AI-generated media, and live streaming into one interactive platform. It helps users explore complex topics through structured discussion while preserving the human role in final decision-making.
Log in or sign up for Devpost to join the conversation.