CallScout

Inspiration

The inspiration for CallScout came from the desire to make financial earnings calls more accessible and understandable for everyone, not just financial analysts. We wanted to create a tool that could cut through the jargon and provide real-time, actionable insights, essentially giving every listener a personal financial expert to guide them through the call.

What it does

CallScout is a web application that provides AI-powered insights for live and past earnings calls. Users can select an earnings call from a list, and the application will play the audio of the call while displaying a synchronized transcript. As the call progresses, CallScout uses OpenAI's o4-mini model to analyze each segment of the transcript and generate a concise, expert-level insight if it contains material information. The insights are displayed alongside the transcript, allowing users to quickly grasp the key takeaways from the call without having to listen to the entire thing.

How we built it

CallScout is a full-stack application built with Next.js and deployed on Vercel.

• Frontend: The frontend is built with React, TypeScript, and shadcn/ui components. It uses HLS.js to handle the streaming of audio and framer-motion for smooth animations. The UI is designed to be a "single pane of glass" where users can see the transcript, listen to the audio, and read the AI-generated insights all in one place.
• Backend: The backend is a Next.js API route that serves as a serverless function. It takes a segment of the transcript as input, along with the conversation history for context, and uses the OpenAI API with the o4-mini model to generate insights. We engineered a detailed prompt to ensure the AI provides high-quality, relevant analysis, focusing on six key areas: Company Performance, Strategic Vision, Execution Risk, Macro Impact, Competitive Landscape, and Sentiment & Framing.
• Data: The application currently uses a set of mock data for earnings calls, with links to audio and transcript files hosted on files.quartr.com.

Challenges we ran into

• Real-time Transcript Synchronization: One of the biggest challenges was synchronizing the audio playback with the transcript highlighting in real-time. We had to carefully manage the state of the audio player and the transcript display, and use useEffect and useCallback hooks extensively to keep everything in sync.
• Prompt Engineering: Getting the AI to produce high-quality insights was an iterative process. We spent a significant amount of time on prompt engineering to guide the o4-mini model to act as a financial analyst and provide concise, relevant, and non-repetitive commentary.
• UI/UX for Streaming Data: Designing a user interface that could handle a constant stream of new information (the transcript and the insights) without overwhelming the user was a challenge. We implemented features like "scroll-to-live" to allow users to catch up with the live transcript if they scroll away.

Accomplishments that we're proud of

• The Core "Aha!" Moment: We are very proud of the core functionality of the application. Seeing the AI-generated insights pop up in real-time next to the transcript is a "wow" moment that validates our initial vision.
• The AI Integration: Successfully integrating the OpenAI API and engineering a prompt that delivers high-quality financial insights is a major accomplishment.
• The Polished UI/UX: We are proud of the clean and intuitive user interface. We put a lot of effort into making the application easy to use and visually appealing, using modern tools like shadcn/ui and framer-motion.

What we learned

• Power of LLMs for Domain-Specific Tasks: This project was a great learning experience in harnessing the power of Large Language Models (LLMs) for a specific domain. We learned that with careful prompt engineering, we can guide the model to perform complex tasks like financial analysis.
• Importance of a Solid Frontend Architecture: Building a real-time application like CallScout highlighted the importance of a well-structured frontend. Using React hooks effectively and managing state carefully was crucial for the success of the project.
• Iterative Development: The git history shows our iterative development process. We started with a basic feature set and gradually added more functionality and polish. This approach allowed us to build a robust application in a short amount of time.

What's next for CallScout

• Real-time Data Integration: The next logical step is to move away from mock data and integrate with a real-time data source for earnings calls. This would involve finding an API that provides live audio streams and transcripts.
• User Accounts and Personalization: We want to add user accounts to allow users to save their favorite companies, track upcoming calls, and perhaps even get notifications.
• Expanding the AI's Capabilities: We could expand the AI's capabilities to provide more in-depth analysis, such as summarizing the entire call, identifying key themes, or even answering user questions about the call.
• Broader Coverage: We plan to expand the coverage to include more companies and different types of financial events, such as investor days and product announcements.

Built With

• llm
• nextjs
• openai
• vercel