about the project

inspiration

this project was inspired by the growing need for businesses to have smarter, more personalized interactions with their customers without spending endless hours on repetitive tasks. i wanted to combine the power of ai with modern voice-based communication to create a platform that could make business-customer interactions more seamless, efficient, and personal.

what i learned

this project taught me a lot about integrating different technologies and apis. i learned how to use twilio for voice-based communication and explored its potential in handling outgoing calls. i also got hands-on experience with openai's realtime api for speech to speech and used gpt-4o-mini for real-time insights like sentiment analysis, live transcripts, and summaries. additionally, i realized the importance of choosing the right tools and technologies early on—i initially started with next.js api routes but ended up switching to a server.ts file at the root for simplicity after struggling with fastify and websockets from a tutorial. that completely negates next, i know, but i was struggling bro.

how i built it

i started with next.js for the frontend and used prisma to manage a postgres database. the platform lets businesses input a ton of information about themselves, including employee names, roles, and availability, which the ai can reference naturally during conversations. businesses can set up custom conversation flows by defining greetings, topics, questions to ask, and how to wrap up the call. all info (business info, customer info, call logs) are stored in the database

once customers are added with their name and phone number (and verified through a twilio code), businesses can schedule calls in batches, and the ai will automatically call customers one by one.

the ai is equipped with real-time monitoring features. during calls, you can view live transcripts, sentiment analysis, and summaries. if a customer requests to speak to a human, the platform raises a bright red flag on the dashboard. these features were powered by gpt-4o-mini for analysis and summarization.

framer motion was used sparingly to add some animations on the home page for buttons—i wanted to do more but ran out of time.

challenges

one major challenge was figuring out how to integrate twilio with ai in a seamless way. i initially wanted to handle incoming calls, but it proved to be far more complex, so i pivoted to outgoing calls instead. time was another major constraint, and i wasn’t able to make the entire site as polished as i envisioned. additionally, debugging websockets with fastify slowed me down early on, which forced me to simplify the backend architecture. also the ai is super sensitive to background noise so im kinda scared for the live demo lol.

overall, this project was a mix of challenges and rewards, and i’m proud of how much i was able to accomplish in a short time. i'll prolly update the readme with step by step instructions on running it later - once i get some sleep.

Built With

• fastify
• framer-motion
• gpt
• next
• postgresql
• prisma
• twilio