Spark

Landing Page
Previous Search Dashboard
Upload/Record Video
Submission Page
Results Page
Analysis Page

Inspiration

We are strong believers in the idea that true mastery of a concept comes with the ability to effectively communicate one's understanding of the subject. Thus, the goal of our project is to assess an individual's understanding through both content and confidence analysis.

What it does

Our mission is to help individuals build confidence and clarity in their communication by providing real-time analysis of their speech. Using advanced AI, our platform evaluates facial expressions, voice patterns, and speech content to assess a speaker’s confidence when explaining a topic. By transcribing speech, verifying correctness with GPT-4, and generating a confidence score, we offer users valuable feedback to refine their delivery. With secure login and progress tracking, we enable users to monitor their growth over time, fostering continuous improvement and mastery in their ability to articulate ideas with confidence.

How we built it

We leverage advanced libraries such as OpenCV, MediaPipe, DeepFace, OpenAI, Whisper, and Librosa, among others, to perform advanced analysis of video and audio input, assessing the client's understanding of a subject by evaluating their confidence, through audio and visual analysis, along with content evaluation through OpenAI's Language Learning Models. We used the Flask backend framework, React and Vite for our front end, and Postgres for our database.

Challenges we ran into

A big obstacle that we encountered early on was setting up our environments, a deceivingly difficult task. In addition, we had difficulty identifying the appropriate Python libraries to convert the blob files sent using an API to our backend, a issue we eventually resolved by choosing to utilize the ffmpeg library.

What's next for Spark

We spent more of our time developing the actual software rather than applying the appropriate models to identify our evaluation metrics. Further steps we could choose to pursue with Spark is more appropriate metric calculations, real-time sentiment and confidence overlay, implementation of gamification elements to improve learning.