ProfAI

• 

  On the fly transcription with Deepgram. Answer directly references timestamp and answer.

• 

  Professor answers accurately and directly cites himself.

Inspiration

Have you ever felt lost watching a lecture? Wish you could just pause and ask the professor a question? Meet ProfAI, your professor's virtual clone who answers any question you have, instantly!

What it does

ProfAI is your personal AI teaching assistant that watches lectures with you. The moment you're confused, just ask your question out loud. It answers instantly in the voice and style of your professor, using context from the lecture and supplementing it with its own knowledge. Think of it like having 24/7 office hours with any professor, seamlessly built into your study flow.

How we built it

• Fish Audio for text to speech with voice cloning
• Speech Recognition for detecting user speech and intent
• DeepGram for on the fly, timestamped, video transcription
• SwiftUI frontend with new Liquid Glass design
• FastAPI backend with custom websocket bridge and communication protocol

Challenges we ran into

• Unreliable wifi
• Application integration
• Latency to obtain the professor's answers and convert it to audio
• Unforeseen circumstances

Accomplishments that we're proud of

• Frictionless integration to student's existing study flow
• Works on all video players (e.g. Youtube, Canvas, QuickPlayer)
• Accurate deepfake of the professor's voice
• SwiftUI to Python bridge

What we learned

• Using websockets to reduce latency in both our interactions with APIs and with our own services
• Designing with SwiftUI's innovative Liquid Glass

What's next for ProfAI

• Automatic context switching between multiple courses you are taking

Built With

• deepgram
• fastapi
• fishaudio
• swiftui