[Ogólna][PiZZA]Muninn

Inspiration

Nordic mitology (Elder Edda)
The very hackathon we're participating in
Uncountable hours spent "watching" online lectures, while not being actively engaged in any of them, therefore not retaining much.
Statistical data about low attention span during online meeting, with them still being the preferred method of communication with large groups.

What it does

Muninn captures presenter's audio, and automatically generates questions meant to test's the listeners engagement in the presentation. It then provides the presenter with a summary, indirectly giving feedback on the quality of their presentation.

How we built it

Using Svelte with Skeleton, we created a webapp for both the presenter and the listeners. Then, after capturing the former's voice we feed it through carefully calibrated instance of Symbl.ai, and generate questions based on the transcript using Mixtral-8x7B. The resulting question is then presented to the listeners, and the answer statistics are displayed to the presenter.

Challenges we ran into

Adequate timing and transcription quality
Polish support
Validity of generated questions
The adequacy of generated questions
Lack of good models pre-trained specially for this purpose

Accomplishments that we're proud of

A live, working demo
The fine-tuning done between all AI-assisted parts of our projects

What we learned

That trying to use AI models reliably is a nightmare
How to minimalize uncertainty when using generative pre-trained transformers as a service
How to correctly partition speech data for prompting purposes.