TeachAIr

TeachAIr Logo

Inspiration

As sophomore CS students, we've noticed how one of stony brook's required and popular courses (CSE307) taught by Professor Kane offers a slew of dense and verbose slides with lots of redundant information.

What it does

TeachAIr is an all-encompassing AI Tutor with both high level teaching and reviewing capabilities. For teaching, TeachAIr will take an input of a slideshow on a subject and will compile a video of an AI speaker going through the slideshow and explaining the subjects in a professional manner. For reviewing, TeachAIr is also capable of taking any assignment, being an instructions file and a submission file, and the AI will review the assignment, annotate it for issues, and generate a slideshow video of TeachAIr once again discussing the topics, just in the context of an assignment analysis.

How we built it

We used NeuralSeek to build this using Agentic AI systems.

We started by designing the AI speaker: it takes an arbitrary slideshow, transcribes all the words into a text document, and utilized an LLM to create a professor-esc script off of the transcription, split by slide for later convenience. For the AI voice, we used Elevenlabs API to do text-to-speech, utilizing looped calls per slide to avoid overload. Finally, since NeuralSeek doesn't have video capabilities, we ported the results to a python script that utilized MoviePy to play the presentation with the voice.

For the AI reviewer, we first defined a method to input both an instruction and submission. The instruction file defines what the LLM analyzes in the submission. The LLM is then tasked to make an annotated document file where it highlights issues in red, explains the reason it is an issue in blue, and describes a potential solution in green. This file is saved and then the errors are used to make a txt file specifically formatted for later conversion to a powerpoint. From here, an LLM takes information from the txt file and turns it into a json format which is used to create the powerpoint slides particularly describing the issues, explanations, and solutions. Finally, the AI Speaker scheme is run on this new slideshow with an adjusted parameter (from lecturer to analyzer) and a new slideshow video with an AI voiceover is created for better explanations.

Challenges we ran into

Frontend was a nightmare with NeuralSeek. While NeuralSeek is great at porting output variables, it's unfortunately terrible at porting output files -- it simply does not have the capability. We could not get Lovable to access the output files without manually downloading and sending them. We attempted troubleshooting with Leon, Lawrence, and the CEO of NeuralSeek himself, and nothing was successful. This was a limitation that we had no way of predicting when we began the project, so we simply had to compile the results separately and create a Lovable website just to keep everything together.

Accomplishments that we're proud of

In a short amount of time, we were able to adapt to a new method for building agentic AI systems and successfully created a nearly flawless backend that could take various inputs for helping and teaching students. NeuralSeek's workflow was a steep learning code for cs students like ourselves who are supremely familiar with programming as it is traditionally known.

What we learned

We learned to function under the highest current abstraction of AI -- the early stages of Agentic AI. Shoutout Lawrence.

What's next for TeachAIr

We have a lot of ideas, with the first being a fully integrated frontend that can really bring our vision to life. After some refinements, we'll be looking to primarily revamp two features:

First, we're interested in having the generated powerpoint more visually interesting and various overall to make learning more exciting.

Second, we want to have the videos include certain active elements such as captions and circling important regions if possible.