ScholarSynopsis

Turns a large number of questions into a smaller set of summaries, intended for lectures/seminars; works across a local network and allows students to submit questions. Embeddings -> Clusters -> LLM

Comment

Inspiration

In Supernova Era by Liu Cixin, the author imagines a system where an incredibly capable AI allows, for the first time, an entire country to speak at once. Each person gives their opinion, and the AI automatically categorizes their sentiment and groups them into clusters, which are then each "represented" by a single person (read the book, its awesome!). Our app attempts to replicate this distilling for questions; because often for large lectures/seminars, it is difficult to have your individual questions answered.

What it does

The website allows students on a local network to submit questions. The teacher/admin can toggle their ability to share questions, as well as clearing the question database. Most importantly, they can query a model to group and distill questions, leaving them with clusters of related questions and a summary of each cluster.

How we built it

The model backend is defined in three steps: List of questions -> Clusters of questions -> Summaries of clusters.

  • The first is gathered from the SQL database.
  • The second is facilitated by taking sentence embeddings of each question, before using a clustering algorithm to generate clusters.
  • The third operates using an LLM; the clusters are fed into a model, with appropriate context, and the summary is displayed for the teacher.

The website backend defines functions for;

  • Submitting a question to the SQL database
  • Emptying the SQL database
  • Toggling question submission
  • Generating clusters and summaries

The website frontend is modestly styled with css to look somewhat okay.

Challenges and Accomplishments

  1. Because none of our team have ever used Flask or Jinja (or HTML), it was a difficult process (80% of the allotted time) getting the website up and running. As it is, features like security are still very bad (anyone can access the admin panel). Regardless, we are all proud of the effort we put in for this project; and we had a fully functioning site by the end, both the frontend and backend!
  2. Our backend summary model turned out to be highly effective, and its question groupings were great at a glance. Also, it worked with the cheapest model on perpexity.ai, making it really affordable.
  3. We got stuck for a few hours figuring out css, but it turned out we had just done "styles.css" instead of "style.css" :)

What we learned

  • Web development is hard
  • Managing a project is hard
  • LLMs are incredible; embeddings are incredible; AI is just very effective.
  • ChatGPT is pretty good at code :)

What's next for ScholarSynopsis

We still need to add:

  • Better UI / making it more intuitive
  • Adding security (sign-in, encryption, etc.) (there is no security right now)
  • Bringing it out of development: deployment, making it easier to host, etc.
  • Making the github better for developers and users <3 (README, wiki, etc.)

Because of its "proof of concept" nature, we'd probably try to write the final product from scratch (we can keep the model though).

Thanks Y'all For Reading! <3

Built With

Share this project:

Updates