JumBuddy

There’s a gap between CS instructors and students’ struggles. JumBuddy analyzes raw IDE activity—files edited and edit patterns—to generate metrics that help professors evaluate teaching effectiveness

Comment

Track: Education

Inspiration

When when taking the mainline CS courses at Tufts (CS11, CS15, CS40), we noticed one key thing about our classmates; the students with the most issues were the worst at communicating their issues.

What it does

There’s a gap between CS instructors and fully understanding students’ struggles. JumBuddy goes to the source, raw student IDE input, and analyzes the files and functions edited and the frequency and fluidity of said edits in order to quantify and create metrics so professors can evaluate the effectiveness of their teaching methods.

How we built it

We designed a VSCode extension that collected keystroke edit history in local memory. For each discrete period of edits, defined as a "debounce period," (contiguous edits within a 5-second period), we calculated granular metrics about insertion and deletion behavior, and a full diff of the file in which those edits occurred. Those metrics and that diff are sent to our backend, where we pre-process and ingest re-formed snapshots of the file throughout its edit history into an LLM for more qualitative insights. Then, we provide this data to our web interface for professors and assistants to easily, at a glance, analyze project hotspots and see the specific thrash and churn values for each individual function name and file. The professor can then chat with an LLM to delve deeper into a specific student's file and full edit history in a timeline fashion from initial pull to current state.

Challenges we ran into

One major issue was communicating with our server from the VSCode client via HTTPS due to errors with how VSCode handles proxies and Supabase's built-in authentication system. Ultimately, we configured a callback-style redirect from within VSCode to our web interface to authenticate students via UTLN and password before passing along that API token to VSCode, solving our issue.

Accomplishments that we're proud of

For a blast from the past, we chose to implement CharLinkedList to see if JumBuddy can see where we struggled. In fact, it worked pretty well, and actually detected that we were struggling with the Destructor!

We're super proud that we got a VSCode extension working and that we're able to see precisely replayed data across multiple students when logged in as a professor.

What we learned

We learned that prioritizing design is greatly important. We spent the first 6 hours brainstorming and working out what we wanted out product to be, and we think that paid off massively. We learned how diffs and version control works for our replay feature. We also learned how create and authentication within VSCode and design a robust, normalized schema to manage a high volume of VSCode flushes and user information.

What's next for JumBuddy

We already have everything containerized through Docker, but the next step would be deployment to the cloud as well as reading student's inputs to the terminal.

Built With

Share this project:

Updates