git-grade

Inspiration

I help to run a student-initiated course at the University of Maryland titled CMSC389R: Introduction to Ethical Hacking. This is the 2nd semester that we're running the course, so we had some ideas for automating the submission and grading process. Introducing git-grade, an automatic grading framework for classes that deliver assignments through git.

Demo

You can check out the asciicast here

What it does

git-grade has several features to assist in collecting and grading assignments:

init: parses a list of GitHub usernames and clones their assignment repositories to a folder. We instructed our students to simply fork our course repository and keep the same name, so we can easily clone them.
verify: parses a config JSON file that describes a folder/file structure against which each student's repository will be tested. This is to ensure that students include the necessary files, scripts, etc as a part of their submission.
fetch: for a previously initialized set of repositories, fetch will simply update each student's repository to ensure their latest work is received.
deadline: given a date and time string, deadline will log which repositories have commits that are beyond a deadline. These need manual intervention, or late-penalties applied accordingly.
test: along with proper submission, homework and projects need to produce the correct results. test allows for each repository to be tested for code correctness and flexibility in how testing is performed.

How I built it

I used Python, since it's a language that throws the fewest syntactical bugs for my style of programming. I picked the GitPython module versus half-baked os.system() calls to develop git-grade more as a framework, rather than a bespoke script. The click module allowed for slick and easy command-line argument parsing and documentation as well.

Challenges I ran into

GitPython and click were both libraries that I had only heard of, and had no familiarity. The toughest part was sitting down and reading enough documentation to figure out what methods and class types I needed. Once I understood the organization and philosophies of each module, it got easier to figure out what I needed to architect a feature. The only other difficulty was applying some of the algorithms and techniques I've learned onto new data structures and schemas.

Accomplishments that I'm proud of

I got to fully engineer a project from start to "finish," and got comfortable with two Python libraries I've been itching to learn!

What I learned

Documentation makes the world go round
Learn how to read compiler/interpreter error messages, no matter how cryptic
Python virtual environments are a godsend

What's next for git-grade

There were a few features that came to mind as I was developing for this:

Support non-GitHub git repositories
- Support various URL types other than HTTPS as well
- Prompt to add key to ssh-agent for cloning repositories via SSH
Perform more robust error checking, and try and fail safely rather than simply crash from an uncaught exception
Command suggestion for mistypes
Manpage entry, if this project gains traction
Test suite for git-grade itself to ensure correct behavior
Flexible testing for non-programming assignments
- Word count for blog-style assignments, as an initial classifier
- Data analysis/verification for experimental labs