The standard view for the user upon entering the website. The 'Standard comparison' privacy check option is selected.
The user has changed the privacy check option to 'Private comparison (JIFF demo)'.
The user has opened up the settings. They can change model provider, model type, and enter an API Key here.
Post-analysis, AirLock displays the documented with highlighted sources of PII. The user can select pages and see reasoning.
The user has scrolled down on the suggestions. AirLock blocks the user from continuing without changing certain PII.
The user can see the suggested final document compared to their original document in the 'Comparison' section.
In the 'Rubric' section, the user can input the rubric file to be parsed, then compare their paper against it.
The rubric has been parsed and the user can start the comparison.
AirLock checks the paper against each criterion and explains why.
AirLock has parsed the file and displays the criterion. The user can now have their paper analyzed.
AirLock analyzed the paper and displays what criterion are hit and what criterion are missed. It also gives a quick reason for each.

Inspiration

We weren't really inspired by anything else. All of our ideas were created through brainstorming among the group. Thesis checkers and Personal Identification Information (PII) checkers are not everyday tools, and although I'm sure there are similar tools to ours out there, we just built it up from scratch.

What it does

When the user first enters the website, they are prompted to enter their thesis/dissertation final into a drop box, and can choose what style of privacy check they would like performed. After starting the check, AirLock analyzes the paper with the chosen model. It parses the document, then compares it against a database containing possible forms of PII, giving each token a k-score based on how identifiable it is. AirLock then returns to the user the document, highlighting all places containing PII. K-scores of 5-19 receive a warning, which AirLock suggests to be changed, but does not mandate, highlighting them in blue. K-scores less than 4 receive a red highlight, and AirLock blocks the user from continuing until these parts are changed. The user can also view the suggested changes side-by-side with their original document in the 'Compare' section. Once the user has corrected their file, they can also check their document's formatting. In the 'Rubric' section, the user inputs the rubric file that they would like their document checked against. After loading the rubric file, AirLock uses a REST(ful) API to a chosen LLM model, which then responds with a specific rubric format. Later, AirLock compares the scrubbed and PII-removed data and can grade the document with the submitted rubric.

How we built it

We used Kiro all throughout the code creation. Once we formulated our ideas, we created each individual component as separate GitHub files, and team members created separate GitHub branches to change code before implementing into the main code. Kiro was essential to our code creation, as the bulk of the code was generated by it. However, as the project grew in size, our style of coding led to Kiro starting to 'hallucinate' and bugs started to pop up. We the studied the way that Kiro worked and came to the realization that as the scope of the context increased, A. Kiro came into a negative feedback loop and iterated on its own mistakes, and B. we realized that we needed to keep our context window short, but also summarize and reintroduce context each time we restarted the agent. We then used Claude to help us fix these problems before merging. One team member, Daniel, then worked independently to merge the separate branches into the main code properly, and debugged it.

Challenges we ran into

One of the biggest problems we encountered was Kiro starting to 'hallucinate'. As the scope of the context increased as we continued expanding upon the project, we realized two things. First, Kiro started to loop on negative feedback and iterate over and over on its own mistakes. Secondly, we realized that we constantly needed to summarize reintroduce context each time we restarted the agent, while also keeping the context window short. Another problem we encountered was merging the GitHub branches of code together into the main code. This style of coding allowed all team members to work independently, and sped up the coding process at first, however, it led to inconsistencies between branches that had to be smoothed out once we went to merge.

Accomplishments that we're proud of

Considering the difficulty of merging the branches of code, once Daniel properly merged them all and the project worked as intended, we all felt a surge of pride. This also tied in with submitting the project to Hackachusetts. We planned on submitting to Hackachusetts from the beginning, but as the deadline got closer, we still had a ways to go. The last 30 minutes was a mad rush to properly submit the project, so once we were able to upload the video of the code demonstration and then submit our project to Hackachusetts just in time, that also led to a surge of pride.

What we learned

We learned how to effectively prompt and use Kiro to help us construct the exact frontend we needed. We also learned about branching and pulling from Github in varied ways. We learned how to properly plan before committing to code to ensure smooth building process whilst mitigating the impacts of errors or miscommunication. In addition, we learned the context framing of Kiro and how to avoid hallucination. We learned how to use k-heat maps and various data analysis tools to ensure a proper data flow pipeline for our first hackathons.

What's next for AirLock

In our ideation, the potential of AirLock' Rubric Checker functionality to be expanded to all types of rubrics and not just BU's thesis/dissertation temples came up early on. This would allow all kinds of projects to be compared to their rubrics and checked for proper formatting. It's very easy to miss formatting issues in projects, especially when it's not the main content of the project, but even small problems like these can prove extremely detrimental to projects. Applying AirLock' Rubric Checker to all kinds of format rubrics would solve this issue,