DOELT - DOE Legislative Tracker

Inspiration

The Hawaii Department of Education (DOE) supports our state by promoting student achievement and fostering educational excellence while ensuring equal access for all. The department's current legislative tracker is in need of a replacement since the last expert in charge of maintaining it will be retiring soon. Without the legislative tracker, the DOE will not be able to efficiently evaluate and track new bills and measures from the state capitol, which may put future generations’ education in jeopardy. As both current and former students at the University of Hawaii at Manoa, we created our program, the Department of Education Legislative Tracker (DOELT), with the goal of not only helping the DOE but also improving the educational environment for future Hawaii students. We believe that this system will be valuable to fulfilling the Department of Education’s mission.

What It Does

DOELT is a modern legislative tracking system that is designed to replace the DOE's current legislative tracker. DOELT integrates a modern user interface that enhances the system's accessibility and usability. It uses a custom data scraper API to ensure that its information always reflects the latest information from the state capitol. The scraper API gets information directly from the Hawaii State Legislature site in realtime and consists of three different scrapers. The first scraper gets information from the “Advanced Reporting” page, which consists of all of the bills and measures for a given year. The second scraper obtains more detailed information from individual bill/measure pages. Finally, the third scraper retrieves important events and links from the “Upcoming Hearings” page.

The scraped measures are accessed via the 'All Bills' page. From here, users are able to search for specific bills and save them into the DOE's database. The bills saved into the system's database can be viewed on the 'DOE Bills' page, where the user can access and edit the bill information in depth and create testimonies. Testimonies’ statuses are tracked in the homepage and they can be downloaded as PDF files. Lastly, there is a 'Calendar' page that allows users to stay informed by displaying information about upcoming hearings.

There are five roles in this application related to the DOE's system for writing and approving testimonies:

Assigner/Processor: creates the initial testimony and assigns it to the Writer
Writer: writes the testimony and sends it to the Office Approver
Office Approver: can edit the testimony and send it to the PIPE Approver
PIPE Approver: can edit the testimony and send it to the Final Approver
Final Approver: can edit the testimony and approve the testimony

How We Built It

The Meteor Application Template Production from the University of Hawaii's Department of Information and Computer Science was the core web application framework of this project. The frontend was built with ReactJS, React Bootstrap, and the FullCalendar library. The backend was built with MongoDB, and the scrapers were built with axios, Express JS, and cheerio. In terms of team collaboration, each person chose different tasks — normally defined as "issues" in software engineering language — throughout the development of the application. To keep track of our progress and task assignments, we used the GitHub Project Board. Discord was used for both meetings and chat-based communication.

Challenges We Ran Into

The web scraper that the Department of Education provided was not easily compatible with the technology stack that we were using and did not provide all of the information that we needed. Building the scrapers ourselves was risky due to the time constraint, but we were able to get it done alongside with the rest of the application.

We also had some issues trying to deploy our website. We tried a number of different platforms and even considered paying to get the site deployed, but in the end we were able to deploy via Digital Ocean.

Making sure that everyone was on the same page was difficult since we only met virtually. Some of us have not used GitHub in a collaborative way before, so we needed a little more time to ensure that everyone understood any changes that were made and how to properly implement their code to the project.

Security

The application is assumed to be run within the Hawaii DOE's network, therefore it can only be accessed by state workers. A security measure that we have is to not allow new people to create accounts directly from the sign-in page. Instead, a system administrator would need to be contacted to create a new account. The administrator would then verify the user's identity and give them access to their account. This security measure prevents people who are not members of the Department of Education from creating accounts. Another security measure is that users with the ‘Assigner’ role are only able to create testimonies and not edit the document. This is because it is the responsibility of the users with the 'Writer' and 'Approver' role. The ‘Assigner’ is able to view the testimony, and save it as PDF, but all of the text fields are set to be read-only. Using HTTPS instead of HTTP would be essential when the state wants to deploy the application for production. HTTPS uses encryption and verification to send and receive requests and responses, therefore it is much more secure compared to HTTP, where requests and responses are readable in plain-text.

Accomplishments that we're proud of

As mentioned above, the scraper that challenge sponsor provided was incompatible with our framework and did not give us all the data that our application needed. It was suggested that creating a scraper was not necessary for this challenge and it was recommended to copy and paste some bills from the Capitol's site or to use a provided dataset. However, we wanted to challenge ourselves so we decided to create our own scraper from scratch. Ultimately, we were successful in building three different Javascript-based scrapers that were tailored to our needs.

We are also proud of creating the dynamic dashboards that can sort and filter data in a multitude of ways, implementing a calendar that uses the scraped data to automatically create Upcoming Hearings events, and for allowing the testimonies to be exported as PDFs.

What We Learned

In addition to the technical knowledge, we have also acquired organizational and time management skills throughout our development process, while becoming familiar with the prerequisites and key components of the application. Understanding our users and how we can create an application where they can be productive was crucial. Lastly, planning and prototyping an entire application was something that some of us did not have experience with, so this project was very rewarding.

What's Next for DOELT - DOE Legislative Tracker

The next step is building multiple levels of administrative authority, which will provide users in various positions or offices access to different application features. Additionally, we need a stronger and more comprehensive management plan for both the system and the users. To be more explicit, the administrator should have the ability to add new accounts for users, change their roles as needed, and delete accounts when users are no longer employed. The dashboard on the homepage could also be improved to give users a more customized experience depending on their role.