This project is definitely inspired by the Stanford Open Data Portal. I was impressed when I first took a look at the portal and realized that there are many datasets from the university that could be interesting and valuable to work on further, whereas at the same time I've never known whether my university or other universities share or are willing to share such datasets. As I was building the data portal for my university, I discovered that in fact every U.S. States and even local areas have their own open data portal to share datasets and information owned by government officials. Given that data is unprecedentedly important these days, I am inspired to develop a data portal for my university.

What it does

The SODP Lehigh Data Portal provides a data catalog for users to browse open datasets for Lehigh University. Users are expected to browse and search for datasets they desire and use the provided links to download datasets and gain insights.

How we built it

Similar to the data portal built by Stanford Open Data Project, this data portal was also started by Create React App, which easily provides a serverless React repository to start my work. Since it doesn’t handle backend logic or databases, I set up an AWS S3 bucket to store objects in .csv and .json. Data was uploaded to AWS successfully and we use Axios to retrieve the objects on request. The portal essentially contains only one page utilizing the benefits that Create React App offers, therefore several components corresponding to different content were created. Using the react-router-dom allows users to navigate to different components and has the appearance as if it is linking to different pages. Along with the javascript files, matching css files were also created to style elements. Finally, the React App was deployed to Heroku and every new push to GitHub will automatically deploy on Heroku.

Challenges we ran into

One of the challenges that I ran into is the retrieving process of the objects stored in AWS S3 bucket. I used Axios to get to the url address but suffered the problem of "XMLHttpRequest blocked by CORS Policy". Though I understood why there is an issue and spent a lot of hours trying to solve this issue but due to time constraint, I did not manage to solve this problem. Another problem that may have connection to this previous problem is that though I was starting in the team of 4, I was the only one being active and it seems like other teammates all quit the datathon. If more people are being active, I think we would be able to distribute the work more evenly and efficiently so that the MVP would be more functional.

Accomplishments that we're proud of

Throughout the few days I work on this project, I am definitely proud of myself being able to get the portal working with essential functionalities added. I also gained a lot more knowledge about creating React Apps and the various apis that are handy to use. I am taking a Capstone project course at my university this semester and the tech stack that we use in that project are similar to what I did for this datathon, so I am confident I could transfer and apply what I learn recently to that project.

What we learned

Participating in the SODP datathon taught me a few things. I am glad that I did not quit even with no other teammates working with me together. I encounter many issues as I was doing the project, but having the belief that I am able to complete building the website, I told myself to be patient and good things would happen. I even had to restart my work all over again realizing that one of the structures I choose may not work well.

What's next for SODP Lehigh Data Portal [Track 3]

The priority for my project is to definitely make the dataset retrieve functional. Only by then I would be able to consider other more advanced functionalities. Another thing is that since the entire goal of this project is to make university data public and accessible, I researched whether my institution maintains open data and found that Lehigh University’s Data Center publics current and past data on university profile, admission, majors and etc. However, these data are either in the formats of pdfs or Tableau, and thus add difficulties to further study and analyze. Therefore, I am also looking forward to transform these available data into more compatible formats like .csv.

Share this project: