After working on Android app with my team for most of the day yesterday, we decided to start from scratch with something new since we were having a lot of trouble with Android Studio. While we were brainstorming new ideas that we could create using Java, we thought it would be cool to scrape images from a website and use them in a Galaga game. While we weren't able to do that, I took the idea of scraping content from a website and made a small application with it.

What it does

It uses JSoup to scrape the title of a website and an image tag from the website and then display that to the user.

How I built it

I built it using Java, JSoup, and NetBeans.

Challenges I ran into

It was really difficult for me to extract certain content from websites using JSoup since it would usually just give me the full HTML code when I only wanted a part of the HTML code. This created a lot of problems during the development of this app.

Accomplishments that I'm proud of

I'm proud that I was able to create an application that could actually scrape content from a website and display it to the user with graphical interface (not just in the command line). This was the first time I actually used JSoup and this was the first time I created an application using object-oriented programming principles (the application uses three different classes to run everything and uses ideas like encapsulation), so while it may not be the most technically advanced or impressive app ever made, I'm proud that I was able to create this in the short period of time that I had.

What I learned

I learned about what JSoup is and how to use it, and I learned about how to create graphics using Swing in Java.

What's next for WebScraper Tool

I'm going to try and expand it so it can scrape more content from a webpage and display it to you. For example, I want to update it to scrape an image from a website and display that image to the user. I also want to give it other functionalities like the ability to scrape information from a table and display that to the user. In the future, I could also try and make it work more like an RSS feed and have it scrape content from websites and present it in one feed that the user can check on.

Built With

Share this project: