What it does
This system traverses the internet cataloguing the websites it visits and pages it sees. It has a wide range of applications including powering a news consuming application or collecting natural language content.
How I built it
This system was built using PHP.
What I learned
I have learned that there is a large volume of data on the internet and as such prioritizing of the links to process and efficiency are critical to a successful application.
What's next for Wander
I will use this as the underlying data gathering mechanism for a future project.