Inspiration
I needed a crawler as most of the open source crawlers are complicated to use and takes effort to learn.
What it does
Provide one fixture directory per domain, run a crawl, and browse saved results from the interactive terminal UI
How we built it
This project is a event driven microservices software. I built the crawling module, then gave it life with UI.
Challenges we ran into
Database management and terminal UI was a challenge. Took it, solved it!
Accomplishments that we're proud of
Easy to use crawling which cuts off most of the complications and easy to customise.
What we learned
Advanced Ruby, Crawling mechanism, Database Management.
What's next for Tarantula
Make it fully customisable and export projects as .json format to export project to - [apis] Proxy
