Inspiration

I came across the tool called katana by project discovery, found it very tricky to use and it takes effort to learn, to took a step to build my own.

What it does

Crawls for web pages, links, files. Then saves them in a separate space. And it is also configurable at some point.

How we built it

The app is structured as microservices software. Every component works on its own. I used Ruby as the language Ruby and Shell as executable.

Challenges we ran into

UI was a challenge as curses are a new topic to me.

Accomplishments that we're proud of

Usability and Simplicity makes tarantula best option for a web discovery tool.

What's next for Tarantula

Machine learning models to solve detected captchas to continue crawling.

Built With

Share this project:

Updates

Submission history