Dark Intel

search results for the search term india
Limit Warning when search's are made before waiting

Inspiration

While doing dark web threat intelligence, my accounts were getting constant bombarded with graphic material in chat or the posts, these made it difficult to share my finding with people who were not familiar with the dark web landscape or how threat actors and hacker behave. So initial I made a web scrape to get all the posts, reconstruct their URL, their view and number of replies and during global hack week I got the opportunity to add a frontend to this project and experiment with Bootstrap in ReactJS.

What it does

Dark Intel lets the user view cyber crime posts and helps threat analysts quickly find popular posts related to key search terms that could help in their investigation.

How I built it

Dark Intel's frontend is built using ReactJS and Bootstrap to give to a more professional look, whereas the backend is written in python using Fast API to create the endpoint and playwright + beautiful soup to scrape the data from the site. I have also containerized the application using docker

Challenges we ran into

The cyber criminal forum kept throwing Cloudflare or kiwi farm turnstiles and captcha to stop the web scraping attempts, I had to create a local storage and save it to a JSON file to be loaded on each backend start. The JSON file contained login cookies as well as some random data I add to bypass the bot detection on the site.

What's next for Dark Intel (Try using Bootstrap Challenge)

Currently, Dark Intel only has a single API that is used to scrape a single cyber crime forum, I will continue working on Dark Intel to add more forums by scraping them and creating API's for them, to increase the coverage of Dark Intel.