About the Project
This project was inspired by the need to reduce unnecessary traffic from crawlers and improve website performance for users in different countries.
I learned how to identify and block malicious or unnecessary bots using techniques like User-Agent filtering, IP blocking, and CDN-level controls. I also explored performance optimization strategies based on geolocation, including edge caching and content compression.
The project was built using a Node.js backend with Cloudflare as a CDN and WAF. I used geolocation APIs to tailor content delivery by region, and applied caching and compression strategies to reduce latency.
One of the main challenges was distinguishing between good bots (like search engines) and harmful crawlers, while maintaining a smooth experience for real users across the globe.
