Who is the intended audience for this project?
Engineers who want to audit the usage and stability of their feature toggles
What problem is this trying to solve?
When engineers make feature toggles, they typically stick around in the codebase for longer than they're needed. Depending on the toggle and the amount of code changes made to that area of the codebase, it's easy for a toggle to become outdated and start sending customers down unknown codepaths. We wanted to make a way that could automatically and randomly audit toggles to find these potential issues.
What is the proposed solution?
Enter NightmareFair, a web crawler that goes through the site and randomly flips feature toggles just to see what happens.
How technically feasible is this solution?
SEO already has a web crawler, so all we had to do was add a feature toggle override to the initial URL (to set it in session) and then crawl the site with it. From there we could observe what pages the crawler was hitting and what status code each page was returning.
How does this solution benefit the intended audience (and anyone else)? What is the impact?
Ideally, feature toggles should be able to be set in any state without creating errors or customer impact. This gives engineers a tool that can verify they don't have any risky toggles that could possibly start making errors or take down the site.
What’s next?
Since this is a risky idea that has the potential to modify production data, we only tested with a fake feature toggle that's not implemented in code. Ideally this will be able to be run periodically by grabbing a random toggle and randomizing it's ON/OFF value. It should be able to report any errors that happen during the crawl back to the team who owns the feature toggle
Log in or sign up for Devpost to join the conversation.