What it does & Inspiration
SurveySays is a website usability testing service. If you're a website owner and you wanted to find potential problems or get feedback on future website ideas, you could use our service to get dozens, hundreds, even thousands of user feedback. Usability testing isn't yet a crowded field, but the largest competitor would be UserTesting.com. Their model charges $49/per user giving feedback. However for SMBs this can price them out of the market. By leveraging Amazon Mechanical Turk, we've built a 100% completed automated, scalable system that can not only provide more volume of feedback, in a shorter time (due to more testing user availability) but also significantly cheaper. At $2 a user, we still make a sizable margin, and if you were a website owner spending $100 on usability testing, would you rather have 2 people's opinions, or 50?
How we built it
We wrote a custom database class and 2 extending classes as a plugin for a WordPress install. The same plugin also handled both form submission (and HIT creation) on Amazon Turk, as well as using WordPress's built in cron to check for newly completed hits. We then leveraged WordPress's existing plugin ecosystem, as well as many conveniently inbuilt functions (like wp_mail) to communicate with our users.
Challenges we ran into
We ran into all sorts of issues with Amazon's Mechanical Turk API. Many parts of the API were undocumented, and there were some parts of the API (notably the Pricing object used in the createHit endpoint) that were blatantly wrong as to both what they wanted, and what they did. We spent about 9 hours just on trying to figure out that one endpoint. Amazon's error system also contributed heavily to our difficulties. Instead of returning the actual error encountered, which would have led to an immediate correction on our part, Amazon returns 1 of 2 generic error messages regardless of the error that actually occurred, even if it knows what error occurred.
Accomplishments that we're proud of
We successfully got the original functionality planned done, as well as some of the stretch goals. Not only do we have a working API but a fully functional user management system with built in billing. This is not just a hack, but well on it's way to being a real business.
What's next for SurveySays
We're going to finish out the stretch goals we did for the user area as well as some add-on features, and then we'll be moving toward being an actual business.