RaterSidekick

RaterSidekick automatically builds a private record of an AI contractor's hours, tasks, earnings and project history across platforms without capturing screenshots or work content, just your record.

Comment

Inspiration

Behind every AI model is a workforce consisting of hundreds of thousands of contractors doing rating, annotation, and red-teaming work on platforms like Outlier, Handshake, Mercor, and DataAnnotation. AI contractors often work through several AI platforms, projects, and payment systems to secure consistent work, but their work history remains scattered across dashboards they do not control. I had an incident where I recorded that I worked on an AI training project for 40 hours on a particular week but when the invoice came, I was only paid for 27 hours, leaving me confused and with no way to check if I made a recording mistake.

My research soon discovered that many contractors end up maintaining spreadsheets, notebooks, timers, payment emails, or custom scripts just to keep a record of how many hours they worked, Which project it was for, What should they have earned, or a backup in case the platform record is incomplete or disappears.

I built RaterSidekick on a simple principle that AI rating platforms dashboards are the platform’s records. Contractors should have a private record of their own.

What it does

RaterSidekick is a private work record system designed specifically for AI contractors, raters, reviewers, annotators, and domain experts. It is a chrome extension and a web dashboard that, after configuration, automatically record work sessions on user selected sites when the contractor begins working and stop when they leave the tasking site, eliminating the need to remember a timer or keep a manual spreadsheet.

How we built it

RaterSidekick combines a Chrome Manifest V3 extension with a serverless web application.

The extension observes only user-configured work sites and completion controls. It records session events, active-versus-idle time, and task completions without reading or storing the content of the contractor’s work.

The dashboard was built with Next.js, React, TypeScript, Tailwind CSS, shadcn/ui, TanStack Query, and Recharts. It provides daily records, project filtering, productivity views, earnings estimates, historical imports, and CSV exports.

The backend uses AWS Amplify Gen 2, Amazon Cognito, API Gateway, AWS Lambda, and DynamoDB for authentication, APIs, processing, and storage.

Challenges we ran into

The hardest engineering problem was that AI work platforms do not have a standard workflow. Their page structures, UI, task controls, timers, and projects are very different and a key requirement for the project was that RaterSidekick be platform agnostic. So our solution was the extension lets users "teach" it their site's behavior once, then automates from there.

Automatic tracking also has to the difference between a user actually working versus doing nothing on a tasking site. So we had to balance accuracy with flexibility through active and idle mouse/keyboard detection and configurable auto start-and-stop behaviour.

Another major challenge was trust. Most tracking tools earn accuracy by capturing everything. We refused screenshots and content capture on principle, which left us with solving session detection with metadata alone.

Importing historical records created a separate challenge because every contractor structures spreadsheets differently. The import workflow therefore includes column mapping, validation, previewing, and normalisation before records are committed.

Accomplishments that we're proud of

RaterSidekick is live! We are proud to have built and shipped a complete working product and not just a prototype.

What we learned

Talking to dozens of contractors taught us the real job isn't just "productivity", even though RaterSidekick save our users hours, it's record ownership. The people who care most are the ones who already keep spreadsheets, cross check payouts, or have been burned by a dashboard discrepancy like I was. Platform dashboards are the platform's record, contractors want their own.

What's next for RaterSidekick

We are looking into the following features and will build them based on our users feedback:

  • Payout reconciliation per platform and pay period
  • Improve the task and project details in the dashboard
  • Tax-year and project payment period reports
  • richer cross-platform reporting
  • RaterSidekick validated career summaries based on the user metadata that can be used to apply to other AI projects.

Built With

Share this project:

Updates