Inspiration
Developers test their own sites by clicking through them once, the way they personally would. But a 10-year-old, a 70-year-old, and a colorblind user all use the same page differently, and those problems only show up after launch. We wanted to see how those people would react to a site before it goes live.
What it does
Sightline simulates real users on your website before launch. You give it a task, like "buy a notebook, pick the dot-grid version, and check out," and it runs that task as several different people at once like a child, an older adult, a colorblind user, and a keyboard only user.
Each simulated user clicks, types, and scrolls in a real browser. If the colorblind user can't find the checkout button, or the keyboard only user gets trapped in a menu, Sightline logs it with the exact screenshot and steps. You watch all of them live on one dashboard.
How we built it
Nemotron is the model behind every simulated user. It looks at a screenshot of the page and decides the next action based on who it is pretending to be.
SpacetimeDB runs the simulated users in parallel without them interfering. Each user's actions and findings are saved through SpacetimeDB reducers, and our dashboard subscribes to those tables, so every click appears on screen the moment it happens.
Playwright controls the browsers. We also built a fake Amazon-style shop and a fake campus portal to test on, with synthetic checkout so no real payments happen.
Challenges we ran into
Nemotron kept clicking the wrong spot again and again when it tried to type in a text box not knowing that it was already focused. Some browsers also froze before sending their first frame when we ran five at once, which we had to debug live.
Accomplishments that we're proud of
We've run more than 86 simulations, with five users testing at the same time. When we deliberately hide a bug in the shop, Sightline finds it and shows the screenshot proving it.
What we learned
We learned that Nemotron likes to get stuck in a loop when there are no visual changes and that SpacetimeDB let us skip building a backend API entirely, since the dashboard reads straight from the database.
What's next for Sightline
We want anyone to be able to open a website that’s hard for them to use and, just by asking out loud, have it change to work for them.
Built With
- javascript
- nemotron
- spacetimedb
- typescript
