Vibe Test - test your vibe coded app

Get a birds-eye view of your app every time you vibe a change, so you never get stuck again.

👆Record Tests: Use your app once to automatically record tests just by interacting.
▶️ Replay & Auto-Update: Replay “test videos” on repeat after every edit, updated to your app's latest version.
✅ Code w/ Confidence: Glance across the grid to instantly spot sneaky bugs, so you always vibe code from a solid foundation.
⏰ Save Hours: Eliminate repetitive manual re-testing. Build more, test less, with confidence your app still works.

🔗 Try It Link: Vibe Test

Inspiration

I interviewed dozens of vibe coders and the ones successfully launching products were spending 80% of the time testing and only 20% of the time building.

Other vibe coders just YOLO, then realize 5 changes later that the AI broke their entire app and never get to the finish line.

So I thought, there HAS to be a better way…

The Journey

Approaches Considered

Traditional auto-test frameworks like playwright / puppeteer
- purpose built, but slow to execute resulting in poor UX
- flaky, constantly needing fixes, sometimes taking more time than just testing manually
- requires heavy backend such as ec2, not well suited to vibe coding (and unsure if allowed in hackathon)
AI browser use
- cool + trendy AI
- but still nascent and simply not good enough to test most applications yet
Session record & replay “videos”
- automatic for users to create tests, intuitive and quick to review results
- novel approach with significant execution risk since it hasn’t been done before as far as I know

Doing something new and unique is either brilliant or dumb, but I’ll always choose that over boring, so I decided to go down path #3.

Project Size

AI gets worse as the codebase and context get longer. I knew this project would be massive, so I built each individual feature as separate Bolt project, then combined at the end.

First, I made a starter project with the basic structure and UI components.
I cloned that as a starting point for each major screen and feature and built them each in separate bolt projects.
Then, at the end, I copy pasted the relevant files into one final project, and prompted the deeper integration.

Vibe Coding Core

I had to pivot several times along the way.

When I first started building, my plan was to fork bolt.diy, the open source core of bolt, and bring it into bolt.new to vibe code my new features on top. But the preview wasn't able to load it, so I had to pivot.

Next, I started vibe coding a simple vibe coding app, just to unblock myself from building the testing features on top.

But, part way through, I realized it would be much more helpful if Vibe Test could literally work with any vibe coded app from any platform… So I pivoted again and created a 1 line code snippet that you copy paste into any vibe coded app to enable vibe testing.

Technical Difficulties

I ran into a major challenge part-way through and almost gave up.

The DOM doesn’t keep consistent unique identifiers across multiple code versions, making it extremely difficult to apply the original test events to a new version of code automatically. And I need to do this without access to the full source, just the minified output in the browser.

Each test is an initial snapshot of the code + a diff event for each user interaction or data change in the app. When I detect a new app version, I record a new initial snapshot, then replace each original test snapshot with the new one, preserving all the user interaction + data diff events so the original test plays on the updated code.

I (partially) solve the unique IDs using a multi-pass approach to stitch together matching DOM elements across multiple versions of code:

I traverse the DOM hierarchy in a consistent order every time, applying initial unique IDs
I look for exact matches and fuzzy matches using the Levenshtein distance algorithm:
- accounting for code insertions / deletions
- matching any available class names
- matching granular style and content data
I score how close of a match an element is across versions using a rubric that weights the various approaches + a cut off for whether it counts as a match or not

This works surprisingly well in 90+% of DOM cases…

Metrics & Next Steps

After ~2 weeks of non-stop building in public on X @eve_silb:

I launched the MVP
hustled to find 300+ users
and gathered a bunch of feedback

Based on user input, I’ve been fixing bugs and making lots of improvements. But there are still some challenges...

Vibe Test works with previews from all popular vibe coding tools except Bolt (!) due to its strict cross-origin policies. If you're vibe coding with Bolt, you'll have to publish first to Vibe Test. But Eric says Bolt is releasing a fix for this soon!

I also keep discovering new areas that Vibe Test doesn’t cover yet. For example: canvas. I’ve come to the realization that I’m chasing the entirety of the web spec and there are still many improvements to add!

The need to embed the code snippet in your app is a longer onboarding flow than ideal so the conversion from visitor to user is lower than basically all other apps I’ve launched before. I plan to explore alternative solutions for this, such as turning the app into a chrome extension or desktop app.