pricecheck heals itself

When OLX changes its pages, my price checker stops reading listings. Now it opens a GitLab issue and three Duo agents fix, review and ship the repair in under 10 minutes, with no human.

Comment

Inspiration

My friend wanted a used 27" monitor and had no idea how much to haggle on OLX, Pakistan's biggest classifieds site. So I built pricecheck: it reads every monitor listing on OLX, has Gemma (running locally) pull out the specs, and has TabPFN predict what similar monitors usually ask, so he gets three numbers: open at, fair, never more than.

It works because it reads the data OLX embeds in its own pages. That's also its weak spot. The day OLX moves that data, every scraper like this one breaks, and usually nobody notices until a user does. Then a developer opens the page source, finds where things moved, patches the parser, and redeploys. That's the boring, urgent work after the code is written, and it's exactly the kind of job I wanted to hand to agents.

What it does

pricecheck now repairs itself on GitLab, from detection to production, with no human in the loop:

  1. Detect. When I run pricecheck and an OLX page matches no layout it knows, breakage.py saves a trimmed copy of the page (seller details and phone numbers removed, tested) and opens a GitLab issue labelled scraper-broken, saying exactly where the listings now live, e.g. window.__OLX_STATE__.search.results[].
  2. Trigger. A flow trigger on work item created starts Scraper Medic, a custom Duo Agent Platform flow with three agents:
    • scraper_medic reads the issue, saves the page as a test fixture, writes a failing test that reproduces the break, fixes olx.py to read the new layout and every old one, opens a merge request and watches its pipeline. If a job fails, it reads the log, fixes the real cause and pushes again. It may not weaken a test.
    • reviewer is an independent agent that didn't write the fix. It checks scope, that no test was deleted or loosened, that all older layouts still parse, that the fixture has no personal data, and that the pipeline for the exact commit is green. It posts a checklist with evidence and answers APPROVE or stops and asks a human.
    • shipper runs only on APPROVE: it fast-forwards main to the approved commit (never a force-push) and posts the report on the issue.
  3. Ship. The main pipeline then runs tests, SAST, secret detection and dependency scanning, packages the site into the package registry, deploys it, cuts a GitLab release for the heal, and smoke-tests the live site, checking it really serves listings, not just a 200.

Nothing runs on a timer: the loop starts only when pricecheck itself finds OLX changed.

Does it actually work? Four rehearsals

OLX can't be asked to redesign on cue, so rehearsal.py (4 variants) rewrites the real OLX pages the way a redesign might (real listings, data moved). Each rehearsal used a redesign the agents had never seen:

Rehearsal Simulated redesign What happened
1 (issue #2) results moved to listingSearch.page.items Fixed in ~5 min. Its first pipeline failed: 2 of my own tests used that layout as their example of an unreadable page. The agent read the log and changed them to a truly unknown layout, keeping every assertion. It couldn't merge: the flow token gets 403 on the merge API. I merged, and taught the flow to ship by fast-forward instead.
2 (issue #3) page data renamed to window.__OLX_STATE__ Fully hands-off: 3m45s from issue to main, pipeline green first try (69 s).
3 (issue #5) listing id renamed externalID → adId First run with the Reviewer: 6/6 checks ✅ with evidence, APPROVE, shipped by the agent in 8m02s (most of it a 5½-minute pipeline). It even avoided a trap: each hit also has a numeric id that is not the listing id. (A first attempt, issue #4, never started: my sandbox setup needed Python 3.10 and the sandbox has 3.9. Fixed.)
4, the recorded take (issue #6) listing title renamed title → name Diagnosis at +0:52, MR at +1:26, pipeline green first try, Reviewer APPROVE at +7:33, shipped to main at +9:51, then the main pipeline tested, packaged, deployed, released ("Heal 23") and smoke-tested the live site: live again at +16:57. This is the run in the video.

Each fix kept every older layout working. pricecheck went from 0 to 28 tests, 7 OLX layout fixtures, and a live page reads correctly after every heal.

How I built it

  • Duo Agent Platform: a custom flow (.gitlab/duo/flows/scraper-medic.yml, versioned in the repo) with three AgentComponents and a router that only reaches the shipper on APPROVE. Triggers: work item created, plus mention/assign for manual runs. Agents use glab and git in the flow sandbox.
  • Flow environment: the sandbox has no internet and Python 3.9, so agent-config.yml unpacks pure-Python wheels for pytest 8, requests, sentry-sdk and tomli bundled in the repo. The agents run the real test suite before pushing.
  • CI/CD: .gitlab-ci.yml with 6 stages (test, secure, package, deploy, release, monitor), the GitLab SAST, secret and dependency scanning templates, a generic package per main build, a release per heal (only for commits that close an issue), and a Python smoke test against production.
  • Detection: olx.py raises a LayoutChanged error instead of a cryptic KeyError. breakage.py finds where listing-like objects moved (anything with a title and a price), prunes the page down to those branches (~20 KB), scrubs private keys and phone numbers, and opens the issue with my token.
  • Monitoring: Sentry already traced every price check; a break is also sent to Sentry with a tag.

Inside the flow

One flow, three agents, one gate

The whole loop is one file in the repo, .gitlab/duo/flows/scraper-medic.yml (187 lines). The part that makes it safe is the router: the shipper can only be reached through the reviewer's exact one-word answer.

routers:
  - from: "scraper_medic"
    to: "reviewer"
  - from: "reviewer"
    condition:
      input: "context:reviewer.final_answer"
      routes:
        "APPROVE": "shipper"
        "default_route": "end"
  - from: "shipper"
    to: "end"

Anything other than APPROVE (a STOP, a REQUEST_CHANGES, a long explanation) ends the run and leaves the issue for a human.

What the reviewer checks

The reviewer is told it did not write the fix and is "the last check before it ships to production with no human in the loop". It must record evidence for 6 checks and post them as a checklist on the merge request:

  1. Scope: the diff only touches the OLX reader, the recipe helper, tests/ and tests/fixtures/. A change to pipeline config, flows or secrets fails the review.
  2. Tests kept honest: no test deleted, no assertion removed or loosened without a reason that keeps the same strictness.
  3. Backward compatible: it checks out the branch and runs the full suite; every older layout fixture must still be read.
  4. No personal data: the new fixture is grepped for Pakistani phone numbers (03xx, +92), contact fields, e-mails and seller names.
  5. Pipeline: the merge request's latest pipeline, for the exact HEAD sha the medic reported, is success.
  6. Fixes the issue: the new test reads the fixture from the issue and asserts real fields.

In rehearsal 3 it passed all 6 with a line of evidence each, then shipped in 8m02s.

The smoke test is not a 200 check

After a heal deploys, the monitor stage fetches the live site's index, then every category's data file, retrying 6 times 10 s apart while the CDN catches up. It fails the pipeline unless every category actually serves listings. A deploy that returns an empty page counts as broken.

Rehearsal 4, step by step

This is the run in the video. I started it by running pricecheck on the rewritten page; from there pricecheck found the break and filed the issue itself.

Time What happened
+0:00 pricecheck hits an OLX page where the listing title moved from title to name, raises LayoutChanged and opens issue #6
+0:52 Scraper Medic posts its diagnosis on the issue
+1:26 merge request open: new fixture, a failing test, the fix
first try merge request pipeline green
+7:33 Reviewer posts its 6-item checklist and answers APPROVE
+9:51 Shipper fast-forwards main to the approved sha
+16:57 main pipeline done: tested, scanned, packaged, deployed, released as "Heal 23", and the smoke test confirms the live site serves listings

Giving agents a working sandbox

The flow sandbox has no internet and Python 3.9, so pip install is not an option. .gitlab/duo/agent-config.yml unzips pure-Python wheels kept in the repo (pytest 8 and its 3.9 backports, requests, sentry-sdk, tomli) and adds a small pc-python wrapper. Every setup step ends in || true: after one rehearsal never started because of a setup error, the rule became "nothing here may stop the agent from starting". If pytest can't load, the agent still runs and the pipeline runs the tests. pandas and numpy aren't bundled, so the pricing tests are skipped in the sandbox and run in CI.

The detector

breakage.py reads the JSON OLX embeds in the page, walks it looking for anything that looks like a listing (a title and a price), and records every path where listings now live. It prunes the page down to those branches (about 20 KB), scrubs phone numbers and seller fields, and writes the issue: what the reader expected, what the page has now, and the trimmed page itself. That issue is the medic's whole brief, so a good issue is half the fix.

Stages covered

Plan (auto-opened, labelled issues; the agents narrate on them), Create (the fix and tests), Verify (pytest in the sandbox and in CI, failing-test-first), Secure (SAST, secret detection, dependency scanning, privacy check in review), Package (site tarball in the package registry), Release (a GitLab release per heal), Configure (flow, triggers and agent environment as code), Monitor (Sentry, live smoke test), Govern (separation of duties: the agent that ships isn't the one that wrote the fix; only an approved commit can ship; every step leaves an audit trail on the issue and MR).

Challenges I ran into

  • The flow token can't merge. glab mr merge returns 403 insufficient_scope. The fix: ship by fast-forwarding main with git, which the token can do, and only to the sha the reviewer approved.
  • Triggers only fire on human actions, so agents can't hand off to other flows. That's why it's one flow with three agents and a router.
  • The sandbox: no PyPI and Python 3.9. One rehearsal never started because my bundled pytest needed 3.10. That's now fixed, and setup can no longer block the agent.
  • A shell-injection bug in my own pipeline. The first release job passed the commit message to the shell, and a backtick-quoted id in the agent's commit message actually ran (uid=0(root)… appeared in the release notes). Notes now go through a file with printf %s, so a message can never execute. Agents write commit messages, so this matters.
  • Honest gate. I have the Developer role, so I can't enforce "pipelines must succeed". The reviewer and the shipper's "approved sha only" rule are the gate, and I say so.

Accomplishments I'm proud of

A real break-to-production loop with no human: 3m45s. The agents fixed their own pipeline failure without cheating on tests, and a second agent audited the first with evidence. Privacy held up: the fixtures that go into a public repo are scrubbed, and there's a test for that.

What it doesn't do yet

  • It hasn't met a real OLX redesign yet. All 4 rehearsals rewrite real OLX pages the way a redesign might; the first true redesign will be the real test.
  • Search pages only. The loop covers OLX's search results, not single-listing pages.
  • It starts when I run pricecheck. Nothing polls OLX on a timer, so a break is found the next time I run an update, not the minute it happens.
  • The reviewer can approve or stop, not send work back. A rejected fix waits for a human instead of going back to the medic.
  • One rehearsal needed me. In rehearsal 1 the token couldn't merge, so I merged by hand; rehearsals 2 to 4 shipped with no human.

What I learned

Write the agent's instructions like a runbook for a careful new teammate: the exact command to poll a pipeline, what "done" looks like, and what it must never do. Every snag in the rehearsals became one line in the flow.

What's next

Run it against OLX's listing pages too (not just search pages); let the reviewer request changes and send the medic back once; deploy to Google Cloud.

Path B: the project I started from, and what I added

Started from: pricecheck as it was on Oct 14, 2026 (https://github.com/miawab/pricecheck, MIT), built as a separate side project for my friend: OLX reader, Gemma extraction, TabPFN pricing, the web page, Sentry tracing. No tests, no CI, no automation, not on GitLab. Added for this hackathon (all of it): break detection and auto-issues (breakage.py, LayoutChanged), the Scraper Medic flow (3 agents, a router and triggers), the agent environment, the full CI/CD pipeline (6 stages: tests, scans, package, deploy, release, smoke test), 28 tests and 7 fixtures, the simulated redesigns for rehearsals, and the deploy from GitLab to Vercel.

Built With

Share this project:

Updates

Submission history