ApplyPilot

Inspiration

Job hunting has turned into an exhausting routine of re-typing the exact same contact details, work authorizations, and questionnaire answers into dozens of clunky ATS portals. Existing tools fall into two extremes: fragile browser extensions that break on modern dynamic forms, or unchecked "auto-apply" bots that hallucinate answers and submit without review. We built ApplyPilot to handle the tedious 99% of form-filling on a live cloud browser, while keeping the candidate in complete control at the finish line.

What it does

• Autonomous ATS Autofill: Takes any job link (Ashby, Greenhouse, Lever) and autonomously fills out inputs, dropdowns, checkboxes, essays, and resume uploads using Claude 3.5 Sonnet and Playwright MCP.
• Remote Cloud Browser with Live View: Spins up stealth cloud browser instances via Steel.dev and streams an interactive video viewer so you can watch the agent work in real time.
• Adaptive SQLite Memory: Learns your answers to custom questions (salary, visa, notice period) and matches them in future applications using exact, substring, and keyword-token overlap search.
• Human-in-the-Loop Interception: Prompts the user once when an unmapped question appears, saves the answer to memory, and never asks it again.
• Guarded Submission ("Last Mile"): Automatically halts right before clicking "Submit Application", providing an execution summary for final human review.
• Web Dashboard & Kill Switch: Manage candidate profiles, upload resumes, edit/search/export learned memory, and interrupt ongoing runs with a single click.

How we built it

• Reasoning Engine: Anthropic Claude 3.5 Sonnet (@anthropic-ai/sdk) navigating via accessibility tree reference IDs rather than fragile CSS selectors.
• Browser & Protocol: @modelcontextprotocol/sdk wrapping @playwright/mcp, dynamically translating 24 Playwright tools into Anthropic tool schemas.
• Cloud Infrastructure: Steel.dev (steel-sdk) for remote CDP sessions and live-streamed browser viewing.
• Safety Interceptor: Custom submitGuard proxy that checks element roles and action types against heuristic regex patterns to prevent accidental submissions.
• Memory Store: Persistent SQLite (better-sqlite3 in WAL mode) powering candidate profiles and fuzzy-matched Q&A memory.
• Web UI & Server: Lightweight Node.js HTTP server and vanilla HTML5/CSS3 dark-mode dashboard with real-time log streaming and RESTful CRUD endpoints.

Challenges we ran into

• Dynamic Shadow DOMs & Custom Inputs: Modern ATS forms avoid standard HTML inputs. We bypassed selector issues by using Playwright MCP's accessibility tree (browser_snapshot), interacting with elements via semantic reference IDs like a screen reader.
• Preventing Premature Submissions: Multi-step forms often have "Next" buttons that resemble submit actions. We built a heuristic guard that analyzes button text and context to permit pagination while intercepting final submission.
• Eliminating Hallucinations: Prompt-engineered strict fallback boundaries so Claude never guesses unknown candidate qualifications—forcing it to trigger our human-in-the-loop tool instead.

Accomplishments that we're proud of

• Flawless Real-World Autofill: Successfully completed complex, multi-field applications on live production portals (e.g., Wealthsimple on Ashby).
• Self-Learning Flywheel: Pre-seeded with 26+ learned answers; after 2–3 applications, ApplyPilot runs almost entirely without human intervention.
• Rock-Solid Test Coverage: 90 automated unit and integration tests across 17 test suites running in under 1 second via Node's native node:test runner.
• Instant Interruption: Built an AbortController pipeline that immediately cancels in-flight LLM calls and browser actions upon user request.

What we learned

• Accessibility Trees > Raw DOM: Accessibility snapshots provide far higher reliability and token efficiency for LLM web agents than noisy raw HTML.
• MCP Standardizes Agent Tooling: The Model Context Protocol made decoupling browser orchestration, custom guards, and local tools clean and maintainable.
• HITL Builds Trust: For high-stakes workflows like job applications, keeping the human at the review boundary is essential for real user adoption.

What's next for ApplyPilot

• Parallel Batch Queue: Queue 20+ job links from LinkedIn and autofill them concurrently across parallel Steel cloud containers.
• Dynamic Resume & Cover Letter Tailoring: Automatically emphasize relevant candidate experiences matching specific job description keywords before upload.
• Browser Extension: A lightweight Chrome sidecar to trigger ApplyPilot directly from job boards with one click.
• Enterprise ATS Adapters: Deep navigation support for legacy multi-page enterprise systems (Workday, Taleo).

Built With

• claude
• playwright
• sqlite
• steel
• typescript