Inspiration
Red teams and security researchers constantly need realistic “user” activity to test detection systems, but most synthetic agents are trivially easy to spot: they launch apps from scripts, click on fixed timers, and leave a process tree that screams “automation.” We wanted to build a synthetic human that actually looks human where it counts — in the process telemetry (Sysmon, Process Monitor, Process Explorer) that a defender inspects.
What it does
Our agent does ordinary office and web work on a Windows VM, entirely through the real keyboard and mouse: it opens Edge and reads a few sites, writes and formats a note in OpenOffice Writer, fills a table with =SUM() in Calc, browses folders in File Explorer, and copies a real web address from the browser into a spreadsheet. Everything runs offline, Python standard-library only, no pip. It’s scored on four axes — process authenticity, human timing/entropy, action complexity, and resource efficiency — and it’s built to pass all four.
How we built it
A clean layered architecture: input primitives (ctypes SendInput typing, curved mouse paths, scrolling, one central pacing module) → skills (reusable actions like “open app” or “fill sheet”) → objectives (short YAML task plans) → a runner that sequences them within a time budget.
Two decisions carry the whole thing:
Launch through the Windows shell. The agent presses the Windows key / uses the Run box and lets explorer.exe start every program — so each app’s parent process is explorer.exe, exactly like a real user. It never calls a process-launch API or spawns a child. We even launch the agent itself from a desktop shortcut so it is parented to explorer.exe too. Human timing from real research. Every pause is sampled from its own lognormal distribution with heavy-tailed “thinking” pauses, and the gaps between tasks use a two-component mixture (Barabási bursty-activity model). Measured over a full run: coefficient of variation 1.32, 92% of gaps over 2 s, a long tail out to ~119 s — versus a timer bot’s CV of ~0.
We validate with our own tools (timing_analyzer.py, timing_validate.py, task_timeline.py) and prove authenticity against Sysmon Event ID 1, Process Monitor (Process Create), and Process Explorer parent-PID columns.
Challenges we ran into Looking human in the logs, not just on screen. Getting explorer.exe as the parent for everything — including the agent process — meant abandoning every subprocess/PowerShell launch path and driving the GUI blind. Driving blind safely. Early on the folder-browser pressed Enter on a selected item and opened a random file; we made File Explorer strictly read-only (arrow keys + scroll, never Enter). Timing that survives a statistician. A single lognormal still looked too regular (CV ~0.5); the two-component mixture was what finally landed in the human band. An offline, cert-expired VM. The mock sites’ TLS certs had expired, so we ship a dedicated “Edge Test” profile; and with no network, we moved code in via base64 + certutil. What we learned
Authenticity is a telemetry problem before it’s a visual one. The hardest, highest-value work was invisible on screen — process lineage, inter-action entropy, keeping python childless and silent on disk (buffered logging). We also learned how much human timing research (Chenoweth & Hayes, Mark, Barabási) maps directly onto a convincing agent.
Log in or sign up for Devpost to join the conversation.