ProfileForge CLI

A CLI tool that auto-generates ready-to-use profiles for coding agents across OS and agent combinations, so developers spend less time configuring and more time building with AI.

Comment 1

ProfileForge CLI

What It Does

The coding-agent-mcp-tools repo publishes Agent Environment Profiles — curated MCP tool configurations for 8 coding agents (Claude, Codex, Cursor, Windsor, OpenCode, Cline, Roo Code, Kilo Code) across 3 operating systems (Ubuntu, macOS, Windows) and 2 tech stacks (Node.js + React, PHP + Laravel). Every combination is a discrete set of 9 markdown files. As the matrix grows, manually copying files, updating the README table, and keeping navigation.md in sync becomes error-prone and unsustainable. One missed update silently breaks the published state for every downstream user.

ProfileForge CLI (profile-cli)

ProfileForge CLI automates the entire profile lifecycle in a four-step pipeline:

1. validate — Pre-publish Gate

Reads every source profile from base-profiles/ and runs five checks on each file:

  1. File Presence: All 9 expected files are present (the setup file name is OS-aware — system-setup.md on Ubuntu, macos-setup.md on macOS, windows-setup.md on Windows).
  2. Required Content: Required content keywords appear per file (e.g., mcp.json in every setup file regardless of OS).
  3. OS-Specific Keywords: OS-specific keywords are present:
    • Ubuntu: bash
    • macOS: brew and .zshrc
    • Windows: PowerShell and winget
  4. Link Resolution: Relative ./ links inside each file resolve to real files in the same directory.
  5. Non-Empty: No file is empty.

Every failure across every agent, OS, and file is collected and reported in a single pass before the command exits — not just the first one.

2. publish

  • Runs validation inline first; if that fails, the command exits with code 1 and nothing is written.
  • On success, copies each file from base-profiles/ to profiles/.
  • Smart Skipping: Skips files whose content is byte-for-byte identical (string comparison, not modification time — git checkout resets mtime, making timestamp checks unreliable).
  • Creates missing target subdirectories automatically.

3. update-readme

  • Inserts or replaces the stack column in the README profile matrix table.
  • State Check: Before writing any link, it checks whether the agent's profiles are actually present in profiles/ for all three OS directories.
  • Agents not yet published get a - placeholder.
  • The table reflects the real state of profiles/ — no assumptions.

4. update-nav

  • Adds a shields.io badge entry to navigation.md for the stack.
  • Duplicate Check: Before inserting, it checks whether an entry for that stack path already exists; if one is found, it skips with no changes.
  • Styling: Badge color and emoji are stack-specific:
    • Node.js + React: Blue #38BDF8
    • PHP + Laravel: Purple #8B5CF6
  • The badge links to the first auto-detected agent's Ubuntu profile entry point.

All-in-One Command

All four steps are also wired into a single all command:

validate → publish → update-readme → update-nav

How We Built It

1. Runtime: Node.js >=18.0.0, Zero Runtime Dependencies

The tool relies exclusively on the Node standard library. The only imports across all source files are fs, path, and process.

  • No External Packages: No chalk, yargs, commander, or js-yaml.
  • Hand-Written Helpers: Every utility, including the YAML parser, is implemented from scratch.

2. Architecture: Thin Command Router & Focused Modules

The architecture centers on a thin command router (src/index.js) that delegates to nine focused modules:

  • config.js
  • generate.js
  • validate.js
  • publish.js
  • update-readme.js
  • update-nav.js
  • status.js
  • parse-yaml.js
  • utils.js
  • Entry point: bin/profile-cli.js

Design Principle: The router holds no business logic; it parses argv, resolves the stack and agent, and calls the appropriate module. Every module is independently testable.

3. Config as Single Source of Truth

config.js defines each stack, serving as the central configuration hub:

  • Paths: Source and target paths.
  • OS Variants: Definitions for Ubuntu, macOS, and Windows.
  • Filenames: The 9 expected filenames (OS-aware: system-setup.md, macos-setup.md, windows-setup.md).
  • Validation Rules: Per-file rules (mustContain, mustContainByOS).
  • Metadata: README and navigation metadata.

Every other module reads from config. Adding a new stack requires adding just one object in one file.

4. Hand-Written YAML Parser

generate.js reads instructions.yaml template files to scaffold new agent profiles. Rather than pulling in js-yaml, we wrote parse-yaml.js from scratch.

  • Capabilities: Handles quoted strings, block arrays, inline arrays, nested objects (up to 4 levels deep), and line comment stripping.
  • Limitations: Explicitly does not handle YAML anchors or multi-line strings, as our instruction files do not use them.

5. Spec-Driven Build

Documentation (docs/scope.md, docs/prd.md, and docs/spec.md) was written before any implementation.

  • Precision: The spec defined exact function signatures, return shapes, pipeline order, error messages, and exit codes.
  • Decision Making: The "open questions" section of the PRD forced the pipeline ordering decision (validate → publish → update-readme → update-nav) to be resolved on paper before runAll() was written.
  • Outcome: The wrong order (e.g., validate → readme → nav → publish) was caught and corrected during the spec pass, not during debugging.

Challenges We Ran Into

1. Agent Detection That Stays Correct as the Repo Grows

Any hardcoded agent list becomes a maintenance liability the moment a contributor adds a new agent without updating the CLI.

  • Solution: detectAgents(stackKey) in config.js reads base-profiles/<stack>/ for each OS variant, collects the agent folders per OS, and returns only the intersection — agents that have a folder in every OS variant.
  • Behavior: An agent with incomplete coverage is excluded with a warning written to stderr, naming exactly which OS directories it is missing from.
  • Benefit: No config changes are required when new agents are added.

2. OS-Specific Validation Without Conditional Branching Per File

Different OS setup files require different keywords:

  • Ubuntu (system-setup.md): Must contain bash and mcp.json.
  • macOS (macos-setup.md): Must contain brew, .zshrc, and mcp.json.
  • Windows (windows-setup.md): Must contain PowerShell, winget, and mcp.json.

Putting if (os === 'ubuntu') blocks inside validate.js would mean touching the validator every time a rule changes.

  • Solution: The mustContainByOS shape in config.js carries the OS-keyed keyword arrays.
  • Implementation: validate.js applies them with a single lookup — rules.mustContainByOS[os] — without knowing anything about which OS requires what.

3. Idempotent README and Nav Updates

Running update-readme twice must produce the same result — no duplicate columns.

  • update-readme.js: Searches the existing table for the stack's header string before writing.
    • If found: It updates the existing cells in place.
    • If not found: It appends a new column.
  • update-nav.js: Checks whether profiles/<stack> already appears in navigation.md before inserting. If it does, it skips with no changes.
  • Benefit: Both commands are safe to re-run after a manual edit.

4. Content Comparison Instead of Mtime for Publish Skip Logic

A naive implementation would check file modification time (mtime) to decide whether to skip an unchanged file. However, git checkout resets mtime on every clone, meaning every file appears "new" to an mtime check, causing every file to be re-copied on every run.

  • Solution: We compare raw content strings (srcContent === destContent) instead.
  • Behavior: A file is only copied if its content has actually changed.
  • Trade-off: This makes publish genuinely idempotent at the cost of reading both files before deciding — an acceptable overhead at this file count.

Accomplishments That We're Proud Of

1. The Validate Gate Is Real, Not Decorative

The validation step rigorously catches issues before any publication occurs:

  • Missing Files: Detects if any of the 9 expected files are absent.
  • Wrong OS Content: Flags missing OS-specific keywords (e.g., bash missing in Ubuntu profiles, PowerShell missing in Windows profiles).
  • Broken Links: Identifies relative links (e.g., ./docker-setup.md) that point to non-existent files.
  • Empty Files: Ensures no file is empty.

Every failure is reported with the exact agent, OS, filename, and reason — before a single file is published. The gate holds.

2. One Command Runs the Whole Thing

The profile-cli all <stack> command automates the entire workflow:

  1. Validates every profile for every agent across all three OS variants.
  2. Publishes only changed files.
  3. Updates the README matrix table.
  4. Updates navigation.md.
  5. Prints a final summary detailing what was copied, what was skipped, and what to commit next.

What was previously a dozen manual steps that would regularly go out of sync is now a single invocation.

3. status Shows Exactly Where Things Stand

The status command provides a clear view of the repository state, grouping profiles by Stack → Agent → OS. It identifies three states per profile:

  • Published: Content matches source.
  • Unpublished: Missing from profiles/.
  • Out of Sync: Exists but content differs from source.

For anything not yet live, it prints an actionable hint:

Run: profile-cli publish <stack> --agent <agent>

4. Zero Dependencies Shipped and Held

We strictly adhered to the constraint of no external packages (no chalk, no yargs, etc.):

  • ANSI Colors: Defined manually in utils.js.
  • Argument Parsing: Implemented as a concise 10-line parseArgs() function.
  • Portability: The tool runs on any machine with Node >=18 and nothing else.

5. The Spec-Driven Process Worked on a Real Build

The open questions table in docs/prd.md forced us to resolve critical decisions before implementation:

  • Pipeline ordering.
  • "Out of sync" detection logic.
  • README table structure.

These resolved decisions meant runAll() in index.js was written once and not revisited. The spec had enough precision that the build phase was execution, not re-design.

What We Learned

1. Spec Before Code Is Not Ceremony — It Surfaces Real Decisions

The pipeline ordering question (validate → publish → update-readme → update-nav vs. alternatives discussed in docs/prd.md) would have been resolved arbitrarily by whoever wrote the code first if we hadn't flagged it explicitly.

  • Outcome: Resolving it in the spec, with reasoning documented, meant no revisiting it mid-build.
  • Lesson: Explicit decision-making in the design phase prevents architectural drift during implementation.

2. Dynamic Detection Beats Configuration for Contributor-Facing Tools

A static ALL_AGENTS list exists in config.js and is used by status to show which agents are not yet in base-profiles/ at all, but the pipeline never uses it to decide which agents to process. Instead, detectAgents() determines what actually runs.

  • Why It Matters:
    • Static Approach: If a contributor adds a new agent folder but forgets to update a config list, the tool silently does nothing.
    • Dynamic Approach: The tool automatically picks up new agent folders without configuration changes.

3. Read-Only Validation Is the Correct Design

Keeping validate.js strictly read-only — calling no write APIs and taking no side effects — provided significant flexibility:

  • It can be run at any point without risk: before publish, as a standalone audit, or as a future CI step.
  • Lesson: Any validation approach that modifies files to "fix" them loses the property of being a safe, predictable gate.

What's Next for ProfileForge CLI

  • --dry-run flag on publish and all — preview exactly which files would be copied or skipped without touching the filesystem. Specified in docs/prd.md under "What We'd Add With More Time."
  • diff <stack> [--agent <name>] — human-readable diff between source profile and published profile before deciding to publish. Also from the PRD.
  • More stacks — Python/Django, Ruby on Rails, Go follow the same agent × OS × stack matrix and would plug into the existing config shape without changes to any handler module.
  • Interactive init flow — guided prompts for contributors creating a new agent's profiles from scratch: stack selection, OS coverage confirmation, file scaffolding, and immediate validate run.
  • GitHub Issues from validation failures — auto-create a labelled issue for each validate failure so missing or broken profiles become tracked work items, not silent gaps.
  • npx profile-cli — publish to npm so maintainers can run the tool without cloning the repo.

Built With

Share this project:

Updates