Skill Passport

• 

  Landing Page

• 

  Auto Formatter Skill Analysis Pic 1

• 

  Auto Formatter Skill Analysis Pic 2

• 

  Auto Formatter Skill Analysis Pic 3

• 

  Auto Formatter Skill Analysis Pic 4

• 

  Auto Formatter Skill Analysis Pic 5

• 

  Anthropic PDF Skills Pic 1

• 

  Anthropic PDF Skills Pic 2

• 

  Anthropic PDF Skills Pic 4

• 

  Anthropic PDF Skills Pic 3

Inspiration

AI coding agents can install Agent Skills from public GitHub repositories. Those skills can include instructions and executable code with access to local files, networks, shells, and environment variables.

Today, developers often decide whether to trust a skill by reading its README or SKILL.md and hoping the documentation matches the source code. I wanted to turn that trust decision into an evidence-based review before installation.

What it does

Skill Passport is a read-only trust verification tool for public AI-agent skill repositories.

A developer provides a public GitHub URL. Skill Passport then:

1. Fetches documentation and source text through the GitHub REST API.
2. Parses fetched Python files without executing them.
3. Deterministically traces observable network, filesystem, shell, and environment-secret behavior.
4. Builds a Repository Behavior Profile with file and line evidence.
5. Compares observed behavior with repository claims in README.md, SKILL.md, and supported manifests.
6. Returns an evidence-backed VERIFIED, REVIEW, or HIGH RISK verdict.

For example, the HIGH RISK fixture reads OPENAI_API_KEY, places it into a payload, and posts it to an external telemetry domain despite claiming, “No network access required.” Skill Passport detects that contradiction and hides the installation command.

The tool never clones, imports, executes, installs, or modifies the repository it analyzes.

How I built it

The core is Python. The fetcher uses the GitHub REST API, and the tracer uses Python's standard-library AST module for deterministic static analysis. FastAPI provides a Server-Sent Events backend so the React web interface can stream each analysis stage as it completes.

The project includes four fixtures: a clean VERIFIED example, a disclosed telemetry REVIEW example, a contradicted secret-to-network HIGH RISK example, and Anthropic's public PDF scripts as a disclosed filesystem-access example.

The terminal version is published to PyPI and can be run without cloning the repository using pipx run skill-passport check <github-url>.

How Codex and GPT-5.6 helped

I used Codex and GPT-5.6 as implementation and review collaborators. Codex accelerated test-first implementation, debugging, frontend iteration, CLI formatting, and documentation.

I made the key product and engineering decisions: the read-only threat model, fixture design, deterministic detection categories, claims-classification rules, safety boundaries, and final validation against live repositories. I manually verified the CLI and web flows and reviewed the evidence and verdicts.

Challenges and lessons

The main challenge was keeping the analysis useful without executing untrusted code. I addressed this by fetching text only and using deterministic AST analysis for Python source files.

Another challenge was distinguishing expected sensitive behavior from suspicious behavior. Filesystem access in PDF-processing scripts, for example, should be visible and reviewable, but not automatically treated as malicious. That distinction led to the three verdict levels: VERIFIED, REVIEW, and HIGH RISK.

Built With

• ast
• codex
• fastapi
• github-rest-api
• gpt-5.6
• pipx
• pypi
• pytest
• python
• react
• server-sent-events
• static-analysis
• vite