Family

• 

  Family

What it does

Family is a private continuous improvement layer for the collaboration between humans and Codex. We build a personalized private eval in the form of an RL environment that evolves with their work, and allows humans to measure how well things like reasoning_effort, skills, goal setting, etc. impact their work. It also continuously updates with the best practices for Codex and OpenAI models.

We believe this approach will lay an important foundation for increasingly capable AI systems with greater personalization and safety.

It offers two useful starting points:

• Already using Codex: explicitly consent to a local activity review within codex, turn representative work into a reviewed RL environment built with PrimeIntellect, and see what changes could be made to improve collaboration.

• New to Codex: describe one desired outcome and leave with one bounded first task, one practice, and visible evidence to inspect.

Family's deterministic scans are local, provider-free, and read-only over source session stores. For each candidate task, Family asks for the exact result, allowed material, forbidden actions, visible success check, and scoring method.

How we built it

Family ships as a Codex plugin with a natural-language coaching skill backed by a deterministic Python 3.12+ and uv workflow engine. We built it in Codex using gpt-5.6 over several projects and threads.

The implementation includes:

• consent-first local evidence inventory and privacy-safe summaries;
• a representative authoring planner that stores derived metadata and digests, not historical raw prompts;
• two trusted synthetic compiler families: repository change monitoring and bounded evidence briefs;
• native verifiers 0.2.0 Taskset and custom Codex Harness artifacts;
• actor/host package separation, mutation proofs, exact regeneration, canary tripwires, and fail-closed validation;
• reversible improvement previews, approvals, task bindings, results, and rollback receipts;
• reproducible stripped private-preview bundles with checksums and activation controls; and
• CI across Python 3.12 and 3.13 covering tests, linting, packaging, plugin validation, the offline demo, and the sanitized bundle.

Codex served in two distinct roles. As a builder and reviewer, it helped translate privacy and evaluation requirements into inspectable workflows, artifacts, tests, and documentation. As the evaluated agent, exact GPT-5.6-sol through Codex CLI ran a sealed diagnostic and a 40-run two-trial research baseline. The deterministic compiler rehearsal itself makes no provider call.

Challenges we ran into

The hardest problem was not generating benchmark cases. It was deciding when evidence deserved to count.

Session history can contain sensitive prompts, paths, labels, and identifiers, so every boundary needed explicit consent, owner-only storage, redaction, and separate retention choices. We also had to distinguish facts that sound similar but are not: excluding answers and graders from an actor package does not prove live runtime unreadability; a canary is a leakage tripwire, not proof that a score is correct; and a high pass count does not establish a stable evaluation.

A second challenge was making the product valuable for someone with no usable history. Family cannot treat missing history as a deficiency, so the beginner path starts from one desired outcome and teaches one inspectable practice.

Accomplishments that we're proud of

Benchmarks for early testers were surprisingly robust, and offered excellent improvements to their workflows. This early work shows a glimpse of what can be possible for personalization of incredibly advanced AI systems.

What we learned

Personalized evaluation is a human-agent authoring process, not automatic profiling. A representative queue can suggest what to review, but only a person can decide whether a task reflects their work and whether its scoring method is legitimate.

We also learned that evaluation stability belongs in the product experience. Showing why a result is invalid can be more useful than presenting an impressive but brittle number.

What's next for Family

Next we will complete beginner and experienced-user pilots, expand the trusted compiler registry only through reviewed task mappings, evaluate representation separately from difficulty, and run a fresh unseen matched comparison before making any causal improvement claim.

We have exciting improvements coming in the next versions regarding safety and preparedness.

Built With

• chatgpt
• codex
• developer-tools
• evals
• gpt-5.6
• openai
• privacy
• safety
• uv
• verifiers