Mikel AI Rule Test

Mikel AI Rule Test turns workplace policies into testable rules, runs realistic employee scenarios, and finds ambiguity before it becomes an operational problem.

Comment

Inspiration

Workplace policies often look clear when they are written, but real employees rarely follow the simplest possible scenario.

A policy may state that employees receive 20 days of annual leave per year, while leaving many practical questions unanswered. What happens when someone joins halfway through the year? Does the entitlement apply to part-time employees? Can unused leave be carried forward indefinitely? What happens when a request spans two calendar years?

These gaps are usually discovered only after an employee submits a request, a manager makes an inconsistent decision, or HR and Payroll are forced to resolve an exception manually.

As an engineering leader working closely with HR and payroll products, I kept coming back to a familiar software engineering idea: business logic should be tested before it reaches production.

That led to the central concept behind RuleTest:

A workplace policy is business logic. Employee situations are test cases.

RuleTest brings the discipline of software unit testing into policy design. Instead of asking whether a policy “sounds clear,” it tests whether the policy can consistently answer realistic employee scenarios.

5. What it does

What it does

RuleTest analyzes a workplace policy and turns it into a structured, testable system.

A user starts by pasting a policy or loading a sample document. RuleTest then uses GPT-5.6 to extract explicit business rules, identify undefined terms, and connect each rule to the exact supporting wording in the original policy.

The user can review, edit, or disable the extracted rules before continuing. This human review step ensures that the AI-generated interpretation remains transparent and controllable.

RuleTest then generates realistic employee scenarios designed to expose edge cases, including:

  • Employees joining partway through a year.
  • Part-time or probationary employees.
  • Requests that cross policy periods.
  • Unused balance carry-over.
  • Missing approval responses.
  • Conflicting eligibility conditions.
  • Termination or employment-status changes.

Each scenario is evaluated against the approved rules and classified as:

  • Pass: the policy provides a clear and consistent answer.
  • Ambiguous: more than one interpretation is possible.
  • Missing rule: the policy does not address the scenario.
  • Conflict: two rules lead to incompatible outcomes.
  • Risk: the policy provides an answer but may cause an operational or fairness issue.

The final report includes:

  • A Policy Readiness Score.
  • A breakdown of passed and failed scenarios.
  • Supporting policy quotes.
  • Relevant extracted rules.
  • A clear explanation for each result.
  • Suggested wording to resolve the issue.

RuleTest can also generate targeted policy amendments and show which failed scenarios each change would resolve.

It does not provide legal advice or regulatory certification. Its purpose is to test policy clarity, completeness, internal consistency, and operational readiness.

6. How we built it

How we built it

RuleTest was built as a structured AI workflow rather than a single chatbot prompt.

The application separates policy analysis into four distinct stages:

  1. Rule extraction GPT-5.6 converts the policy into explicit business rules, identifies undefined terms, and attaches exact source quotes to each rule.

  2. Scenario generation GPT-5.6 creates realistic employee scenarios based on the policy type, extracted rules, missing concepts, date boundaries, employee categories, and operational exceptions.

  3. Policy test execution Each scenario is evaluated strictly against the policy text and the user-approved rule set. Results are returned as structured data with a status, severity, explanation, supporting evidence, and confidence score.

  4. Policy amendment generation Failed tests are grouped into related issues, and GPT-5.6 proposes the smallest practical wording changes needed to make the policy clearer and more testable.

The frontend was built with:

  • React
  • TypeScript
  • Vite
  • Tailwind CSS
  • shadcn/ui

The backend uses:

  • Firebase Authentication
  • Firestore
  • Firebase Cloud Functions
  • OpenAI API
  • GPT-5.6
  • Zod for runtime schema validation

OpenAI requests are made through server-side Firebase functions so that API credentials are never exposed to the browser.

All AI responses are validated against strict schemas before being stored or displayed. This was important because RuleTest depends on relationships between rules, scenarios, results, evidence, and suggested amendments.

Codex was used throughout development to:

  • Create the initial application structure.
  • Design TypeScript and Zod schemas.
  • Implement Firebase Cloud Functions.
  • Build the multi-step React workflow.
  • Refactor prompts into separate AI operations.
  • Add validation and error handling.
  • Generate test fixtures and sample policies.
  • Debug integration and deployment issues.
  • Review the repository and improve documentation.

The project was developed primarily within Codex session:

[INSERT CODEX /feedback SESSION ID]

7. How GPT-5.6 is used

Tôi khuyên thêm section riêng này vào description hoặc README, vì ban tổ chức đặc biệt muốn thấy việc sử dụng GPT-5.6.

How we used GPT-5.6

GPT-5.6 powers the reasoning-heavy parts of RuleTest.

It is used to:

  • Convert natural-language policy text into structured business rules.
  • Distinguish explicit rules from implied assumptions.
  • Identify undefined terminology and missing conditions.
  • Generate diverse employee edge cases.
  • Evaluate whether a policy gives a clear answer to each scenario.
  • Trace results back to exact policy wording.
  • Detect ambiguity, missing rules, and contradictions.
  • Generate targeted policy amendments.
  • Explain results in language accessible to HR and operations teams.

The model is instructed not to introduce external legal requirements. It evaluates only the policy supplied by the user and clearly separates what the policy states from what the model believes may be missing.

Structured outputs and runtime validation help ensure that the results can be rendered as a reliable product workflow rather than unstructured AI text.

8. How Codex was used

How we used Codex

Codex acted as the primary development partner for RuleTest.

It helped turn the initial product concept into a working application by supporting both implementation and engineering decision-making.

Key areas where Codex accelerated development included:

  • Breaking the product into vertical slices.
  • Designing the data model for policies, rules, scenarios, results, and patches.
  • Generating strongly typed React components.
  • Implementing and refactoring Firebase Cloud Functions.
  • Creating reusable OpenAI service layers.
  • Adding Zod validation for model responses.
  • Building error states, loading states, and retry flows.
  • Creating sample data and edge-case fixtures.
  • Reviewing security boundaries between the client and backend.
  • Debugging build, type, and deployment errors.
  • Producing setup documentation and README instructions.

Codex was particularly useful for keeping the AI workflow structured. Instead of relying on one large prompt, the project was implemented as several composable operations with explicit schemas and validation boundaries.

The majority of the core functionality was built in the following Codex session:

[INSERT CODEX /feedback SESSION ID]

9. Challenges we ran into

Challenges we ran into

Making AI output reliable enough for a product workflow

The first challenge was that policy analysis naturally produces complex and variable output. A rule can reference several employee categories, multiple conditions, exceptions, and source clauses.

Unstructured text was not reliable enough for the application, so we introduced strict schemas for every AI operation. GPT-5.6 responses are validated before they enter the UI or database, and invalid responses produce recoverable errors rather than broken screens.

Preventing the model from inventing legal requirements

RuleTest is designed to evaluate the supplied policy, not replace a lawyer or compliance database.

The prompts explicitly instruct GPT-5.6 to avoid adding external legal rules and to distinguish between:

  • What the policy explicitly states.
  • What the policy implies.
  • What remains undefined.
  • What is entirely missing.

This boundary was essential to keeping the product useful and responsible.

Generating meaningful scenarios instead of generic examples

Early scenario generation could produce cases that were technically different but tested the same underlying issue.

We improved this by classifying scenarios into edge-case categories such as dates, employment types, exceptions, missing definitions, conflicting conditions, and boundary values. We also added deduplication and required each scenario to test a distinct policy question.

Preserving traceability

A finding such as “this policy is ambiguous” is not useful unless the user can see why.

We designed every result to connect back to:

  • The employee scenario.
  • The relevant rule IDs.
  • Exact quotes from the policy.
  • The reasoning behind the classification.
  • A proposed clarification.

This traceability required more engineering than simply displaying a model-generated answer, but it made the experience far more credible.

Balancing completeness with hackathon scope

Policy management can easily expand into legal research, regulatory compliance, HRIS integrations, approval workflows, and collaboration.

We deliberately limited the hackathon version to one focused loop:

Policy → rules → scenarios → tests → findings → fixes.

This allowed us to deliver a complete product experience rather than a collection of incomplete features.

10. Accomplishments that we’re proud of

Accomplishments that we're proud of

We are proud that RuleTest turns a familiar software engineering practice into a new workflow for HR and business operations.

The application does more than summarize a document. It converts policy language into a structured model, generates test cases, executes those tests, preserves evidence, and proposes changes that can be reviewed by a human.

Specific accomplishments include:

  • Building an end-to-end policy testing workflow.
  • Producing structured and schema-validated AI outputs.
  • Connecting every finding to exact source wording.
  • Allowing users to review AI-extracted rules before testing.
  • Generating realistic policy edge cases automatically.
  • Distinguishing ambiguity, missing rules, conflicts, and operational risks.
  • Creating targeted policy amendments linked to failed scenarios.
  • Delivering a demo that can be tested immediately with a built-in sample policy.
  • Completing a coherent product experience using GPT-5.6 and Codex within the Build Week timeframe.

The feature we are most proud of is the feedback loop between policy failures and suggested fixes. RuleTest does not stop at identifying a problem; it explains the smallest practical change that could resolve it.

11. What we learned

What we learned

The biggest lesson was that the quality of an AI product depends heavily on how the reasoning process is structured.

A single large prompt could produce an impressive-looking report, but it was difficult to validate, debug, or connect to an interactive workflow. Splitting the process into rule extraction, scenario generation, test evaluation, and amendment generation made the system easier to reason about and significantly more reliable.

We also learned that human review should not be treated as a fallback. In RuleTest, it is a core product feature. Allowing users to approve or modify extracted rules before running tests creates a stronger partnership between the model and the domain expert.

Another important lesson was the value of traceability. Users are more likely to trust an AI-generated finding when they can inspect the exact source quote, rule, scenario, and reasoning behind it.

Finally, we learned that policy writing and software development share more concepts than we initially expected. Both involve rules, conditions, exceptions, undefined states, conflicting behavior, and edge cases. Thinking of a policy as executable business logic opened up a new way to evaluate document quality.

12. What’s next for RuleTest

What's next for RuleTest

The next step is to expand RuleTest from a single-policy analyzer into a continuous policy quality platform.

Planned improvements include:

Policy regression testing

When a policy is edited, RuleTest will rerun the existing scenario suite and show which previously passing cases now fail.

Custom scenario libraries

Organizations will be able to maintain reusable scenario packs based on their workforce structure, employment types, approval processes, and historical exceptions.

Policy version comparison

Users will be able to compare two versions of a policy and see how the changes affect employee outcomes.

Cross-policy conflict detection

RuleTest will test whether separate policies, such as leave, flexible work, payroll, and termination policies, produce contradictory outcomes when applied together.

Collaboration and approval workflows

HR, Payroll, Legal, and Operations teams will be able to review findings, assign issues, approve policy changes, and record decisions.

HRIS and knowledge-base integrations

Future integrations may include Google Drive, Notion, SharePoint, HR platforms, and internal policy repositories.

Organization-specific policy models

RuleTest could learn an organization’s terminology, employee categories, approval structure, and policy conventions to generate more relevant scenarios.

Policy-to-system verification

A future version could compare written policy rules with the actual configuration of payroll or HR software to identify gaps between documented intent and implemented behavior.

Our long-term vision is for policy testing to become a standard step before any workplace rule is published, just as automated testing has become a standard step before software is deployed.

Built With

Share this project:

Updates