GenUI

Turn model-authored JSON into validated, native SwiftUI—and generated controls into the next step of a real conversation.

Comment

Inspiration

Generative UI is compelling on the web and projects like json-render are an incredible start, but native apps have a different bar. A generated interface still needs to participate in SwiftUI layout, accessibility, theming, state, validation, and the app's existing interaction model. I wanted a way for a model to describe useful interfaces without generating executable Swift code or dropping a web view into an otherwise native product.

GenUI is that bridge: a Swift library that turns model-authored JSON into validated, adaptive SwiftUI.

This is a follow-up to the first version I built during a previous OpenAI hackathon. For Build Week, I turned that prototype into a broader, tested library and connected it to a real agent chat workflow.

GenUI's document structure is based on the json-render catalog and specification format, adapted for native SwiftUI, local validation, and host-controlled actions and resources.

What it does

GenUI provides a bounded document format and 30 native components covering layout, text, forms, controls, tables, charts, images, and feedback states. It supports writable state, conditional visibility, repeated content, application-defined components, and host-owned actions and resources.

The example app shows GenUI inside a real agent conversation:

  1. The conversational model decides that UI would help and calls one generate_ui(description) tool.
  2. A specialist produces a candidate GenUI document.
  3. The app decodes and validates it locally.
  4. Structured diagnostics allow at most two repair attempts.
  5. The host receives one canonical artifact and renders it as native SwiftUI.

Display-only results stay inside the assistant turn. Interactive results move into the existing composer, where native controls update the next message and the user explicitly submits it. The generated UI becomes part of the conversation instead of a detached demo screen.

Why it matters

Agent interfaces often force a choice between rigid prebuilt screens and effectively unbounded generated code. GenUI is a middle layer for native app developers: models can choose and configure useful UI, while the host keeps control of the component catalog, actions, resources, sensitive data, and final presentation.

How I built it

The core is a Swift 6.3 package for iOS 17 and macOS 14. The wire model, decoder, validation rules, state runtime, component registry, renderer, themes, and host-action boundary are separate, testable layers. The catalog is also executable documentation: the same component definitions drive model guidance, validation, the gallery, and tests.

Wavelike provides the model and chat integration. Bun and TypeScript power a guarded compiler-evaluation and corpus-generation pipeline.

How I used Codex and GPT-5.6

This was a sustained Codex build, not a one-shot generation exercise. Codex helped develop the protocol, renderer, component catalog, forms and state systems, theming, example app, compiler workflow, and evaluation harness. It also ran the Apple development loop: package and Xcode tests, Simulator launches, screenshot and accessibility inspection, focused fixes, and phone and tablet verification. The work landed incrementally across 114 commits.

GPT-5.6 generated and repaired model-facing UI documents for the compiler corpus. GPT-5.6 Sol also reviewed guarded checkpoints for semantic shortcuts that deterministic validators initially missed. Those findings became mechanical acceptance gates and regression tests.

I published the resulting GenUI Compiler Corpus under CC BY 4.0: 5,000 accepted synthetic examples, split into 4,000 description-to-UI generation examples and 1,000 diagnostic-guided repair examples. Fine-tuning is the next step; the current demo does not depend on a fine-tuned model.

Challenges

The hard part was not drawing a card from JSON. It was defining a safe contract that remains useful once UI has state and behavior.

Repeated content needs stable identity. Inputs need type-correct writable paths. Forms need deterministic validation. Secure values must not leak into generated actions. Resource loading must remain host-controlled, and malformed documents must fail locally without taking down valid siblings.

Visual testing also exposed real implementation problems, including adaptive table breakpoints, invisible skeleton states, and a device-sized debug stack overflow. Each fix was narrowed to its runtime contract and locked down with focused tests or snapshots.

Verification

  • 211/211 Swift package tests passing
  • 82/82 compiler and data-generation tests passing
  • 22/22 focused iOS compiler and chat tests passing
  • 25 deterministic snapshot baselines
  • 5,000/5,000 corpus records accepted with final verification passing

What I learned and what's next

A useful model-facing format must make correct behavior easier to express than decorative imitation. It also needs diagnostics precise enough for either deterministic code or a bounded repair step to act on.

Next I want to version the protocol, make the example integration easier for other developers to test, and fine-tune smaller specialist models against the existing evaluation harness. The app-facing boundary stays the same even as the specialist moves between hosted and on-device execution.

Resources

Built With

Share this project:

Updates