• 

  Boot splash

• 

  Shell screen

• 

  Block game

• 

  Roguelike game

• 

  Kotorun scroller game

• 

  Memo Application

• 

  Sokoban game

About the project

Inspiration

KotoOS started from a simple question:

What would a small, modern computer look like if its entire software stack were designed specifically for a tiny embedded device?

I wanted to build something inspired by compact personal computers and programmable handheld systems, but using modern embedded technologies instead of recreating an existing platform.

The target device is the ClockworkPi PicoCalc, a small keyboard-equipped computer built around the Raspberry Pi Pico family. Its hardware is extremely constrained compared with a PC: memory is limited, there is no conventional operating system underneath, and every allocation, frame, peripheral, and byte transferred to the display matters.

Rather than treating those constraints only as limitations, I decided to make them part of the design.

That led to KotoOS: a small application platform written primarily in Rust, with its own virtual machine, application bytecode, programming language, SDK, graphics and audio systems, Japanese input, GUI framework, simulator, and packaged applications.

During OpenAI Build Week, the project also became an experiment in another question:

How far can GPT-5.6 and Codex collaborate with a single developer across an entire embedded computing stack?

Not just at the application layer, but across firmware, hardware drivers, a compiler, a virtual machine, UI systems, performance optimization, networking, and real microcontroller hardware.

KotoOS was publicly released as version 0.1 on July 13, the first day of Build Week.

At that point, it supported the RP2040.

Two days later, version 0.2 added RP2350A support while preserving RP2040 compatibility, along with audio streaming.

During the v0.3 development milestone, KotoOS gained KotoUI, multilingual application support, and Wi-Fi networking. By the final stretch of Build Week, real RP2040 hardware could connect to Wi-Fi, synchronize time using SNTP, and establish TLS 1.3 communication.

The result was an unusually intense week of AI-assisted systems engineering on real constrained hardware.

---

What it does

KotoOS turns the PicoCalc into a programmable handheld computer with a growing collection of applications and games.

The system includes:

• A custom application runtime and Koto virtual machine
• The KotoOS application language
• A bytecode-based application package format
• A retained-mode GUI framework called KotoUI
• Keyboard-first focus and interaction
• Japanese text input with SKK
• SD card storage
• Audio streaming and music playback
• Graphics, tile rendering, compositing, and streamed full-color images
• Application assets and localization
• Simulator support for desktop development
• RP2040 and RP2350A hardware support
• Wi-Fi networking on supported hardware
• SNTP time synchronization
• Experimental TLS 1.3 communication on RP2040
• A VS Code-based development workflow
• Multiple bundled applications and games

One of those applications is Kotorogue, a roguelike game that runs as a real packaged KotoOS application directly on the physical PicoCalc.

Koto applications run inside a bounded virtual machine rather than directly on the hardware.

The VM deliberately avoids many features normally associated with desktop scripting languages. Applications have fixed resource limits, deterministic instruction budgets, bounded heap usage, and no general-purpose runtime allocator.

The result is an unusual programming model: applications are high-level enough to use enums, structs, methods, retained UI components, localization resources, and SDK builders, while remaining predictable enough to run on a microcontroller.

For example, Koto supports statically allocated typed records and fixed buffer fields:

struct NoteStorage {
    raw: buf[asset_len(
        "locales/en-US.txt",
        "locales/ja-JP.txt",
        "locales/qps-ploc.txt"
    )],

    resource: buf[
        ui_text_resource_capacity(
            NOTE_RESOURCE_LINES,
            NOTE_RESOURCE_RAW_BYTES
        )
    ],

    doc: buf[64],
    event: buf[96],
}

static note_app: NoteStorage = {};

These records look somewhat Rust-like, but their semantics are designed specifically for the Koto VM.

There is no new, garbage collector, or dynamic object allocator. Storage layout is derived at compile time, and a struct reference is effectively a typed base address into application-lifetime memory.

This gives Koto a programming model influenced by embedded Rust's resource discipline without attempting to reproduce Rust's full ownership system.

---

How we built it

KotoOS is built primarily in Rust using a no_std embedded architecture.

The firmware uses Embassy for asynchronous embedded execution and runs on Raspberry Pi RP2040 and RP2350A-class microcontrollers.

The system is split into reusable subsystems including:

• koto-vm — the application virtual machine
• koto-gfx — graphics and compositing
• koto-audio — audio playback
• koto-ime — Japanese input
• koto-input — keyboard and input handling
• koto-storage — storage access
• koto-runtime — application runtime services

Applications are compiled into Koto bytecode and packaged together with their assets.

A desktop simulator shares as much of the same runtime and application stack as possible, allowing applications to be developed and tested without flashing hardware for every change.

One of the main architectural goals is to keep simulator and physical device behavior equivalent. If an application runs in the simulator, the same application package should run on the device with the same semantics.

KotoUI follows a retained-mode, keyboard-first architecture.

Applications submit semantic UI descriptions and updates, while the host owns rendering, focus management, validation, and damage tracking.

The component model is intentionally small and bounded. It is designed around labels, buttons, checkboxes, lists, single-line text fields, panels, and modal dialogs rather than a general-purpose desktop widget tree.

KotoUI keeps component state allocation-free and compatible with no_std. It uses caller-owned data, stable widget identities, bounded focus traversal, and explicit dirty rectangles so idle frames do not repaint unchanged content.

Instead of asking applications to manipulate raw wire-format packets directly, the SDK provides typed builders such as:

UiMountBuilder
UiUpdateBuilder
TextResource
UiListRowsBuilder

The compiler also performs compile-time resource calculations.

For example:

const RESOURCE_BYTES =
    asset_len(
        "locales/en-US.txt",
        "locales/ja-JP.txt",
        "locales/qps-ploc.txt"
    );

and:

buf update[
    ui_update_capacity(
        10,
        GALLERY_APPLY_TEXT_BYTES + GALLERY_LIST_BYTES
    )
];

These values are folded at compile time.

When a translation asset grows, the next build can automatically recalculate the required storage instead of relying on manually maintained byte constants.

The overall design tries to move mechanical resource arithmetic away from application authors and into the compiler and SDK, while keeping meaningful application limits explicit.

How GPT-5.6 and Codex were used

Throughout Build Week, I used GPT-5.6 and Codex as engineering partners across almost the entire stack.

GPT-5.6 was used extensively for:

• Architectural reasoning
• Debugging strategies
• API and language design
• Performance analysis
• Design reviews
• Planning implementation milestones
• Reasoning about memory and resource constraints

Codex worked directly across the repository and was used for:

• Inspecting dependencies and existing architecture
• Implementing changes across multiple modules
• Refactoring reusable no_std subsystems
• Extending the compiler and virtual machine
• Implementing and reviewing KotoUI and SDK APIs
• Running tests and tracing regressions
• Investigating performance bottlenecks
• Porting KotoOS from RP2040 to RP2350A
• Working on graphics, audio, PSRAM, storage, and networking
• Writing regression tests, validation tools, implementation plans, and architecture documentation

The development process was highly iterative.

My typical loop was:

design → implement → build → test in the simulator → test on real hardware → measure → analyze → refine

For embedded work, generated or modified code could never be treated as correct simply because it compiled.

Real hardware remained the source of truth.

That was especially important for PSRAM performance, display transfers, audio scheduling, SD card timing, Wi-Fi behavior, memory pressure, TLS communication, and support for new hardware.

Code that looked reasonable in isolation was not always correct, stable, or fast enough on the physical device.

AI significantly accelerated the iteration loop, but measurements and behavior from the actual PicoCalc ultimately determined what stayed.

---

Challenges we ran into

The biggest challenge was working within extremely tight resource limits without turning application development into low-level memory bookkeeping.

On the RP2040 profile, minimum observed free SRAM reached only about 4.4 KiB out of 264 KiB while the system was still running the VM, graphics, Japanese input, audio, storage, and application services.

The VM itself also has a small bounded local frame, and deeply inlined SDK code can easily consume too many slots.

At one point, applications such as File Note used a single large buffer with manually maintained offsets:

raw
resource
document
state
event
mount packet

This was efficient, but fragile.

Adding or resizing one region required manually recalculating every following offset.

The solution was to extend Koto with statically allocated records and fixed buffer fields.

Now the compiler owns the layout while functions still pass only one typed base reference, preserving the original slot efficiency.

Another challenge was balancing abstraction with predictability.

A conventional high-level language might solve many problems with heap allocation, dynamic containers, garbage collection, or runtime reflection.

Those approaches would make resource usage much harder to predict on a microcontroller.

Instead, KotoOS repeatedly uses compile-time information:

• Fixed buffer capacities
• Compile-time asset sizes
• Statically derived storage layouts
• Bounded UI packet sizes
• Deterministic VM fuel budgets

This occasionally makes the language more unusual, but it keeps runtime behavior understandable.

PSRAM was another major challenge.

Early transfer paths achieved only about 1.2 MB/s, and later revisions initially plateaued at roughly 4 MB/s.

Reaching practical performance required extensive experimentation with PIO, DMA, transfer sizes, clock settings, protocol timing, and buffering.

Reference implementations helped demonstrate what the hardware was capable of, but adapting those ideas into a Rust and Embassy-based architecture was not a simple code translation.

PIO state machines, DMA lifetimes, GPIO ownership, asynchronous execution, buffers, and hardware timing all had to work together safely and reliably.

Display performance was also a constant concern.

On a 320 × 320 display, seemingly small rendering decisions can dominate the frame budget.

We spent significant time measuring rasterization, compositing, dirty rectangles, PSRAM access, RGB565-to-RGB666 conversion, and SPI transfer costs.

Optimizing the system required repeatedly measuring where time was actually being spent rather than assuming the bottleneck.

Networking introduced a new set of constraints during the final stretch of Build Week.

Bringing Wi-Fi to the RP2040 profile exposed boot issues, memory pressure, stack usage, and interactions between subsystems that had previously been independent.

After working through those problems, KotoOS reached Wi-Fi connectivity and SNTP time synchronization on real hardware.

The next challenge was secure communication.

TLS 1.3 significantly increased memory and stack pressure on an already constrained RP2040 system. Getting a successful encrypted connection required further memory reduction, runtime investigation, and real-device validation.

The result was a successful TLS 1.3 connection from a physical RP2040-based PicoCalc — achieved only shortly before the final Build Week submission.

Localization exposed another unexpected class of problems.

Initially, applications manually tracked resource buffer sizes and line offsets.

Over time this evolved into compile-time asset inspection and SDK-owned resource representations.

The compiler can now derive information such as:

• Packaged asset byte size
• Common text line counts
• Maximum localized line ranges
• UI resource storage capacities

This removed a large class of hand-maintained magic numbers from applications.

---

Accomplishments that we're proud of

One of the accomplishments I am most proud of is that KotoOS has grown beyond a firmware experiment into a small but coherent application platform.

It now has:

• Its own VM
• Its own bytecode
• Its own application language
• Its own SDK
• Its own GUI framework
• Its own application packaging model
• A simulator
• Real hardware support
• Japanese input
• Localization
• Audio streaming
• Networking
• Games and productivity applications

I am also proud of how quickly the platform evolved during Build Week.

KotoOS version 0.1 was publicly released on July 13 with RP2040 support.

Two days later, version 0.2 added RP2350A support while preserving compatibility with the original RP2040 target, and introduced audio streaming.

During the v0.3 development milestone, the project added KotoUI, multilingual application support, and Wi-Fi connectivity.

By the end of the week, KotoOS could connect an RP2040-based PicoCalc to Wi-Fi, synchronize time through SNTP, and successfully establish a TLS 1.3 connection.

These changes crossed very different layers of the system: hardware support, firmware, memory management, networking, the application runtime, UI, localization, and developer tooling.

Another accomplishment is that many improvements made the system easier to use without simply adding more runtime machinery.

For example:

app + NOTE_MOUNT

could become:

app.mount

without introducing a runtime object.

Similarly, a manually calculated buffer size such as:

buf rows_blob[68];

can now be expressed using SDK-owned compile-time rules.

The implementation becomes easier to understand while the generated runtime representation remains small and deterministic.

Porting KotoOS from its original RP2040 target to RP2350A while preserving RP2040 compatibility was another important milestone.

The system can ship separate firmware images for multiple hardware profiles while using the same application packages and SD card content.

An especially meaningful moment came when someone outside the project successfully ran KotoSim on macOS and started creating their own experiments, including a Mexican-hat surface renderer and an oneko-style interactive demo.

That was the first time KotoOS really felt like a platform that other people could play with, rather than only a system I had built for myself.

---

What we learned

The biggest lesson from building KotoOS is that higher-level abstractions do not necessarily require higher runtime cost.

In embedded systems, the key question is often when complexity is paid.

If the compiler can calculate something once, the device should not calculate it repeatedly.

If a resource can be bounded at build time, the runtime does not need a dynamic allocator.

If a storage layout can be derived from a type declaration, the application should not maintain offsets manually.

This led to a design philosophy that appears throughout KotoOS:

Move complexity toward build time, keep runtime behavior small and predictable.

I also learned that good abstractions often emerge only after implementing the low-level version first.

KotoUI initially exposed much more packet-level detail to applications.

Only after building real applications such as Gallery and File Note did it become clear which patterns belonged in the SDK.

The same happened with Koto itself.

Features such as struct, static, impl, fixed buffer fields, and compile-time capacity helpers were not added because they looked nice as language features.

They were added because real applications repeatedly encountered specific limitations.

This has given Koto an unusual character.

Its syntax is influenced by Rust, but its semantics are optimized for a small deterministic VM.

In a sense, Koto takes some ideas associated with embedded Rust — explicit resource ownership, static lifetimes, bounded memory, and predictable execution — and translates them into a much smaller application language.

Build Week also taught me something about AI-assisted engineering.

Codex was most useful not simply as a code generator, but as a repository-level engineering partner capable of tracing dependencies, implementing multi-file changes, running tests, and iterating across boundaries.

GPT-5.6 was especially valuable when reasoning about architecture, constraints, debugging strategies, and tradeoffs before implementation.

But the experience also reinforced an important limitation:

In embedded systems, the model cannot replace the hardware.

The final answer always came from compiling the firmware, flashing the device, measuring what actually happened, and feeding those results back into the next iteration.

The most productive workflow was therefore not:

prompt → generated code → finished

It was:

reason → implement → test → observe → measure → revise

AI made that loop dramatically faster, but real-hardware validation remained essential.

---

What's next for KotoOS

The immediate priority after Build Week is stabilization.

The v0.3 development cycle introduced major new capabilities very quickly, including KotoUI, localization improvements, Wi-Fi networking, SNTP, and experimental TLS 1.3 communication.

That rapid progress also exposed regressions and areas that need more debugging.

The next steps are therefore to stabilize these features before treating v0.3 as a fully mature release.

Planned work includes:

• Stabilizing Wi-Fi and networking support
• Refining TLS memory and stack usage
• Adding application-facing network APIs
• Exploring HTTP-based services such as weather clients
• Exploring MQTT telemetry and remote dashboards
• Improving the Koto language and compiler
• Finishing KotoUI integration across more applications
• Improving KotoUI developer ergonomics and documentation
• Better on-device application development workflows
• Exploring on-device Koto compilation
• More complete audio and music tooling
• A visual novel engine
• Larger game demonstrations, including RPG-style applications
• Better SDK documentation and examples
• Continued performance and memory optimization on real hardware

The networking work opens an especially interesting direction.

A PicoCalc running KotoOS could become a small portable viewer for remote data: IoT sensors, servers, home automation systems, or industrial telemetry.

The first step was simply getting the device connected.

Then came SNTP.

Then TLS 1.3.

Future work can build higher-level protocols and applications on top of that foundation.

I also want to keep exploring the boundary between a scripting language and a statically bounded embedded runtime.

KotoOS is intentionally small, but the long-term question behind the project remains ambitious:

Can a microcontroller-sized computer have a developer experience that feels expressive and modern without giving up deterministic resource usage?

And after Build Week, there is another question I want to keep exploring:

How far can AI-assisted engineering go when it is paired with real hardware, measurable constraints, and a development process where the physical device remains the source of truth?

KotoOS is my attempt to find out.

Built With

• audio
• bytecodevm
• developertools
• dma
• embassy
• embeddedsystem
• japaneseime
• javascript
• koto
• no-std
• opensource
• picocalc
• pio
• psram
• python
• raspberrypipico
• rp2040
• rp2350
• rust
• skk
• spi