Lingji

Inspiration

Turning a CSV into meaningful sound normally requires both data-mapping and music/audio expertise. I built Lingji so a person can decide what the data should communicate while a browser agent handles precise mapping, arrangement, inspection, and verification inside the same live composition.

The flagship demo uses 64 exoplanets. It lets people hear patterns across temperature, radius, orbital period, distance, and stellar temperature without hiding how the sound was produced.

What it does

Lingji is a browser-based data-sonification studio that works both manually and through WebMCP.

A person can upload and inspect a CSV, map columns to musical properties, generate a deterministic score, play and edit it, and lock notes, tracks, or mappings. Clicking any note traces it back to its source row, raw value, normalization, rule, and musical result.

A browser agent can use nine semantic WebMCP tools to inspect the same live project, explain a selected note, respect human locks, modify unlocked mappings and arrangements, render a playback plan, and validate the source-to-sound chain.

How we built it

Lingji is a static React and TypeScript application built with Vite. Sound is synthesized locally with the Web Audio API; there is no backend and no AI audio-generation service.

The dataset, mapping, composition, and project-state engines are separate from the interface, so human controls and WebMCP tools call the same underlying functions. The page registers nine tools with document.modelContext.registerTool(...). Mutations are revision-aware, protected changes are rejected, locks survive regeneration, and validation checks generated events against their source data and rules.

Challenges we ran into

The hardest part was making collaboration safe without making it rigid. Regeneration must preserve deliberate human edits and locks while still giving the agent useful control over the remaining composition.

Provenance was another core challenge. A note is not useful for data exploration when its origin is hidden, so Lingji carries source row, raw measurement, normalization, and mapping rule through every generated event. We also preserve scientific field identifiers for agents while showing readable labels and units to people.

Accomplishments we're proud of

• Lingji remains fully usable without an agent.
• Nine working WebMCP tools expose non-trivial live project operations.
• Human creative constraints are enforceable locks, not suggestions.
• Every generated note is inspectable rather than an unexplained AI output.
• The 64-exoplanet demo runs entirely in the browser.
• Deterministic validation verifies the internal source-to-note chain after agent changes.

Screenshots

The 64-exoplanet composition

Measurements become explicit sound rules

A selected note explains itself

The agent respects the human's locked choice

Final validated score

What we learned

Human-agent collaboration needs more than giving an agent buttons to press. The agent needs semantic access to the artifact, and the human needs visible control over what may change.

Sonification also becomes more trustworthy when every sound can answer, "Which data produced this, and by what rule?" Lingji intentionally does not claim that validation proves the dataset is true or that the composition sounds good. Validation checks mapping consistency; the person judges meaning and musical quality.

What's next

Next steps include side-by-side sonification comparisons, more musical scales and synthesis options, richer dataset navigation, and user testing with educators, researchers, data communicators, and accessibility practitioners.

Built With

• React
• TypeScript
• Vite
• WebMCP
• Web Audio API
• Vitest
• CSS
• Codex

Built With

• codex
• css
• react
• typescript
• vite
• vitest
• web-audio-api
• webmcp