Sprout

Sprout — Your work, within reach Sprout is a voice companion that helps blind and low-vision people create, understand, and revise digital work through conversation.

Inspiration An idea might begin with a few words, but turning it into a presentation or understanding a spreadsheet often requires navigating a visual interface.

We wanted to make that process more accessible. A person should be able to describe what they want, hear what was created, question the result, and ask for changes.

The World Health Organization estimates that at least 2.2 billion people have a near or distance vision impairment. That inspired us to explore how conversational tools could give people more control over everyday computer tasks.

Sprout grew from that idea: helping users turn their intentions into work they can understand and share.

What it does Sprout lives in a floating Windows companion. Users can speak naturally, ask follow-up questions, and interrupt a response when they want to change direction.

Its main showcase is creating and reviewing editable PowerPoint presentations. Users can request a presentation, include their own pictures, navigate to a slide, hear its content, make supported revisions, and undo changes. Explicit saving and same-laptop resume help them return to their work.

Sprout also includes workflows for:

Spreadsheet analysis: Compare supported datasets, inspect source records, and export Excel workbooks and presentation reports. Personal finance: Explore spending changes, hypothetical budget adjustments, and fixed-rate payment estimates. Electricity bills: Review extracted bill information, correct fields, and understand changes in usage and charges. Computer assistance: Research online, open supported applications, locate named folders, describe captured screens, and perform bounded navigation. Coding: Generate and run simple Python calculations and open generated code in VS Code. Across these tasks, Sprout gives users a way to ask, inspect, correct, and continue.

How we built it We built Sprout’s desktop interface with Electron, React, and TypeScript, supported by a local Fastify backend.

Hugging Face hosted Qwen models interpret conversational requests, describe images, and generate structured content. Typed validation checks proposed actions, while deterministic tools handle file operations and numerical calculations.

ElevenLabs Scribe provides streaming transcription, and ElevenLabs speech synthesis delivers spoken responses. Cancellation checks connect conversation to execution so interruptions can stop outdated work and playback.

PptxGenJS creates editable PowerPoint content, while ExcelJS produces workbooks with formulas and supporting records. Databricks verifies supported analyses against source data. Neon stores explicitly saved project state and revision history.

Original presentation images and folder bindings remain local, and resume verifies the original sources before restoring a project.

Challenges we ran into The biggest challenge was keeping conversation, files, and application state consistent.

“Make it shorter” needs a clear slide target. An interruption can change a task already underway. A navigation request needs confirmation that PowerPoint selected the correct slide.

We also discovered that polished AI output can lose important meaning. A revision might fit the slide while dropping a qualification. That led us to evaluate content fidelity alongside formatting and layout.

We built stable slide identities, guarded file updates, recovery copies, cancellation handling, and verified readback to address these problems. Revision reliability remains an active development priority.

Accomplishments we’re proud of We connected spoken requests to real, editable files and verified key workflows with actual providers, cloud storage, and installed PowerPoint.

Our latest integrated source suite passed 1,251 automated unit tests. Another 26 scoped browser cases passed with mocked providers and native interfaces. We visually inspected seven mixed-scene exported slides with no visible clipping or overlap.

Actual save-and-resume checks preserved presentation content and undo history, and export checks confirmed that original image bytes survived generation.

In a fictional revenue demonstration, changing the comparison to continuously operating locations changed reported growth from 52.2% to 11.9%. That showed why giving users a way to question an answer matters.

These results measure engineering progress. We have not yet measured time savings or independence gains with blind and low-vision participants.

What we learned The follow-up is central to useful assistance.

Users need to understand what happened, inspect the evidence, and recover when something goes wrong. Requests such as “explain that,” “change this part,” and “undo that” shaped how we built Sprout.

We also learned that passing tests and receiving valid model output are only parts of verification. The final artifact must reflect the user’s intent, and the assistant’s feedback must accurately describe what succeeded.

What’s next for Sprout Our immediate priorities are improving revision fidelity and completing release checks and physical voice rehearsals.

Our development build includes branching diagrams, keyboard diagram exploration, richer spatial descriptions, and integrated verified-chart workflows. These additions are awaiting release acceptance.

Next, we want to work with blind and low-vision participants and measure independent task completion, completion time, correction success, and requests for assistance.

Their feedback will guide Sprout’s growth to