top view without cap
Label Refill
label coming out of PocketDots
Camera Scanning UI
Voice Input UI

PocketDots

Inspiration

It started with a number that stopped us cold.

There are 253 million people living with visual impairment worldwide. Braille is their written language, their access to education, employment, contracts, prescriptions, and independent living. And yet, fewer than 10% of blind individuals in the United States are braille literate today. We assumed the reason was awareness, teaching, or resources. Then we found the real bottleneck: the machines that produce braille cost between $2,000 and $10,000, and they are almost exclusively found in institutions.

That means the average blind student, individual, or family cannot own one. Cannot use one at home. Cannot print a page of braille without asking an institution for access, or a sighted person for help.

The more we sat with that reality, the harder it was to look away. Braille literacy is not a niche accommodation; it is the difference between independence and dependence, between dignity and having to ask someone else to read your own mail. We felt strongly that this was a solvable problem and that nobody had solved it yet because nobody had truly designed for the person, rather than the institution.

That became our brief: build a braille printer for the individual. Small enough to live on a desk. Smart enough to operate alone. And accessible enough that independence is no longer a privilege reserved for those with institutional access.

What It Does

Jac-in-the-box is a portable, AI-powered braille printer measuring just 15 x 11 x 8 cm smaller than a lunchbox, light enough to carry anywhere. It accepts any text input and produces embossed braille output in seconds, with a fully voice-guided experience so a visually impaired user never needs to look at a screen at any point.

A user types or speaks any text. It could be a paragraph, an article, a prescription, a homework page, and within seconds, they hold printed braille they produced themselves, independently, without needing anyone's help. It is the first braille printer built from the ground up with the independence and dignity of its user in mind.

How We Built It

The Mechanical Pipeline

The core of Jac-in-the-box is a precision embossing system, custom-designed and firmware-controlled. The process follows four stages:

Paper feed: A paper roll is inserted and advanced by a paper roller into the embossing zone.
Column selection: A servo-driven daisy wheel with the 7 pre-embossed dot combinations rotates to the correct angle to line up the first column of the braille.
Embossing: A solenoid fires upward, pressing the die into the press (pre-embossed dot on the daisy wheel)to create raised dots on the paper in between.
Ejection: A stepper motor attached to the roller moves the paper 2.5mm to emboss the second column of the braille. Then continues for the entire label before pushing the paper out the exit.

Since each braille cell consists of a 3 times 2 dot matrix, where each dot is 2.5 mm apart from each other. A stepper motor provides valuable non-cumulative angular accuracy tolerance to allow precise embossing on the paper.

The Intelligence Layer

The ESP32-S3 DevKitC-1 orchestrates the entire pipeline, including driving actuators, managing wireless connectivity, and running on-device AI inference simultaneously.

A state of the art AMD GPU is used for our transcription, Vision and Large Language models through cloud computing to reduce overhead, while edge TTS models are used to simulate a friendly helper to assist our users and simplify their app experience. Our pipeline accommodates both voice and camera input. Openai whisper serves as the transcription engine, while Yolo serves as our visual model. The outputs of these two tracks are then further refined through our LLM model for an accurate and meaningful final label.

Finally, the system compiles a structured JSON with an encryption code and instructions that specify exact cell positions, line breaks, and column counts, all validated against our hardware's physical constraints. The ESP32-S3 consumes this JSON directly, with zero intermediate processing.

Challenges We Ran Into

Getting the mechanics to cooperate

The hardest physical challenge was timing. The solenoid, servo, and stepper motor each operate on different response curves, and synchronizing them precisely enough to produce clean, consistently-spaced braille dots was far more unforgiving than we expected. A mistimed solenoid fire by even a few milliseconds produces a dot in the wrong position. We spent significant time tuning the firmware control loop and introducing precision delays to bring the mechanical pipeline into reliable synchronisation.

Problem-solving with limited hardware resources

Due to limitations in available hardware components, many of our initial mechanical solutions were not feasible as originally envisioned. Rather than compromising the concept, we adapted by developing alternative mechanical and technical approaches that still realized the core functionality of the system.

For example, a conventional design would require three individual solenoids to generate braille patterns. Given our limited access to these components, we engineered a workaround by using a single servo motor to mechanically rotate and actuate pins, achieving a similar output through a different mechanism. Similarly, when addressing the paper feed system, we lacked appropriately sized high-friction rollers. The only available rubber wheels were oversized at 2.5 inches, so we improvised by wrapping rubber bands around existing components to increase grip, creating an effective low-fidelity feeding solution.

In terms of accessibility, our original goal was to integrate hardware features such as a camera, microphone, speaker, and haptic feedback for reminders. However, without access to these components, we shifted this functionality into a companion app. This not only compensated for hardware constraints but also allowed us to design a more streamlined and accessible interface for both visually impaired users and a broader audience.

Finally, component availability also impacted our electrical design. A key challenge was the lack of logic-level MOSFETs, which are typically essential for driving solenoids. Through rapid prototyping, testing, and consulting available resources, we identified alternative approaches that allowed us to successfully operate the system despite these constraints.

Integrating a full AI pipeline into a lunchbox

Managing Gemini, AMD object detection, and ElevenLabs within a single ESP32-S3 meant carefully balancing memory, latency, and communication overhead. Each API call needed to complete within a time budget that kept the experience feeling responsive. We architected the pipeline so that Gemini translation runs once per job, AMD inference runs continuously but asynchronously, and ElevenLabs audio is pre-buffered, ensuring no single component blocks the others.

Fitting everything into 15 x 11 x 8 cm

The form factor was a constraint that touched every decision. Every component had to earn its place as we were aiming for precision, but compactness and weight. We went through multiple iterations of the internal layout, balancing the paper roller geometry, the solenoid throw distance, the daisy wheel radius, and the PCB footprint against the hard outer dimensions. Every millimetre was negotiated. Also, due to tie constraints, we will have to print the parts before soldering, so we can't be sure of the minimum amount of space we need, which could to a great extent, shrink the product's dimension by more.

Design constraints from the target audience

Since we are working with a visually impaired audience, we were really trying to incorporate only the technology that would be most useful and friendly to the group. That said, we had to get rid of a lot of some more delightful or advanced technology for the physical constraints that we have to take into consideration. All the parts and movements are intentional in a way that would support their actions and behaviors more. For example, for them to independently use this machine, how will they be able to recognize the item, select the word label, or even feed the paper?

Accomplishments That We're Proud Of

We are proud of all of it, but if we're honest, the moment that meant the most was simpler than any of the technology.

It was the first time we held a printed sheet of letter paper with three pins of braille, the moment when refound rubber bands have great grips to feed the paper, the moment that the none active solenoid was punched for the first time, where a lack of solenoid returns with a more creative solution, the moment where trays connect, when gears actually met. It is those quick thinking and innovative solutions that got us from a stage where we only had an ESP32 reserved in the hardware list.

What We Learned

Teamwork and work distribution are key. Allocating each of their strengths, but also not only limit to what you know, but what you are willing to learn
Never get restricted by the "correct answer", which may limit the creativity in creating new modules.
Importance of time management throughout different stages
Challenging situations bring out the best in us

What's Next for Jac-in-the-box

Jac-in-the-box works, but we know it's not finished.

The truth is that what we built this weekend is a proof of concept. But there is meaningful engineering work ahead before this is ready to be in someone's hands as a reliable daily tool. The form factor can get smaller still. The mechanical tolerances need tightening. The firmware needs hardening for real-world edge cases that a 36-hour hackathon doesn't surface.

Our immediate next step is refinement: iterating on the hardware design to further reduce the footprint, improving the reliability of the embossing pipeline, and stress-testing the AI integration across a wider range of inputs and conditions.

From there, our ambition is to bring it to market so that any visually impaired individual anywhere in the world can access it. But we want to do that responsibly. A device built around independence and dignity has to earn that trust through reliability first.