About the Project We set out to explore a simple question: What if writing and drawing did not need a screen at all?
Most learning and creative tools today still force people to switch between paper, keyboards, tablets, and software. This breaks flow and makes natural expression feel mechanical. Our project was inspired by the idea that handwriting and hand gestures are already powerful input methods — they just need better digital bridges.
So we built a dual‑device system:
a Smart Pen for precise writing and drawing on physical surfaces
a Wearable Holographic Motion‑Tracking Pin that captures mid‑air gestures
Together, they allow users to write, sketch, and communicate ideas using both paper and free‑space movement, converting everything into digital and holographic data in real time.
What We Built We designed two prototype‑ready devices:
- Smart Pen The Smart Pen looks and feels like a regular pen, but internally it contains:
an IMU (accelerometer + gyroscope) to track motion
an optical sensor near the tip to detect surface movement
a pressure sensor for stroke intensity
a microcontroller and battery to process and transmit data
This allows the pen to convert physical strokes into accurate digital handwriting and drawings.
- Holographic Motion‑Tracking Pin The wearable pin clips or magnetically attaches to clothing. It contains:
a depth / IR motion sensor to track hand gestures
a microcontroller and battery
a status LED for feedback
This device captures mid‑air gestures for drawing, writing, and command input, making it possible to interact without touching a surface.
How It Works The system is built around a shared hardware–software pipeline:
Smart Pen Flow Pen motion + pressure → IMU + optical + pressure sensors → Microcontroller → Bluetooth → Digital strokes in the app Holographic Pin Flow Hand gestures → Depth / motion sensor → Microcontroller → Gesture recognition → Commands or holographic strokes Both devices stream data wirelessly to a software layer that:
reconstructs handwriting and drawings
cleans strokes using AI
supports real‑time transcription and multilingual output
enables holographic or AR‑based visualization
What We Learned This project taught us how hardware and software must be designed together. A beautiful device is not enough — the sensors, data flow, and user interaction must all support each other. We also learned how small physical details (sensor placement, internal layout, power constraints) strongly affect digital accuracy.
Designing two devices to work as one system helped us think beyond a single product and toward an input ecosystem.
Challenges We Faced Designing compact internal layouts for sensors, batteries, and PCBs
Making the Smart Pen feel realistic while still being digitally powerful
Ensuring the Holographic Pin could be worn comfortably and track gestures reliably
Balancing technical feasibility with hackathon time and cost constraints
We solved this by using off‑the‑shelf components, modular internal design, and a clear separation between sensing, processing, and software layers.
Why This Matters Our system makes digital creation more natural, accessible, and inclusive. It benefits:
students taking notes
artists and designers
engineers sketching ideas
people with motor or accessibility needs
By removing the dependency on screens and keyboards, this platform opens a new way to think, learn, and create.
Log in or sign up for Devpost to join the conversation.