POGOS - OptimFrame

main screen

Inspiration

In medical deserts and humanitarian settings, corrective lenses exist (donations, end-of-line stock, old glasses), but opticians and frames don't. A lens is only useful if a frame fits its exact shape, and the left and right lenses of a prescription may not even match. Opticians trace a lens with an expensive tracer. We wanted to replace that tracer with the phone in your pocket and a 3D printer.

What it does

OptiFrame is a mobile web app (no install, no account) that goes from a photo to a printable frame:

Scan: lay a lens next to a bank card and photograph it, live with guidance or from a photo. The card (85.60 × 53.98 mm) gives the scale and corrects the perspective.
Measure: the app traces the lens outline and reports the boxing width A, height B, perimeter and lens shape (round, oval or rectangular). Several lenses in one photo? Tap the right one. Outline not perfect? Fix it with a touch editor (drag points, magnifier loupe, pinch to zoom). The step-by-step images (card found, rectified top view, outline) are shown for every scan.
Design: choose among 19 frame styles, including 8 community designs adapted from openly licensed Printables models, shown only if they suit your lens shape. Every style follows the same rule: the inner opening is your exact scanned outline plus a constant clearance, and only the outside takes the style. Edit the arms and the nose bridge, pick a colour and a finish (glossy, satin, matte, metallic, crystal).
Try on: a live AR try-on runs on the device, with face tracking, lighting estimated from the camera image, a soft shadow on the face and the arms fitted to your head width.
Export: a seven-part kit, all 3D printed in one filament: front frame (with a V-groove that holds each lens, nose pad arms, barrel hinges and rivets), two arms, two nose pads and two printed hinge pins. Formats: 3MF, STL, OBJ, GLB, plus a 1:1 SVG of the contours to print and check against the real lens.

Everything runs in the browser. Photos and video never leave the phone. The interface is bilingual (French and English).

How we built it

Measurement (computer vision, OpenCV.js in a Web Worker): we detect the bank card from colour and grey-level edges, then snap each card edge to sub-pixel precision with perpendicular profiles and robust line fits that ignore the rounded corners. A homography gives a top view in millimetres. The lens is found with noise-adaptive edge detection: speck filtering removes table grain, a morphological opening removes scratches and spurs, and candidates touching the photo border or the card (shadows) are rejected. Blur, card size and cut-off lenses produce clear messages.
Data and evaluation: there is no public dataset of lenses next to bank cards, so we wrote a synthetic photo generator (Python, OpenCV). It renders lenses with bevelled rims, refraction and highlights next to random cards and backgrounds, through a tilted phone camera with blur, noise and JPEG compression, with exact ground truth. We then tuned the pipeline on our own real photos (concrete table, hand shadows, stickers, two lenses per photo).
Frame geometry (Three.js): the inner opening is a parallel offset of the scanned contour. The outer edge is the style shape scaled until the thinnest wall equals the rim thickness, slid toward the temple and capped on the nose side so the bridge gap stays open. Community styles come from Printables models: we rasterise each front, find the lens holes and measure the outer rim radially. Each rim is one closed solid with a 45° V-groove (printable without supports). Hinges have knuckles on one axis with a pin hole.
Try-on: MediaPipe Face Landmarker. A One Euro filter steadies the pose, the scale is locked from the iris size, the light is estimated from skin patches, and the face mesh is used as a depth mask and shadow catcher.
App: React, TypeScript and Vite as an installable PWA, deployed on Vercel.

Challenges we ran into

Transparent lenses on real tables: our first pipeline worked on synthetic photos but failed on real ones. Concrete grain welded onto the lens rim, hand shadows looked like lenses, and scratches pulled the outline. Fixing that took edge-speck filtering, morphological opening and shadow rejection.
Millimetre accuracy from a photo: a 2 px error on the card's corners is a few percent of scale. The rounded card corners skew a naive quadrilateral, so we fit the straight middle of each side instead.
Printable geometry: an inward-offset bug flipped the lens-retaining lips outward, and collinear points created zero-area slivers. Automated closed-mesh tests caught both before export.
Fitting any style around any lens: we had to make sure the outside never thins below the minimum wall and the two rims never touch at the bridge, for 19 styles and arbitrary scanned shapes.

Accomplishments that we're proud of

On 12 synthetic test photos with exact ground truth, 11 measured with a median error of about 0.1 mm and a worst error of 0.82 mm on A and B (the jury's full-marks threshold is 1 mm). The twelfth, a nearly invisible lens on white, was not found.
Four real photos of the same lens, taken at different positions and tilts, agreed within 0.7 mm on width and 0.4 mm on height.
Automated tests check that every rim and hinge part is a closed printable solid, that the groove holds the lens, that a straight pin passes through every hinge, and that every style keeps the exact clearance and minimum wall.
It all runs in a phone browser, with no server and no images leaving the device.

What we learned

Synthetic data is great for exact numbers but too clean. Real photos with shadows and textured tables broke the pipeline in ways we hadn't imagined.
Many "AI" problems start with careful geometry: scale, perspective, offsets and watertight meshes.
Mobile-first touch design (loupes, fingertip-sized hit areas, thumb-reach toolbars) matters as much as the algorithm.

What's next for OptiFrame

Validate against caliper measurements on many real lenses, and add a trained segmentation model for low-contrast lenses.
Print and test the frames: groove depth, hinge-pin fit and nose pad comfort.
Field-test with optometrists and NGOs (Université de Montréal School of Optometry, SN-SF).