Inspiration
What it does
Inspiration
A 3D alignment can look convincing and still place a part in the wrong pose.
In a CAD-guided inspection workflow, that distinction matters. Repeated geometry, symmetry and incomplete scans can produce a strong geometric fit without identifying the correct orientation. If a downstream system treats that fit as proof of correctness, it can carry an unreliable pose into planning or machining.
We built AutoFrame 3D around one question: What evidence should an operator see before trusting an estimated pose?
What it does
AutoFrame 3D is an interactive workspace for inspecting scan-to-CAD alignment and investigating misleading registration results.
The interface brings the measured geometry, CAD overlay, pose information and evaluation evidence into one view. Users can explore recorded benchmark cases, compare available method results, inspect alignment metrics and reveal benchmark ground truth to check correctness.
Its central distinction is acceptance versus correctness. Passing a fitness or residual threshold means a result satisfies that geometric criterion. It does not establish that the estimated pose is correct.
The project includes:
- Recorded benchmark mode for reproducible inspection of saved results.
- Synthetic demonstration mode for illustrating pose mismatch and correction.
- An experimental AI investigation workflow for explaining supplied evidence through NVIDIA Nemotron on Nebius Token Factory.
Recorded results and synthetic demonstrations are labelled so users can understand what each view demonstrates.
What makes AutoFrame 3D different
AutoFrame 3D addresses a critical blind spot in 3D inspection: a convincing geometric alignment can still represent the wrong pose.
Our workspace brings CAD overlays, pose inspection, registration metrics and benchmark evidence together so users can examine an alignment before trusting it downstream. It makes the distinction between geometric acceptance and pose correctness visible and understandable.
The project combines an interactive 3D interface, reproducible benchmark cases, synthetic pose demonstrations and downloadable evidence. An experimental NVIDIA Nemotron integration through Nebius Token Factory extends the workspace toward evidence-grounded investigation.
Our strongest achievement is turning a difficult registration problem into a clear, inspectable workflow—helping users understand what the available evidence supports and where further verification is needed.
How we built it
We combined a Python registration and evaluation backend with a FastAPI service and a React/Vite frontend. The interactive 3D viewer uses Three.js to make spatial mismatches easier to inspect.
The evidence workflow connects saved cases, method results, thresholds and provenance with downloadable reports. Benchmark ground truth is a separate evaluation reference rather than an input available in normal deployment.
For the experimental copilot, we integrated NVIDIA Nemotron through Nebius Token Factory. Its intended role is to interpret supplied registration evidence and help an operator investigate a result. Geometry computation and pose evaluation remain separate from the language model.
Provider calls use bounded request budgets, strict response validation and saved diagnostic records.
Challenges we faced
The central challenge was distinguishing a geometrically plausible alignment from the correct pose. Symmetry, repeated surfaces and incomplete scans can make an incorrect orientation appear convincing.
We addressed this through an inspection workflow that separates geometric acceptance from benchmark correctness and presents CAD overlays, registration metrics and evaluation evidence together.
Another challenge was making a complex 3D pipeline understandable. We developed interactive pose inspection, clearly labelled demonstration modes and downloadable evidence so users can explore results and understand their context.
Accomplishments we are proud of
We turned a difficult registration failure into something an operator can inspect: the estimated pose, the acceptance criteria and the evidence of correctness are visible together.
We also built a reproducible recorded demonstration, evidence exports and an interactive interface that communicates the difference between a visually plausible alignment and a verified result.
What we learned
Low alignment error is useful evidence, but it is not a guarantee of correct pose.
We learned to separate three questions: Did the geometry pass a threshold? Was the pose correct under benchmark evaluation? Did the AI explanation accurately reflect the supplied evidence?
Each requires its own validation.
What is next
Our next priorities are completing and reviewing the bounded live investigations, improving detection of ambiguous alignments, and testing with physical scanner data and independently measured poses.
AutoFrame 3D is a research prototype. It does not yet certify machining readiness or demonstrate autonomous physical correction.
Demo and source
- Watch the demonstration
- Explore the release code and setup instructions ## How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned
What's next for AutoFrame 3D
Built With
- fastapi
- nebius
- nemotron
- nvidia
- open3d
- python
- react
- three.js
- typescript
- vite