shape

• 

Inspiration and Problem

Shape targets the most expensive part of Computational Fluid Dynamics that happens before you press “run”: geometry/topology understanding (what bends exist, where junctions are, what can be simplified or made symmetric) and the pre-processing loop (cleanup → mesh → realize something is wrong → redo). This is a real bottleneck in practice; mesh generation and geometry prep are widely described as labor-intensive and can take a large fraction of total workflow time. In piping systems specifically, the difference between “one elbow vs two elbows”, “a tee vs a smooth junction”, or “tight vs gentle bend” directly impacts pressure loss because fittings/bends introduce additional losses beyond straight-pipe friction.

That makes the impact broad and high-value: from cryogenic and space flow hardware (LH₂/LN₂ transfer lines and propellant feed systems) to hydrogen/LNG infrastructure, chemical plants, HVAC, and industrial piping; Anywhere geometry mistakes or missed simplification opportunities can cause wasted simulation runs, wrong boundary conditions, or oversized designs.

Shape’s approach (centerline graph → segment detection → learned embeddings + curvature heuristics → agent) turns that hidden “engineering intuition” step into an interactive tool, so teams can reduce setup time, avoid reruns, and make simulation workflows accessible to more designers; not just CFD specialists.

What it does

Shape is a geometry-understanding pipeline + UI for pipe-like geometries (currently mesh .msh for speed in the demo):

User uploads a pipe mesh. Backend extracts/reads a centerline graph (nodes+edges). The centerline is segmented into straight / arc / junction using curvature + graph topology. Each segment is resampled/downsampled into a standard “model-friendly” graph. A trained graph-embedding model computes embeddings and helps infer properties like arc angle buckets.

A Gemini 3 agent turns the structured geometry summary into natural-language answers and follow-up explanations inside the UI (e.g., “How many arcs above 30°?”, “Describe this pipe”).

How we built it

1) Synthetic dataset (open)

We generate a large synthetic dataset of pipe centerline graphs (straight, corners, arcs at many angles, junctions) and publish it on Hugging Face to make the work reproducible. Generates 10000 samples per category (17 arc classes + straight + corner + junction (2X) = 21 categories, 21k samples total). https://huggingface.co/datasets/bayang/shape

2) Graph encoder (GATv2)

We train a GATv2-based graph encoder in PyTorch Geometric (message passing over centerline nodes/edges) using GATv2Conv. We pool node features to a single graph embedding using attentional aggregation.

3) Supervised Contrastive Learning (SupCon)

We train the encoder with Supervised Contrastive Learning so segments of the same motif/angle become close in embedding space, while different motifs become far apart. SupCon loss (InfoNCE form) -> https://pytorch-geometric.readthedocs.io/en/2.5.3/generated/torch_geometric.nn.conv.GATConv.html

4) Geometry segmentation (curvature + topology)

Junction detection: graph degree ≥ 3 Curvature estimation: Menger curvature on centerline triplets (robust for discrete polylines). Straightness: 1−cos(θ) style checks on local tangent changes Resample/downsample: uniform arc-length spacing so the model sees consistent graph “style”.

5) Gemini 3 agent for interactive Q&A

We pass the extracted structure (segments + angles + counts + lengths) into Gemini 3 to:

• summarize the pipe topology,
• answer user questions,
• and explain “why” (e.g., “this is an arc because curvature stayed high across these points…”).

The paper explain -> https://github.com/tihiera/shape/blob/main/shape%20-paper.pdf

How did we use Gemini3

1. Function Calling (Tool Use)

We define 8 DSL tools as Gemini function declarations. Gemini decides which tools to call based on the user's natural language query:

Tool	Purpose	Example Trigger
list_segments	List all segments	"describe this geometry"
filter_segments	Filter by type/angle/length	"show arcs above 90°"
count_segments	Count (optionally filtered)	"how many straights?"
sum_field	Sum a numeric field	"total length of all arcs"
group_by	Group by type with stats	"break down by type"
topk_by	Top-K by a field	"3 sharpest bends"
describe_segment	Full detail of one segment	"tell me about segment #5"
highlight_segments	Highlight in 3D viewer	"highlight the U-bends"

2. Multi-Turn Reasoning

Gemini can chain multiple tool calls in a single query. For example:

User: "Find all arcs above 60° and highlight the sharpest one"

  Round 1: Gemini calls filter_segments(type="arc", arc_angle_deg__gt=60)
           → gets 4 matching arcs

  Round 2: Gemini calls topk_by(field="arc_angle_deg", k=1)
           → gets segment #9 (180° U-bend)

  Round 3: Gemini calls highlight_segments(segment_ids=[9])
           → returns highlight data for frontend

  Round 4: Gemini generates natural language answer:
           "There are 4 arcs above 60°. The sharpest is Segment #9,
            a 180° U-bend with radius 7.6. I've highlighted it for you."

3. Conversation Context & Follow-ups

Each user message is sent to Gemini with the full conversation history, including summaries of previous tool calls and their results. This enables follow-up queries:

User: "how many arcs above 40°?"
AI:   calls filter_segments(type="arc", arc_angle_deg__gt=40) → 6 arcs
      "There are 6 arcs above 40°..."

User: "and above 80°?"
AI:   understands this is a follow-up, calls filter_segments(type="arc", arc_angle_deg__gt=80) → 3 arcs
      "Narrowing down: 3 arcs exceed 80°..."

User: "does it contain straight sections?"
AI:   calls filter_segments(type="straight") → 7 straights
      "Yes, there are 7 straight sections..."

The chat history (stored in chat.jsonl) includes tool call metadata so Gemini always knows what was previously computed.

4. System Prompt Engineering

The system prompt (ai/prompts.py) provides Gemini with:

• Role definition: "You are a 3D geometry analysis assistant"
• Domain knowledge: Segment types, fields, and their physical meanings
• Live segment data: Injected at query time via build_segments_context()
• Follow-up instructions: Explicit rules for handling contextual queries
• Response style: Natural language, bullet points, specific measurements, no raw data dumps

5. Configuration

Setting	Value
Model	gemini-3-flash-preview
SDK	google-genai (Python)
Temperature	0.1 (deterministic, factual)
Max tool rounds	8 per query
Auth	API key via .env file

Why Gemini 3 Flash?

• Function calling: Native support for tool/function declarations; Gemini decides which tools to call and how to combine them
• Multi-turn: Can chain 2-5 tool calls to answer complex queries in a single interaction
• Context window: Handles 20+ messages of conversation history with tool call metadata
• Speed: Flash variant is fast enough for real-time WebSocket streaming (typically 2-5 seconds per query)
• Structured output: Returns clean function call parameters that map directly to our DSL engine

How to use shape frontend

You would need the mesh that represent the physical pipe. It is available under here for testing and demonstration:

File	Description	Geometry	Size
simple_bend.msh	Single 90° pipe bend	1 arc segment	258 KB
u_bend.msh	U-shaped 180° return bend	Straights + U-arc	343 KB
s_curve.msh	S-shaped double bend	2 opposing arcs + straights	359 KB
t_junction.msh	T-junction (3-way split)	1 junction + 3 branches	401 KB
complex_network.msh	Multi-branch pipe network	Multiple junctions, bends, straights, and corners	1.4 MB

Challenges we ran into

• Centerline extraction from STEP is non-trivial. Meshing a CAD surface doesn’t directly give a centerline; for the demo we switched to .msh to keep the pipeline reliable.
• Label leakage / shortcuts. Some global features (like “total bend”) can make the task too easy and reduce true learning. We removed/limited such features so the model relies more on geometry.
• SupCon plateaus & class imbalance. Junction variants (T vs Y) were hard to separate and caused unstable per-class performance; merging them into a single junction class removed ambiguity and improved stability.

Accomplishments that we're proud of

• An end-to-end demo: upload mesh → detect centerline segments → highlight in 3D → ask questions → get explanations.
• A published synthetic dataset (open + reproducible).
• A working GATv2 + SupCon training pipeline using standard PyG blocks.
• Strong validation metrics on the current synthetic distribution (and a clean path to increase difficulty as we scale realism).

What we learned

• In geometry ML, data design is everything: if classes are too “perfect,” retrieval can saturate early and metrics won’t reflect real-world robustness.
• Simple geometry algorithms (curvature, degree, resampling) + learning-based embeddings can complement each other: heuristics give structure, embeddings give “semantic similarity.”

What's next for shape

• Proper STEP/solid ingestion with robust centerline extraction (e.g., VMTK-style centerline on tubular surfaces) and handling thickness/coaxial centerlines.
• Expand dataset realism: noisy sampling, random rotations/scales, varying radii, near-degenerate junctions, mixed 3D out-of-plane bends, and partial/incomplete centerlines.
• Add symmetry detection + simulation hints so the agent can recommend actions like: “exploit symmetry plane,” “reduce domain,” “expected pressure-drop hotspots,” “likely multiphase risk at these curvature regions.”
• Long-term: a foundation model that runs in a sandbox and learns geometry → physics causality for cryogenic/space systems.

Built With

• aistudio
• antigravity
• gemini-3
• javascript
• python
• pytorch
• tailwindcss
• vercel