Blue Lobster

main view
blueprint view
dashboard view

💡 Inspiration

Industrial 3D printing operates at the cutting edge of manufacturing, yet the way we monitor these massive machines is stuck in the past: static dashboards, fixed maintenance schedules, and black-box alerts. Unplanned downtime on an HP Metal Jet S100 costs time, materials, and money.

We were inspired by a simple question: What if the machine could actually explain itself?

The HP Metal Jet S100 challenge pushed us to think beyond traditional monitoring and into simulation-first intelligence. Instead of reacting to failures after they happen, we wanted to model why they happen, when they will emerge, and how to act on them—all in a way that’s transparent, conversational, and grounded in real physics.

⚙️ What it does

We built Blue Lobster, a full-stack Digital Twin & AI Co-Pilot designed specifically for the HP Metal Jet S100.

At its core, the system does four things:

Simulates Physics: Accurately models the physical degradation of 9 critical components across the Recoating, Printhead, and Thermal subsystems.
Optimizes with RL: Uses a Deep Q-Network (DQN) reinforcement learning agent to learn optimal maintenance strategies, outperforming standard fixed schedules.
Visualizes in 3D: Provides a real-time Next.js + React Three Fiber 3D dashboard to scrub through time and visually pinpoint hardware failures.
Agentic Diagnostics: Features an AI co-pilot that monitors telemetry, diagnoses issues, and explains root causes in real-time.

The Key Difference: The AI never guesses. Every insight is grounded in read-only simulation data and backed by explicit evidence and citations.

🏗️ How we built it

We structured the project into three tightly integrated layers:

1. Physics & Simulation (The Engine) Using PyTorch, we implemented degradation models using Weibull distributions, exponential decay, and domain-inspired heuristics. We built a Digital Twin loop that simulates the machine under various scenarios (e.g., nominal, humid factory, chaotic environments).

2. Reinforcement Learning (The Brain) We trained a DQN agent (and tabular Q-learning backups) to decide exactly when to perform maintenance. By balancing the high cost of intervention against the catastrophic risk of machine failure, the agent autonomously discovers optimal maintenance lifecycles.

3. The Agentic Co-Pilot & 3D UI (The Interface)

Backend: A FastAPI architecture that intercepts user queries. To overcome LLM token limits and context bloat, we used pytoony to compress thousands of rows of raw JSON telemetry into a highly token-efficient TOON format using an 11-tick sliding window.
LLM Tool-Calling: We integrated Gemini 3.1 Flash / Anthropic using strict tool-calling (_tool_latest_state, _tool_component_history). The agent retrieves hard data from the SQLite historian, guaranteeing zero hallucinations.
Frontend: A Next.js (App Router) and Tailwind CSS dashboard featuring a highly technical, cyberpunk-industrial aesthetic. It includes SVG line charts, degradation bars, and an interactive 3D model of the machine powered by Three.js.

🚧 Challenges we ran into

Context Window Overload: Time-series telemetry data is notoriously token-heavy. Feeding raw JSON into an LLM caused massive latency and context exhaustion. We solved this by implementing pytoony to serialize the data and passing only a targeted 11-tick sliding window around the moment of failure.
Preventing AI Hallucinations: It’s easy to build a chat wrapper; it's incredibly hard to make an LLM strictly adhere to industrial realities. We had to heavily engineer the system prompts and enforce a read-only SQLite database connection so the AI could only speak when citing explicit database ticks (t) and run IDs.
Balancing the RL Reward Function: Training the RL agent to know when to fix a part was non-trivial. Designing a reward function that perfectly balances routine maintenance costs with catastrophic failure penalties took multiple iterations before the agent behaved sensibly.
Bridging the Stack: Syncing the Python physics simulations, the FastAPI SQLite historian, the React Three Fiber 3D animation ticks, and the asynchronous AI streaming required meticulous state management.

🏆 Accomplishments that we're proud of

Building a fully working, physics-driven Digital Twin from scratch—not just a mocked UI.
Achieving >99% validation accuracy on our predictive component models.
Creating a zero-hallucination AI that provides traceable, timestamped evidence for every diagnostic claim.
Successfully training an RL agent that mathematically proves it can adapt better than fixed maintenance schedules.
Designing a gorgeous, responsive, and intuitive 3D interface that makes highly complex system telemetry actually enjoyable to read.

🧠 What we learned

We learned that combining Simulation + AI is exponentially more powerful than using either alone. The simulation provides the ground truth, while the AI provides the accessibility and reasoning.

We also learned that in industrial tech, explainability is everything. Operators don't just want an alert that says "Machine Failing"; they need to trust why the system thinks that, backed by hard data.

Technically, we leveled up our skills in PyTorch physics modeling, Next.js 3D rendering, token-compression algorithms, and strict LLM tool-calling.

🚀 What's next for Blue Lobster

Our goal is to turn industrial machines from black boxes into systems that can simulate, explain, and optimize themselves. Next steps include:

Real Hardware Integration: Swapping out our synthetic physics simulator for live, real-world telemetry from physical HP Metal Jet machines.
Expanded Failure Modes: Introducing complex, multi-component cascading failures into the Digital Twin.
Fleet-Level Intelligence: Scaling the RL agent to manage predictive maintenance schedules across an entire factory floor of machines rather than just one.
Spatial AI: Enhancing the 3D visualization so the AI Co-Pilot can highlight and physically point to degrading components in the 3D viewer during its explanations.⏎