Main Dashboard
Track Invidividual Machines
Narrow Down Bottlenecks
Pass Cleaned Data to Other Agents
Memory for Simulation can be stored and sent to agents directly via MCP connector

Inspiration

In November 2022, Reuters reported that protests had reduced operations at Peru’s Las Bambas copper mine to just 30% of its usual capacity. The report described recurring disruptions involving roadblocks that interrupted the movement of supplies and copper.

Source: Reuters, published by Mining Weekly

For a site manager, a roadblock creates questions that a production chart alone cannot answer. When did it begin? Which trucks were affected? What else was happening? Have we faced something similar before?

Caterpillar’s challenge asked us to give machines and jobsites a persistent memory. We focused on a specific scenario: protests and strikes that block a site’s transport routes.

We wanted the experience of one shift to remain useful long after that shift ended.

What it does

We built a time machine for a jobsite.

Using the history available in our database, users can return to a recorded day and replay the site’s operations in a spatial simulation. They can narrow an investigation to a particular time, location, and event, then examine the relevant machines, conditions, and production data together.

Our demonstration follows a roadblock that interrupts truck movements. A supervisor asks, “Why was production down?” The AI helps locate the relevant incident and retrieve its recorded context, while the replay lets the supervisor inspect what happened.

The relevant context can include:

When and where the roadblock occurred.
Which trucks were affected and how their movements changed.
Available terrain, weather, and temperature data.
Production measurements during the disruption.
Recorded information about the event and the site's response.

A protest or strike is identified through recorded event information, rather than inferred from stopped trucks alone. Protest-related data is limited to headcounts and stated reasons, without names.

Why memory matters

An incident should remain useful after someone finishes investigating it.

Our database preserves the history that makes an event replayable and searchable. Bringing the relevant records together gives the incident context: its time, place, affected equipment, surrounding conditions, and operational impact.

That creates a foundation for remembering similar situations across shifts. When another roadblock occurs, the team can revisit previous events and examine what happened instead of reconstructing the past from scratch.

This is our interpretation of Caterpillar’s memory challenge: preserve experience in a form that both people and AI can use.

How we built it

We organized the prototype around four connected parts:

Persistent storage: a database holds the available historical site data and events.
Replay and simulation: a spatial interface reconstructs the modeled operation of a selected day and lets users inspect specific moments.
AI-assisted investigation: natural-language questions help users locate an incident and gather the relevant context.
MCP access: our Model Context Protocol connector makes the exposed tools and context available to compatible external AI applications.

The database preserves the history. The replay makes it understandable. The AI helps investigate it. MCP makes that memory accessible beyond our interface.

Our prototype uses simulated fleet operations. Connecting real fleet telematics is a future extension.

Challenges we faced

The central challenge was deciding what information belongs together when investigating an incident. A production drop can coincide with weather changes, equipment delays, and a road closure. We needed to connect the relevant records without treating every nearby event as the cause.

Another challenge was making historical data understandable. A user needs to see where the disruption occurred, how it developed over time, and which machines were involved. The replay provides that shared spatial and temporal context.

We also had to keep historical reconstruction distinct from hypothetical simulation. Replaying a stored scenario shows what the available records describe; testing another response produces a modeled outcome.

What we learned

Useful memory requires more than storing data. It requires enough context to understand an event and retrieve it when it matters again.

We also learned that visualization and AI serve complementary purposes. AI helps users find and summarize relevant information, while the replay gives them a way to inspect the situation themselves.

Finally, MCP extends the usefulness of that memory. An external assistant can access the capabilities we expose and use jobsite context within another workflow.

Accomplishments we're proud of

We connected a persistent operational history, a visual replay, natural-language investigation, and MCP access around one concrete use case.

A user can move from a broad question—“Why was production down?”—to a specific event, inspect its context, and make that information available for further use.

What's next

Our next step is to turn incident memory into practical follow-up workflows:

Automatic incident-report drafts: assemble the timeline, affected trucks, conditions, and production measurements for human review.
Delay-evidence packs: collect relevant records for project reviews and potential contract or insurance documentation, with estimated losses clearly labeled.
Shift alerts and handoffs: connect Slack or Teams so incoming crews receive relevant incident history and previous responses.
Incident tickets: connect project-management tools such as Procore or Jira to turn an investigation into tracked follow-up work.
Pre-shift planning: combine weather forecasts with supported what-if simulations.
Real fleet data: integrate telematics through ISO 15143-3-compatible data sources.
Training: let new supervisors practice using recorded site scenarios.