Volunteers with highly specialized skills - such as operating heavy machinery, like cranes, forklifts and excavators - are integral to recovery, cleanup and rebuilding efforts following a disaster. Yet, even some of the most experienced volunteers are not used to working in a disaster zone.
Most heavy machinery operators work in construction: the sites are regulated and predictable. Materials are also predictable - in standard weights, shapes and sizes. Disaster zones are chaotic with debris that defies easy organization and removal, slowing the recovery process.
The hazards surrounding disaster response vary by response operation, resource settings and geographic location. Volunteers must be able to calibrate and engage their skills in a low-risk environment to avoid inadvertently injuring themselves or fellow responders, or impeding cleanup and rebuilding.
Virtual reality provides the perfect immersive simulation environment, allowing volunteers to prepare for working safely, efficiently and effectively in a real disaster zone.
What it does
Our prototype VR experience provides a low-risk simulated environment for mobile hydraulic crane operators that allows them to acclimate to a particular disaster environment while practicing common safety protocols.
In the prototype VR simulation, users sit in the operator’s cab, on top of the deck. The Vive controllers act as the joysticks that raise and lower the boom, rotate the cab and extend the crane arm.
Operators must practice removing building debris and navigating hazards commonly found in a disaster zone, such as on-site first responders.
How we built it
We worked together to develop a control interface using the Vive controllers as proxies for crane joysticks. We also used game physics and optimized collision models to enable us to pick up debris, while avoiding mannequins to simulate people working near the recovery zone.
Our artist and modeler customized art assets from the Unity Asset store to present a clear approximation of building debris following a natural disaster. The crane was sourced from Turbosquid and is a commonly used model by fire departments and the military for disaster rescue and recovery.
Audio was sourced through freesound.org and tweaked in Audacity.
Challenges we ran into
Perhaps our biggest challenge was due to the physical space constraints at the Hackathon. We worked in very close proximity to other teams and experienced significant lighthouse interference - for the most part we were able to work around each other, but it slowed our initial prototyping.
Quaternion coordinate systems are complex, and the way they pass translation data throughout the hierarchy can be difficult to troubleshoot. Make sure your axis are aligned to world space!
Crane operations are trickier than they first appear. Combining real-world controls with virtual controls depends on a thorough understanding of the way those controls are deployed in the real world. It was challenging to create a prototype without direct access to subject expertise. Ideally, we would have been able to do more hands-on research directly with subject matter experts.
It was difficult to get the rope simulation, which operates the crane’s cable, and physics to work authentically. In the simulation, the crane cable needs to feel heavy and react as it would in the real-world. Refining this mechanic took longer than expected.
We experienced vexing rotation glitches, which we eventually overcame.
Accomplishments that we’re proud of
We are very proud of our concept and feel that we developed a unique premise. We feel that, with further development, it could be a viable training tool that has the potential to reduce injury and make construction sites and disaster zones safer. Simulated training is cornerstone of training for pilots and healthcare professionals. It has great potential for the construction industry and disaster response.
We are proud of the way the game feels. The controls feel authentic and it feels exciting to be sitting in the cab.
Our new members are proud that there were able to get Unity setup and working and manipulate the physics within the experience.
What we learned
We had two team members who were new to programming and Unity so experienced a robust block of immersion learning during the Hackathon. One of these members feels inspired to learn more about programming and unity and is eager to continue building her expertise.
A different division of team labor may have made the programming process smoother. In the future, we would practice pair programming to more effectively multitask, balance skill expertise and keep the project moving.
The simulation should eventually include a wider variety of surprise variables, allowing operators to practice responding to unexpected events that they may encounter during their recovery work. This mechanism is similar to approaches used in flight simulators for pilots and medical simulations for physicians and nurses.
The VR simulation could be expanded to a wide variety of heavy machines, such as excavators or bulldozers.
The simulation could also be expanded to include a wider variety of disaster scenarios, including both natural disasters and human-made disasters.