SatDataForce

• 

  Team Collaboration Geo-Rx

Bridge Copernicus satellite data with AR/VR technology for effective emergency and health service responses taking into account the hidden dangers of deformations on health care buildings, facilities and key lifeline structures in EU communities and regions. This is a real world, real data response to on how to maintain a healthy and fully operational COVID19 health system enhanced and enabled by a new toolbox of technology integration.

Inspiration

Zagreb experienced an earthquake on 22nd of March that caused widespread damage to health service facilities link. This proved to be the latest of a number of other health service disruptions caused by primary or secondary disasters that had detrimental effect on the ability to provide effective intervention and response to Covid19. Others include earthquake damage to Covid19 response center in Salt Lake City, Utah, USA link and collapse of a hotel used for hospitalisation of Covid19 patients in Wuhan link.

Carl Pucci, CEO of Datel/Ovela in Estonia and Jan Ståhlberg, CEO of SiliconHagen had a vision to bridge deformation data from the Copernicus satelitte with immersive visualisation and insight through AR and VR. The mission was to generate a new and more effective preparedness across European Union member states for structural and geo-physical deformations that can impinge health services functioning at full capacity for alle communities for a Covid19 response now, tomorrow, next week and in a year.

What it does

The Geo-Rx solution

It is a multi-level planning, intervention and training solution targeting different levels of professionals involved in ensuring that health services are operating at maximum capacity. All applications use real time geo located data with a possibilty to process and deploy within hours. The three levels of the solutions are:

(1) Existing broad and big picture model using readily available data and modelling provided by Ovela reviewed by engineering and emergency response planners to identify potential failure modes within our healthcare infrastructure environmental system - see picture references above. (2) A new Augmented Reality application using Akular’s platform for on site identification, description and inspection of areas of concern as identified by deformation data from Ovela’s modelling. (3) A new Virtual Reality simulation and training application for first responders using data and scenario building from Ovela and Akular’s modelling.

Augmented Reality

This new satellite data enabled application can be used by maintenance staff, structural engineers, sub-contractors and others with the need to inspect deformation data associated with health care facilities, community buildings, infrastructure points and other key points of concern for a fully operational emergency response. This is a real time data inspection involving the identification and inspection of where and how deformation is placed for an accurate assessment of damage and "fit-for-use". This could involve either maintanance work, building emergency repairs, construction work, redesignating patient wards (i.e. ICUs), moving key services to other sites etc.

A recording function will allow for audio file and immediate edit, so that this information can be sent to health authorities, ministry, European Commission, WHO or other key stakeholders involved in resource management, funding etc.

Future versions of the AR application can incorporate different scenarios devised by structural and geophysical engineers, incorporating continued building damage, collapse, flooding, blocked access and other incidents that impact a particular location and site.

Virtual Reality

Increased evidence of learning effectiveness and efficacy through the use of virtual reality for emergency responses and planning (i.e. see the NYC OEM simulation) as it allows for preparation, prevention, and mastery in safe environments and enhances learning through trial, error, and success.

XR for emergency training

VR is more effective than traditional methods for first responder emergency training (e.g. evacuation scenarios)

XR training is less costly than real-life training

Simulating hazardous situations may trigger similar stress responses experienced in real emergency situations

Can be used to create real-time crisis management simulations (e.g. mass casualty incidents in hospitals)

XR for visualization (real-life planning)

Many datalayers that are too complex to be viewed through a desktop screen (i.e. for 3D mining models, complex data visualization, big data river disaster management )

More accessible to non-expert stakeholders

Intuitive approach for displaying spatial information

XR for awareness

Immersiveness of the experience leads to higher engagement and retention

Higher risk awareness through increased presence (feeling of being “there” in a real emergency situation)

How it was built

Collection of data from the Copernicus/ESA Sentinel Earth Observation system processed by Estonian company Ovela's AI modelling engine "Sille". This provided the team with an indepth analysis of deformations of a case site, Bytom in Poland. Structural engineers from Schnabel Engineering were able to provide in-depth analysis of the impact of deformations on the communities, infrastructure and health facilities in the area.

Slovakian augmented reality company Akular was able to use their platform Akular 2.0 link to geolocate deformations on a hospital site in Bytom Poland to provide a tablet based augmented reality view of how deformations was affecting the structural integrity of the various buildings on the site. This is a completely new tool for data analysis and insight that will improve the ability to use real time data for improved communication and action to ensure that health service facilities and buildings are kept at optimal operational capacity.

Finally Danish company SiliconHagen was able to convene experts in virtual reality to produce storyboards and draft reditions of how a unique new training platfrom was able to use real time geo location data for improved and near-real life training of first responders to prepare them for scenarios that would hinder them in normal fast and effective response times.

Challenges we ran into

We had set ourselves very ambitious milestones and goals for our vision and mission. We did not have enough time to be able to develop all the technology integration aspects we had planned. While we were able to integrate data into the augmented reality application, it took longer than expected to integrate it into virtual reality that could be used and tested in real life. This remains one of our next steps.

Having a global set of companies means that timing of task handling and team co-ordination over a few days over the weekend was challenging. This brought challenges in terms of hand-over of tasks, experiments, testing and prototyping.With more time, we could have had more cross team experiential testing and development.

Accomplishments that we are proud of

We did it! We managed to get different technology and data based platforms to integrate and produce completely new products that we are going to keep developing and improving into the future (hopefully fast tracked with some award money from the Hackathon).

A massive inter company and inter team understanding and celebration of how we are now able to bridge Copernicus Satellite data and augmented and virtual reality. Where this was subject of ideas and conversations before, the hackathon made it possible to make it a viable product.

What we learned

Datel/Ovela and Schnabel Engineering learned about how their InSAR modelling was able to be markedly improved through immersive visualisation for inspection, analysis and training purposes in augmented and virtual reality.

Akular and SiliconHagen learned how satellite data is able to provide a much more detailed, rich and near real life input to augmented and virtual reality training tools. This had not been tried by any industry providers before.

All team members learned that we were fully aligned and collaborated amazingly and that we need to continue working and improving on our concepts for full product development and implementation.

What's next for SatDataForce

We will continue to have weekly meetings to assess how we can progress further with the current proof of concept products we developed.

Whereas we managed to produce an augmented reality working prototype that can be used in real time on location, we will now also produce a virtual reality training product that can work as part of a warning, inspection and training suite that is able to reach different levels of health service professionals - from authority level to first responders.