Team Reality - Stickiemaps (Defence - City Shield)

Enhanced Military Training with HoloLens

Comment

Inspiration

In military operations, situational awareness and quick access to critical information can be the difference between success and failure. Our inspiration for Stickiemaps came from the need to enhance the efficiency and effectiveness of military training and operations. We envisioned a solution that leverages cutting-edge AR technology to provide military personnel, especially squad commanders, with real-time, hands-free access to vital data and interactive tools. The ability to visualise and interact with 3D maps and operational data in an immersive environment inspired us to create a tool that could revolutionise military training and mission planning.

What it does

Stickiemaps is a productivity app designed specifically for military squad commanders. It utilises the Microsoft HoloLens 2 to deliver a hands-free, augmented reality experience. Key features include:

  • Interactive Heads-Up Display (HUD): Provides real-time information and interactive UI elements that follow the user, allowing for easy tracking of logistics, operations, and squad statistics.
  • Map Generation via Vuforia: Using Vuforia’s image recognition, commanders can view and interact with 3D maps generated based on unique physical markers (stickers). This allows for quick and intuitive access to mission-critical maps and data.
  • Customisable Information Panels: Users can attach different images to trigger specific prefabs, such as detailed terrain maps or tactical overlays, providing flexibility and adaptability in various mission scenarios.

How we built it

We built Stickiemaps using a combination of powerful tools and technologies:

  • Unity: Served as our main development environment, allowing us to create a robust AR application.
  • Vuforia: Enabled the recognition and projection of 3D maps and interactive elements based on physical markers.
  • Microsoft HoloLens 2: Our primary deployment platform, leveraging its advanced AR capabilities and hands-free operation.
  • MRTK (Mixed Reality Toolkit): Provided essential components and frameworks for building mixed reality experiences.
  • Photon PUN 2 (Photon Unity Networking): Though initially considered for multiplayer functionality, we streamlined our approach to focus on single-user interactions to enhance performance and reliability.
  • Programming Languages: C#, C++, JavaScript for scripting and interactive components; ShaderLab, GLSL, HLSL for graphics and visual effects.
  • Other Tools: Integrated design and modelling tools to create realistic and functional prefabs and UI elements.

Challenges we ran into

Developing Stickiemaps came with its share of challenges:

  • Technical Limitations: Balancing the complex requirements of a full-scale simulator with the practical limitations of available hardware. We had to pivot from a more ambitious multiplayer simulator to a streamlined, focused productivity tool.
  • Integration Issues: Encountered conflicts when integrating PUN 2 and MRTK, leading us to re-evaluate our approach and focus on the strengths of the HoloLens 2 for displaying information.
  • Learning Curve: Adapting to the specific demands of developing for HoloLens 2, especially in optimising for hands-free interaction and ensuring smooth user experiences.
  • Resource Constraints: Managing the limited time and resources available during the hackathon to develop a fully functional and polished application.

Accomplishments that we're proud of

Despite the challenges, we are proud of several key accomplishments:

  • Successful Deployment: Managed to deploy a fully functional AR app on the HoloLens 2 that meets the core objectives of our project.
  • Innovative Use of Vuforia: Effectively used Vuforia to provide an intuitive and flexible way to access and interact with 3D maps and operational data.
  • User-Friendly Design: Developed a user-friendly interface and hands-free interaction model that simplifies complex military operations and enhances situational awareness.
  • Adaptability: Demonstrated our ability to pivot and adapt our project focus in response to technical challenges and feedback from DSTA staff including engineers.

What we learned

The development of Stickiemaps was a tremendous learning experience for our team:

  • AR Development: Gained deep insights into the nuances of developing augmented reality applications, particularly for specialised hardware like the HoloLens 2.
  • Problem-Solving: Learned to navigate and overcome integration challenges and technical constraints, leading to a more streamlined and effective solution.
  • Collaboration and Adaptability: The importance of teamwork, adaptability, and iterative development in responding to feedback and evolving project requirements.
  • User-Centric Design: Understanding the importance of designing with the end-user in mind, focusing on ease of use, clarity, and practical functionality.

What's next for Team Reality - Stickiemaps

Looking ahead, we have exciting plans for the future of Stickiemaps:

  • Enhanced Features: Explore adding more advanced features, such as AI-driven analytics and real-time data integration to provide even more robust situational awareness and decision support.
  • Expanded Use Cases: Adapt the core technology to other sectors beyond military training, such as emergency response, law enforcement, and industrial applications.
  • User Testing and Feedback: Conduct extensive user testing to refine the interface and functionality, ensuring it meets the needs and expectations of its users.
  • Collaboration and Partnerships: Seek partnerships with military and technology organisations to further develop and deploy Stickiemaps in real-world scenarios.

Built With

Share this project:

Updates