Inspiration and What it Does
Our DATAdev Project Team created a really interesting Tableau Extension that allows the data from Tableau dashboards to be easily seen, explored, and communicated in HoloLens augmented reality (AR) headsets -- and Oculus virtual reality (VR) headsets.
Our Hackathon inspiration and passion is driven by one fundamental truth: Immersive Computing is the next level of data visualization, analytics, and storytelling for hundreds of millions of enterprise knowledge workers.
Immersive Computing is exactly what our team wakes up every day to do -- and what we are delivering for Tableau with this DATAdev project.
Our Project allows the Tableau community at any level of expertise to easily take their existing Tableau data from a Tableau Dashboard - with no changes or transformations -– and add Immersive Computing by just adding the Tableau Extension we built during this Hackathon.
This means Tableau users can incorporate immersive visualizations, including AR and VR headsets, into their existing workflows -- communicating bi-directionally with with existing Tableau Dashboards - no changes to the Dashboards required.
And, the Tableau community can also see their data immersively with “real-time 3D” visualizations on laptops, touchscreens, tablets, smart phones, and more. It is genuinely impressive - and powerful.
How we built it
Our Project Team designed and developed a Tableau Extension for this Hackathon that connects with the Tableau Extension APIs as well as the APIs we have in our Immersive Computing Runtime libraries. Our Runtime APIs allow data to be explored and viewed and communicated “immersively” across AR & VR headsets - as well as traditional laptops, tablets, and screens.
What we did
After browsing the many well-documented APIs for 3rd party integrations with Tableau we arrived at the Dashboard Extension API being the best for what we envisioned.
We then cloned the extension API github repo and a couple members of our team followed the tutorials Once we understood the Tableau side of the picture, we needed a way to run our existing .Net Runtime API in the web browser. For this we turned to Blazor, and WebAssembly.
From there, development of a new extension began – one where we could run the IA Runtime API within a Tableau dashboard.
First was the development of the basic connection UI, allowing users to connect directly to na HL2 or Oculus on the Local Area Network
Then we drew upon the tutorial code to retrieve the selected Marks from the Tableau API, but soon found that doesn’t return all the available columns in a data row. Knowing that the high-dimensional visualizations of our application would benefit greatly from having more complete data access, we dug deeper.
The Worksheet.getUnderlyingTablesAsync() and getUnderlyingTableDataAsync() functions seemed to be just what we were looking for.
Added listeners so when selection or filters change, the data is retrieved and sent to the headset. Lastly, we built an HTML5 UI to add, remove, and edit Visualizations in the headset. And Voila!
Browsers block non-secure HTTP/WebSocket connections from HTTPS pages by default, so to connect to a HoloLens headset on the local LAN, the user has to ‘Allow’ this content for the Tableau website. To get around this, we will place a secure IA Runtime Server in the cloud, through which data can be fed from the dashboard to the headset and back.
We want to develop and embed a WebGL renderer of our visualizations directly into the dashboard extension, so users w/o AR headsets will see a flat screen rendering of our higher dimensional animated visuals.
Challenges we ran into
Surprisingly, there were no big challenges encountered.
Accomplishments that we're proud of
We allow any Tableau user at any level of expertise to take their existing data (and metadata) in Tableau - with absolutely no changes or transformations needed – and in a matter of minutes have it be visualized, analyzed, and communicated immersively.
What we learned
The Tableau Extension APIs are intuitive and align well with what we needed to build our Tableau Extension.
The short summary is that the Tableau APIs allowed us to develop a bi-directional intuitive interaction between an immersive computing device like the HoloLens AR headset and the Tableau Dashboards via our Runtime architecture. This is powerful and remarkably easy to add to the workflow of millions of Tableau users worldwide.
What's next for Tableau in HoloLens 2 & Oculus
What we developed for this Hackathon is both powerful for all Tableau users - and also visually stunning. If someone wants to see the “art of the possible” - this is it.
Let’s meet so you can hear our thoughts on next steps. There are so many ways that we can natively incorporate immersive computing in the road map of Tableau products.
Thank you for staging this Hackathon.
We look forward to showing all of this to you in greater detail in a video meeting we should make happen --- or even more exciting, in a headset. Let’s do this sometime soon so you can “see the whole picture” -- and we can discuss possible next steps.
Bob, Andrew, Aaron, and Randy