Inspiration

Tableau Prep Conductor is currently the only way to schedule and manage Tableau Prep Flows on Tableau Online/Server and does not automatically handle "just in time execution" of dependendent Flows. With the new ability to author Flows in the browser coupled with the robust Tableau Server Metadata and REST API functionality around Prep Flows, the functionality of Tableau Prep Conductor can be replicated and expanded with any modern workflow management platform.

What it does

Tableau Prep Orchestrator is a open source alternative to Tableau Prep Conductor that uses a Dockerized extension of Apache Airflow to automatically gather your published flows and set downstream dependencies into 2 self-organizing Airflow DAGs.

How we built it

  • Docker
  • Apache Airflow
  • Python
  • Tableau Server Client API Wrapper
    • Tableau REST API
    • Tableau Metadata API
  • YAML configuration file

Challenges we ran into

The Metadata and REST API calls are fairly straightforward, however, stitching together seamless worklow as the docker conatiners spin up and produce an initial state that isn't confusing for the end user was the main challenge. Below were the key challenges in streamlining the process:

  • The first DAG has to finish a successful run before second can be initialized and how to handle this without throwing an error message in Airflow upon start up
  • Default Airflow functionality is to call a task finished when the REST API call to Tableau Server returns a response 200. However, in the case of running Prep Flows on Server, it just means the Flow has been successfully queued to run. I had to build a loop that periodically polls Tableau Server for a Flow Run's current status and force Airflow task success only once the Prep Flow succeeds.

Accomplishments that we're proud of

  • Stringing together sequential calls from multiple Tableau APIs and endpoints to create an automated Tableau Prep workflow engine

What we learned

  • Creative ways of interacting with Tableau APIs to manage state across Airflow and Tableau Server simulatneously
  • Better understanding of how Tableau Server handles running Prep Flows
  • How to tweak Airflow and Docker configs for a non-traditional use case

What's next for Tableau Prep Orchestrator

  • Store sensitive tableau server connection information in secrets instead of YAML after Docker containers are deployed
  • Document detailed productionization
    • Deploy with Docker Swarm
    • Add authentication
  • Upgrade to us Airflow 2.0

Built With

  • apache-airflow
  • docker
  • python
  • tableau-metadata-api
  • tableau-rest-api
Share this project:

Updates