Inspiration / What it does

This project builds a integration between the CORTX API and Hashicorp Terraform. Infrastructure As Code tools like Terraform can allow organizations to standardize infrastructure and policies across multiple regions, environments, applications, etc. With this provider, developers can now deploy CORTX resources along their other infrastructure rather than relying on custom scripts or one-off deployment patterns.

This provider is meant for provisioning S3 compatible resources (buckets, objects, access configs) on an already existing CORTX server. While Terraform is a great tool, and can be used for bootstrapping a CORTX server (See: ./examples/aws-demo-server-setup/), the provider is meant working with S3 API resources, not bootstrapping the server itself.

The provider is completely cloud-agnostic, as long as a developer has access to a CORTX data endpoint and credentials for that CORTX deployment (homelab, k8s, AWS, datacenter, etc.), they can manage resources with TF.

How we built it

The provider itself is written in Go, the examples shown in the video use Hashicorp Terraform to deploy resources with the AWS provider (e.g. for example instances and networking) and the CORTX provider (for storage resources).

Challenges we ran into

I tried configuring CORTX in a few environments to make sure I understood what was going on. On CloudShare and my home network, this was easy enough.

When I moved to writing an example using an AWS EC2 instance (See: ./examples/aws-demo-server-setup/), the networking became relatively challenging. Luckily, I was able to follow the EC2 guide from Seagate to help work through it. Between AWS security groups, the OVA's firewalls, ACLs, etc, developing a smooth, 1-click provisioning of a new CORTX server was pretty challenging, but eventually I came up with a solution to these networking questions that let me just terraform apply to bootstrap a new, isolated CORTX instance.

Accomplishments that we're proud of / What we learned

I've written Terraform modules (collections of resources in Terraform's DSL) before, but I've never written a provider library that interacts with a custom API before. This is a pretty interesting challenge, as it allows more people to start working with CORTX. In addition to getting to write some Go for the provider library (I don't often write Go, it's always a nice change of pace!), some of the networking/firewall challenges described above were pretty rewarding to resolve!

What's next for CORTX x Terraform

Right now, we only support the most basic S3 operations, but I'd like to expand this to a proper provider that takes the pain out of manual resource management.

I'm keying off of this sheet to assess to what extent the provider covers all the features of the CORTX S3 implementation. Because some features of the AWS s3 API don't exist on CORTX, knowing which will be supported and which to develop is pretty important.

Built With

Share this project: