Inspiration
You can't have a serious conversation about the Enterprise without considering Microsoft, and when it comes to integration the company has been a Gartner Magic Quadrant leader for on-prem Enterprise Application Integration (EAI) as well as Business-to- Business (B2B) integration thanks mainly to the strength of its premiere middleware offering – BizTalk Server.
BizTalk Servers, deployed at over 11,000 medium to large businesses, governments and institutions around the world, process billions of transactions such as EDI (Electronic Document Interchange), HL7, and SWIFT messages every day in mission-critical business processes across every conceivable industry.
So when I learned that Seagate was creating CORTX – an open-source Hyper-scale object storage system, specifically designed to store more data, less expensively, on-prem, I recognized an opportunity to integrate it with BizTalk Server.
Unlike integrating specific tools or applications to CORTX one at a time, what interested me about this Integration Challenge was the idea of creating just a single integration to BizTalk that would then allow any conceivable Enterprise Business Process to read/write data from/to CORTX.
Knowing how the Enterprise thinks about integration and that BizTalk drives many of the most mission critical business processes in the world, it was compelling to me to consider connecting CORTX to BizTalk so that it could be immediately and seamlessly incorporated as a first class mass capacity object storage "Enterprise Service" so that "any business process" can just start using it.
In nearly 20 years of architecting, developing and deploying dozens of complex integration solutions with BizTalk Server for big Enterprise and Government, I can tell you first hand that nearly all of them have some kind of long term data retention requirement, often dictated by regulatory compliance or other governance consideration. And not all businesses can or want to store this data in the public cloud, which makes on-prem inexpensive object storage for this largely unstructured data a very interesting alternative for enterprises.
Therefore, providing an easy way to connect CORTX to BizTalk and by proximity the Microsoft Enterprise stack, dominated by the .NET Framework and its 10s of millions of .NET developers, just seemed like a great idea. And so I got busy and wrote the CORTX BizTalk Adapter as my submission to this Integration Challenge.
What it does
The CORTX BizTalk Adapter enables CORTX to participate as a source or destination system in any BizTalk integration solution, including complex custom business processes developed with BizTalk’s world-class Orchestration engine.
How I built it
I used the Microsoft Windows Communication Foundation (WCF) Line –Of-Business (LOB) Software Development Kit (SDK) and its tooling for Microsoft’s Visual Studio to generate the .NET code empty template for the WCF-LOB Adapter. I then used the Amazon AWS .NET SDK as the S3 client library to implement the CORTX integration code within the generated adapter template. The resultant adapter was automatically compatible with BizTalk Server, because BizTalk is itself built with .NET (more than 1,000,000 lines of C#) and has integral support for WCF.
Challenges I ran into
Honestly, no technical challenges were encountered developing this integration. CORTX worked as advertised!
Accomplishments that I'm proud of
I am most proud of being able to complete the integration of CORTX with BizTalk in such a seamless way in a short period of time. Development of the adapter went so smoothly that I had plenty of time to focus on showcasing it in several demos.
What I learned
While I have extensive experience developing against Microsoft’s Azure Blob Storage .NET SDK and REST APIs, this was my first time working with Amazon’s S3 API. I was thrilled to learn that the Amazon AWS .NET SDK could be used “as is” to write .NET applications and plugins to perform the Create, Read, Update and Delete (CRUD) operations against CORTX using its S3 compatible interface. Amazon’s S3 .NET API provided a full fidelity S3 client implementation to CORTX from .NET, opening CORTX up to the full Microsoft development stack.
What's next for CORTX BizTalk Adapter
The CORTX BizTalk Adapter presented here is just a proof-of-concept with respect to features and functionality it supports. Adapters created with the WCF-LOB SDK support rich metadata discovery and schema generation for the systems they integrate with. It would be interesting to explore the possibility of taking the CORTX adapter started here and build it out further, perhaps with deep integration into MORT and its key/value and metadata APIs. Such an adapter would not only be able to read and write unstructured data (e.g.; files) to CORTX at the MOTR layer, but would also be able to support writing structured data with metadata. Furthermore, the advance capabilities of the adapter would allow a user to generate XML and/or JSON Schema definitions from the metadata so that BizTalk tooling can learn the structure of the data stored in CORTX, and then tool against that data upon retrieval of it (i.e.; Transform it with XSLT,, process it in Orchestration workflows , etc.,). Another idea would be to add an encryption/decryption option for the Adapter.
Log in or sign up for Devpost to join the conversation.