Airbyte Singlestore Connector

Recurly -> SingleStore Connector
Connector logs
SingleStore configurations

Inspiration

Airbyte is becoming a widely used tool to bring data from different sources to different destinations. It is open source and easy to manage and deploy. Unfortunately, the current sources and destinations Airbyte ships with don't contain SingleStore and here comes the inspiration to build a "destination" for SingleStore.

What it does

The destination built as part of this hackathon is an Airbyte specific destination that could be use in conjunction with any Airbyte source to write the imported data from the source to SingleStore.

How we built it

Using the instruction provided by Airbyte to build a destination connector in Python https://docs.airbyte.com/connector-development/tutorials/building-a-python-destination and following the documentation of SingleStore https://docs.singlestore.com/managed-service/en/reference/sql-reference.html

Challenges we ran into

Flattening the nested JSON coming from the source response and creating the corresponding tables was the biggest challenge. The other option would be storing the response as JSON in the database which would make the destination much simpler.