I wanted to be able to display a list of pull requests closed recently on a Confluence page as part of a change management documentation process, but couldn't find a straightforward way to do so. This got me to think that we have so many sources of data all over the place but no real central way of pulling and displaying them where we want it/how we want it. So I started working on Conduit!
What it does
Bring your data back to you
Conduit is a platform that lets you create pipelines that pulls data from various sources (called producers) back to you. Currently there is support for HTTP APIs, but there may be more in the future.
Transform and present your data the way you want it
Conduit allows you to create templates to present your data how you want to. The easy to use templating engine includes features from looping through an array of items to util functions that can transform data. The template also blends in with the Confluence page styles and supports additional formatting such as code blocks, blockquotes, tables, etc.
How I built it
The backend API is written in Golang. It uses Firestore to store the pipelines that are created since the pipelines configuration has to be flexible.
The Confluence integration/macro is built on top of the Atlassian Connect Express framework (which is in Node.js), and the macro editor frontend is built using React. The integration also has a Postgres store.
Both services are currently deployed onto Heroku.
Challenges I ran into
- Atlaskit UI source code examples are not viewable and it can be hard to integrate sometimes since the components are using outdated libraries
- Confluence has bugs that prevented the macro from working as expected (eg. Editing a macro doesn't refresh the macro preview panel, Can't use
Accomplishments that I'm proud of
- Getting more familiar with React/Redux and building a full UI with these technologies
- Learning how to use GitHub Workflow/Actions
- Successfully creating a macro on Confluence
What I learned
- GitHub Workflow/Actions
- Atlassian Connect
What's next for Conduit
- More producers/ways to pull data apart from HTTP APIs
- Better transformation functions
- UI/UX improvements
- More integrations