Audiograms for Square

Inspiration

A few months ago Substack released a tool that can turn a snippet of a post into a shareable Substack audiogram. I wanted to do the same with a normal static Square catalog item.

What it does

The project connects to the Square Catalog API to retrieve the item name, description and image. That information is then used to generate text and speech that is then combined a shareable audiogram, in video format.

How we built it

This project uses a Rails app to connect to Square and the Google APIs. The audiograms are generated with a custom ExpressJS app.

Challenges we ran into

Google's APIs are really confusing when it comes to finding documentation. ExpressJS and Node were new to me. I needed to learn how to write Express and Node code and then figure out how to deploy it to production!

Accomplishments that we're proud of

I'm very proud of the work done to get the audiogram generator up and running. Because the audiogram is the result of a multistage pipeline of 3rd party API calls, getting them to work in order every time was really exciting.

Synthesized audio generated by Google is returned as base64-encoded content, learning how to work with that was a good feeling.

What we learned

That Google's AI API documentation is really hard to work with. Learning ExpressJS, Node, Canvas and how to deploy ExpressJS/Node was really cool.

What's next for Audiograms for Square

1) Make Audiogram generator service more reliable 2) Allow users to customize orientation of audiogram video 3) Allow users to change wave form style 4) Allow users to change wave form color 5) Allow users to change wave form position 6) Allow users to add text 7) Allow users to post directly to IG

Built With

• amazon-web-services
• express.js
• google-text-to-speech
• googlepalm
• javascript.
• node.js
• rails.
• ruby.
• squareapi