AirTable is a very powerful platform for collaborating and analyzing numerical data, but there aren't a lot of features for text data. Yet, text data is everywhere, people spend huge amount of time dealing with them as part of their work: customer feedback, emails, chat records, social media posts, reports... You name it.
Normally, to get any kind of insight from text data, you need data scientists (or expensive, specialized enterprise software) to analyze them. This requires lots of programming, and expertise with Natural Language Processing.
We want to make it easy and accessible by everyone: just a few clicks + drag-and-drop. No coding required.
What it does
Smart Text lets you do basic data science for text data on AirTable with just a few clicks. With Smart Text, you can run natural language processing with any text data you select, and get insights such as sentiment score, keywords, most-frequently used nouns / adjectives, or topic clustering (coming soon). You can feed the insights back into your table, and share the results with others. No coding is required. Smart Text takes care of all the complexity under the hood.
How we built it
We already have an NLP engine, which we used to power many products we have built in the past (see qokka.ai). We repurposed the NLP engine and built a web-server with APIs as middle-layer, specifically for AirTable use cases. The web-server handles user sessions, security, data transformation, polling, and many other things.
In the frontend, we built a simple UI following AirTable's guideline, which communicates with our API server.
In this v0.1 release, we only implemented 5% of what we want to show on the UI. So far it is only using a tiny fraction of the functionalities we have for the engine :)
Challenges we ran into
Although we have had the idea for a long time, we only heard about this contest 4 days before the submission deadline. We had to forfeit a lot of sleep!
Also, we only have very minimal idle computing power we can use for this contest. If one person uses our block to analyze lots of records, or many people to use our block at the same time, the block might run really slow.
If we win the prize money, we could use that to upgrade the server and optimize our infrastructure, so it could power everyone on AirTable.
Accomplishments that we're proud of
We made it! We really built this in 4 days. Although it is far from perfect, it works most of the time. We tried it on some product review data (Sennheiser HD-650). It is first time we feel we could analyze text data so easily, and see what all these reviews are about in just a few seconds.
What we learned
AirTable is awesome. We can't wait to build more features for Smart Block and make the user experience seamless for more use cases. With the easy-import and rich-text tablecell features provided by AirTable, not only we can quickly port some of the fancy text analysis visualizations we already built in other products to AirTable, we could also make use of our powerful crawler, use AirTable as a database, and interactively guide users to collect more data. There are so many possibilities.
What's next for Smart Text
Lots of features:
- Topic modelling and analysis
- Text clustering
- Auto segmentation for timestamped data (e.g. transcripts)
- Great looking visualizations (again see qokka.ai), such as word clouds, zoomable clusters, etc.
- Built-in multi-purpose crawler (e.g. if you have a list of e-commerce product links in your table, Smart Text can crawl the reviews for you and let you import them into the table, with just a few clicks)