DATATALK

Top liked products - get query results in a formatted table
Get the brand with most products in the inventory - run any arbitrary script using the Firestore SDK
Create a new book with data for El Quijote
Round up all product prices to two decimal positions
Inline data modification after querying, and data export supported

Inspiration

We wanted to imagine what the future of a content management system would look like. We think that new generative AIs can help us build new human-computer interfaces.

Over the years, we accumulated a variety of Firestore scripts in all of our projects. We aimed to eliminate those folders filled with JavaScript and TypeScript scripts and use a human and natural interface for these operations. Now, we can perform these operations, and it’s not limited to us; even non-techies can manage it. We no longer need a ticket in Jira/Monday/Asana and sprint planning to change a field and rename it; we can do it directly.

We were inspired by Jupyter Notebooks and chatbots. We think it's a good UI where you can use human commands instead of code to generate the code and then render the data with a beautiful UI.

What It Does

We built a conversational interface for performing any kind of operations on Firebase Firestore. You can edit data, add new rows, inspect data, delete records, copy from one collection to another, and calculate aggregations. At the base, we have integrated all operations available in Firestore with code generation that can calculate frequencies and other statistics.

With a command like, "Increase the price of all products by 10%," we are running a batch job that updates the entire collection in real-time.

How We Built It

We used Vertex AI and Gemini 1.5 Pro to respond to commands. We created a prompt using context data, including samples from every Firestore collection of the customer. We learned that using only the keys of the objects existing in the collection was good enough. We also added conversation history so the following commands or inputs have a context.

We placed the API behind a cloud function that responds in both streaming and synchronous modes. Consequently, results start appearing right away in streaming mode, creating the impression of blazing speed.

In the backend, we are using Cloud Functions and Firestore. For images and resources, we use Storage. And in the frontend, we have a React app hosted on Firebase hosting. We used TypeScript for coding on both sides, and in the backend, we created an express app. The generated scripts rely on the the frontend Firebase SDK.

The requests are authenticated with Firebase Auth, the same tool used for user management.

Challenges We Ran Into

Making it fast and optimizing performance was tricky. We had to reduce the number of examples and downsize the sampling object from the collections, eventually passing only the keys of the object in the prompt. Bigger prompts make the app too slow, so we have reduced them as much as we could. We cache the project context to gain a little speed.

We started building it with a configuration step where customers would give us permission to create a service account to manage several projects. We decided to remove that configuration step and simplify the demo submission process for a single project. We did build the code, using KMS to encrypt the service account keys and utilized Google Cloud APIs for all configurations, but it is not included in this demo.

Accomplishments That We're Proud Of

We are very happy with the first version of DataTalk and see huge potential in it. We have built a safe sandboxed environments that allows running scripts in the frontend, and we can provide the right UI widgets when the generated code is executed.

We were concerned about the limits of Gemini and how far we could push them. However, close to the end, we started performing complex queries akin to joining data scientist work. We aren't training neural networks, but we can perform many Business Intelligence requests on top of Firestore queries. We are also amazed at the quality of the code generated; with very few examples, we were able to run any Firestore operation.

What We Learned

We can teach Gemini a framework. Our frontend needs code to always return a default JS function to be executed and Gemini understands that perfectly.

You don’t need to build a prompt with all the data in the database to perform all kinds of CRUD operations. Initially we thought we would benefit from the 1 million token window, and thought we could throw many data samples for better context, but it was affecting speed. This might be possible in future versions of Gemini.

The size of the prompt affects drastically speed, in our first versions we had to wait up to 10 seconds for the first token to arrive.

What's Next for DataTalk

We have only implemented some interactions and have integrated the results into some specific widgets (like displaying collection data in an editable table). We could include forms for editing specific documents. We could connect to multiple databases, also different than Firestore. We could improve analytics results by displaying data in charts. We will allow to save and recover sessions, or extract script results into dashboards... Sky is the limit!

After the hackathon we will allow any user to use it in their projects within a managed service.

We feel this tool has great potential :)