Inspiration

The area of the Quality of the Qualitative research is in higher demand with the growth of communication mechanisms. But lack of tools available to simplify the analysis to get deeper insights.

What it does

This component is built to support thematic clustering in qualitative research where you can cluster large data sets like text reviews or chat histories in to certain clusters and analyze further. We employ TiDB serverless vector search feature to find similarities. Using a feature that is inbuilt to DB itself not only simplify the process but also ensures privacy and limit the dependencies to 3rd party AI tools.

How we built it

Its a SPA (Single Page App) built with AngularJS front end and NodeJS backend that it connecting to TiDB serverless. End user app is deployed to azure app service and live at the moment.

How it works

Reviews sending via the UI are passing to NodeJS api. Review will then pass to OpenAI api and converting to vectors within the api layer. Vectors are saving in TiDB serverless along with other review data sending through. Vector Search can be perform against saved vectors and search cluster vector to find the similarities.

Challenges we ran into

Had to depend on 3rd party to Convert text to vectors. TiDB vector data type is not recognized in mysqldump utility. Therefore need to cerate custom MySQL backup scripts when backing up data.

Accomplishments that we're proud of

We made a pretty decent component that is working and also hosted to test live

What we learned

AI can be further simplified and embed in to your DB itself as a private language model

What's next for CrowdVibe

This is planning to integrate with our main research platform

Share this project:

Updates