We use chats a lot and some of them are 100-200 people big. You have to deal with them because you don't usually know who would jump in into conversation. But when two or more conversations start simultaneously - better not discuss.
What it does
We decided that we can tackle it by intelligently detecting this conversations and moving them to separate threads before it is too late. Our app also show some monitoring info-graphics.
How we built it
We use Django and TwistApp API to provide experience completely embedded into the Twist itself. Our clustering algorithm relies on user mentions and message semantic similarity (using NLTK, WordNet and SciPy). We use force simulation from D3.js visualization lib to show reasoning of the algorithm in real time.
Challenges we ran into
You know, error messages are often not so clear as we expect. We ran into some weird thing in Twist API and D3 and even had to look into code when documention was not enough.
Accomplishments that we're proud of
We did it! And also devised pleasant UX and good heuristics for similarity detections.
What we learned
How to use all those APIs, how to cluster data based on distance matrices. How to compare unstructured mess that is called "chat messages"
What's next for TalkFork
We gave Doist feedback on API, asking them to allow integrations show their own UI and operate on messages more free. These would allow us to provide more streamlined experience. Also we see some ways to improve our detection algorithm relying on sentence structural similarity.