Inspiration

Through the hell-fire that was 2020, we are all looking towards the promise of a post pandemic era. Among the vast number of catastrophes many populations are facing, none are as daunting as the promise of isolation. For those of lucky enough to be trapped in with a family member or two, these effects may be more subtle, but for those seniors who are being forced to isolate in long term care homes, loneliness is affecting them more than any of us could imagine. In a society that runs rampant with ageism, we wanted to empower our seniors with connections beyond their grandkids. We drew inspiration from our own grandparents who's sweaters kept us warm when we needed them. Now it's our turn to help all those grandparents who need the warmth of connection to get through this winter of isolation.

What it does - Overview:

Knit is a social video chatting platform that creates spontaneous social situations that allow for candid conversation between two individuals with common interests.

When participants first enter the web-app, they will be asked to enter in a short sentence about what kind of conversation they are looking for. They can mention interests, hobbies, specific topics etc. This will be used to gauge what kinds of topics the participant would like to discuss. The free range in wording gives the participant more freedom to express themselves rather than to be constricted to narrow filters. We then utilize Facebook's wit.ai NLP (Natural Language Processing) to parse the data and extract the key words. These keywords are then put through our matching algorithm that will take in the key words and look for other users who share similar interests/goals that they would like to discuss about. Once a match is found, both users will be sent to a different page where they will be able to engage in candid conversation with their partner. After they end the session they will have the option of joining another call with the same interest filters, or they can go back to chose different topics of interest altogether. Through the straight forward design, any grandma can meet new people, at the click of a button.

How we built it

We went through many brainstorming sessions to help determine which tools would be the most appropriate in the construction of the app, taking into account it's nature. In the front end, we utilized React.js to construct an elegant and consistent UI, utilizing React Components to limit repeated code. The backend was constructed with JavaScript and all of our data was stored in a Firestore Collection which provided fast and efficient data transfer necessary to pair participants quickly. To process the incoming prompts and data, as mentioned above, we utilized Facebook's wit.ai api, and to run the Video calling service, we utilized Vonage's video service api. The addition of a video calling service was essential because it provided the most stimulating interaction which is essential for connecting with people. To host the application, we utilized Firebase hosting which was able to incorporate the database well.

A Deeper Dive into our Algorithm

After taking in a prompt from our participant, we funnel the string into FaceBook's wit.ai api that uses NLP algorithms to identify the key terms in the sentence, having trained a version of it to identify key words in a sentence. The root terms of these words are then extracted, removing any suffixes to create broader interest categories/topics through a process known as lemmatizing. Our algorithm then stores a "queue" of participants who are interested in certain topics. When a new participant completes the interests form, the algorithm searches the queue for a participant with a matching interest. If one is found, then they are given the option to join the room, or to go back into the queue. In the event that a match is not found in the queue, the participant and their interest will then be stored into the queue for a later matching. They could also choose to join a conversation with anyone else, regardless of interest (WIP). When a conversation is over, both parties rate the interaction from 0-10. High scores will result in the topic having a higher "weight" for the specific user. When finding another partner for a conversation, the match for the highest-weighted topic will be chosen. This helps us determine which topics from the user's list are likely to provide the best conversation experience

Below is a snapshot of our code that displays the queuing process:

async function joinQueue(userID){
    let user;

    try {
        store.collection("sessions").get().then((querySnapshot) => {
            querySnapshot.forEach((doc) => {
                user = doc.data()[userID];
            });
            return user;
        }).then(user => {
            for(let token in user.tokens){  //When adding user
                if(q.hasOwnProperty(token)){    //Match Found
                    let userID2 = q[token];
                    try {
                        store.collection("sessions").get().then((querySnapshot) => {
                                let user2;
                                querySnapshot.forEach((doc) => {
                                    user2 = doc.data()[userID2];
                                });
                                return user2;
                            }).then(user2 => {
                                let roomid = generateSessionName();
                                user.room = roomid;
                                user2.room = roomid;
                                user.state = 3;
                                user2.state = 3;

                                console.log(user);
                                console.log(user2);
                                console.log("Match Made! Room Code: " + roomid);

                                try{
                                    let obj = {}; let obj2 = {};
                                    obj[userID] = user;
                                    obj2[userID2] = user2;

                                    upd(obj);
                                    upd(obj2);
                                } catch (e) { console.log("Depositing data failed" + e); }
                            });
                    } catch (e) { console.log("Failed User2 + Generate Room ID " + user + " || " + e); }
                }
                else{   //No Match
                    q[token] = userID;
                    user.state = 2;
                    console.log("No Match for user: " + userID +  ", keyword: " + user.tokens[token]);
                }
            }
        });
    }
    catch (e) { console.log("Couldn't load User1 " + user + " || " + e); }
}

Challenges we ran into

The project that we decided to undertake proved to be a challenge especially considering the fact that 3 of us had very limited to no experience working with javascript apis and libraries. Some of the major challenges that we faced were:

  1. Connecting the React front end with the JavaScript backend Connecting the React front end proved to be quite a challenge. While working with Firebase in mobile application, the implementation of connecting a Firestore collection with a web application proved to be a much greater challenge. After scouring through the documentation we were able to fix many of our bugs, and connect our database to our main application.

  2. Development of our matching algorithm Developing in a relatively new language proved to be quite a challenge. There was a point where our algorithm was not matching two identical words. After breaking down the problem, analyzing the return error messages, and way too many console.log()'s than we'd care to admit, we were able to pinpoint the bug and fix it, improving our algorithm.

Accomplishments that we're proud of

As the saying goes, you can't make an omelette without breaking a few eggs... in this analogy our team's total sleep time was the eggs. Despite the inevitable loss, we were able to power through and create some great features of the app. We are very proud of our ability to manage and create different rooms through the server side application as well as construct an algorithm using javascript apis. Despite the hardships we persevered and were able to learn a lot out of this experience. This was also one of the first time's we tried implementing an "AI" mimicking model to learn associations.

What we learned

This was a common theme throughout the hack. All weekend our team was learning new things, whether it may be a new api, a new language, or a new framework. Our team got the experience working on a React app which we did not have much familiarity with and were still able to complete key components of it. Along with this, we worked with brand new tech stacks, and took different approaches to solving our problems. One thing that we learned from this experience is that even when faced with something completely new, it is still possible to succeed.

What's next for Knit

In pursuit of our ultimate goal we have a few features that are yet to come, here are a few:

  • Device accessibility: Not everyone has access to a computer, so we want to make this application more mobile and tablet friendly to widen our reach, and connect more people.
  • Voice controls: Despite the simple interface we have now, we want to make the process even simpler by introducing voice controls. This will limit the device interaction which can often fluster those who are unfamiliar with it.
  • Additional processing of data after it is stored: To improve the quality of the conversation and matching between people, we are hoping to later implement an analytical aspect which will evaluate what conversation topics stimulate the most conversation.
  • Creating conversation starters: Meeting a new person can be tricky, even with common interests, so we want to help make the conversation go as smoothly as possible.
  • Voice recognition to power conversation starters: Building off the previous goal, we aim to use voice recognition to discover which questions are asked the most often and lead to highest conversation score. These could then be provided either to people with topics (where people with that topic generally ask the question a lot, ex. "Are you excited for the Olympics" for sports, etc).
  • Automated moderators: We understand that there are many opportunities for such a platform to be misused, so we hope to add moderators to calls to prevent such misuses of our application.
Share this project:

Updates