Textoru

Screen app design - 07
Screen app design - 06
Screen app design - 02
Screen app design - 05
Screen app design - 01
Screen app design - 04
Screen app design - 03

Inspiration

People are still using handwritten documents in their daily life (Business meetings, Lectures, To-do lists, etc.). But these are easy to misplaces and the content is not that accessible and understandable. The contemporary answer is digitalization of this content.

Handwriting are proven to help people memorize the content better (1). Thus, even though apps exists, which try to answer the needs and requirements in this field, we believe that there is an area besides digitalization, that can be explored. We see this area as optimization of searchability and content-aware analyzation possibilities.

(1) Comparing Memory for Handwriting versus Typing, J. Smoker et al.

What it does

Textoru is an AI-webapp that optimizes handwritten documents, by allowing the user to digitize previously scribbled notes, and collect them in a database. This way, the user can research within their newly sorted collection of notes, and receive advanced search results from our AI.

The advanced search features are able to refer to both direct mentions of searched words and indirect references. For instance, being able to understand that when “my sister loves her dog, he is very caring”, the “he” refers to the earlier mentioned dog. This is called co-referencing. The content is also optimized by doing entity recognition of subjects in the text (people, companies, places, etc.), to provide more insight of the text, giving a better overview to the user.

These optimizations aids the user in being more efficient, effective, and engaged.

How we built it, and the challenges we faced along the way

In order to create this artificial intelligence, we`ve been through a design-process that has involved several exercises to help shape and built our platform.

We spent the entire friday evening trying to enhance our creativity by doing exercises that boost that aspect of the game. Once we felt confident that our minds were opened up enough, we started brainstorming and used our time coming up with project ideas for the different tracks and challenges. By the end of friday, we had at least 50 post-its and a bit of high level software architecture, to give an idea of subjects and areas that we found interesting enough to engage ourselves in.

We had a big meeting saturday noon, and figured that we’d go for the AI track, and try to make an analog text to digital converter, using a phone to take pictures of handwritten notes/documents. We started developing this platform on several different fronts; 2 were dedicated to make the AI algorithms behind the database storing all of this information, one were developing an android app that would hold the algorithm and send the pictures to our server. And some were also designing the interactions/UI of the app, and one would make a front-end website, that would send POST, and GET HTTP requests..

However, this all proved a major challenge, that would eventually lead us to stop developing the app to take and send the pictures for the server, since it was way too much work for just one person to do alone. Also the AI algorithms did a big number on us, having half of our group dedicating almost all of their time to make this part of our project work. Later on the day we would then send 2 people to go and research and interview some people in the hackathon, to get a better understanding, of when and where people e.g. students use notebooks, and write documents in the hand. We then made a persona and a use case scenario where we focused our energy on realising as much as possible from that vision. Saturday evening we designed the user experience of the “ideal app” (that would be the product we would have made if we had the time), and visualized the interaction with the app.

Sunday noon we finally had the front-end sync up with the database/server and AI we had built the day before, which was the entire problem that we were worried we wouldn’t get to finish in time. But it has been optimized and finished to an extend, where it is representing our platform in a way we deem fit.

Accomplishments that we're proud of

We believe that our product could be commercialized and that the features we defined bring a “plus” to the existing market of digitalization. We are happy to have worked on a proof of concept and case study, which let’s know how digitalization can afford improvement and optimization for the benefit of the user. In the short time and the given conditions, we are proud of having an AI and a search engine working and being able to show it.

What we learned

How to collaborate with people with various background and experiences and improved our own individual skills. We also trained our rapid product development skills and mental resolve in stressful conditions.

What's next for Textoru

We have a lot of new further ideas which could improve the user experience et make the product even more commercially appealing. Some part of the team may want to develop it further.

Built With

Submitted to

Junction Tokyo 2018

Created by

In the beginning Was helping on kickstarting our idea generation process, where we did various exercises to boost our creative mindset; thus helping us come up with ideas to what we could start making.

I was over the designproces, where i came with ideas and contributions to the UX and the UI.
I started the programming on the mobile app, which we later discarded, where I however learned a lot about json conversion to java, and communication with opencv.

Other than that, I was in charge of the presentation of our project, where I created slides, and presented the project for the judges.

Mathias Abdesakens
I basically worked on the search engine (Elasticsearch) and the backend of the web app.

Munkhdorj Baatarsuren
Hello world.
My main contributions are creating AI natural language understanding engine and backend, front-end and system design as well.
Our coreference resolution is based on Deep Reinforcement Learning for Mention-Ranking Coreference Models by Kevin Clark and Christopher D which I believe is a current state of art technology.

This was my first time leading a team in a hackathon and I've learnt a tons specially the time management.

It was a bit sad that our mobile app and front end was not finished completely so that judges didn't see our app's full potential even though the backend part was impressive as lot of other participants amazed by.

But overall I would say it was a beautiful experience.

Enod Bataa
I eat lunch
I worked initially with ideation and product development, making sure that we were on "the right track". After Enod proposed the idea that became Textoru, we worked to set up a backlog and overview of the development, that needed to be done.

I helped Manon with designing the integral parts of how we wanted Textoru to look, both with UX/UI and visualization.

When we finally went into development, I focused my attention on UI - but since we began too late with this, it weren't complete enough to present sunday.

Tobias Lund Petersen
deleted deleted