The sentence undestood and the linked search results
A search result for how to draw a circle
A search for a shadow effect
As we are coding, playing or doing any activity that involves retrieving an information quickly, we sometimes don't have the time to look it up ourselves while still obtaining a consistent answer concerning the current context of action. We had the idea to provide an application that allows quick information retrieving using vocal keywords, direct the search to user-defined contexts and displays relevant information on a screen at the end, enabling easy information access.
What it does
- The app waits for the user vocal input
- When speech is being detected, processes it and queries API and search pages linked to the context of where the user currently is.
- Such contexts are determined according to which app the user is currently running.
- A search is performed on the sources linked to the context. For example, the source Stack Overflow is queried when the user is on their IDE.
- The result of the search is displayed on a page listing all the matches found in the context.
How we built it
Using Electron, we capture the micro of the user, send it to wit.ai which identifies the intent of the sentence based on our training and send it back to the client. We then use react to render the data, which consist either in a HTML potentially containing markdown containing code (which is pretty-rendered), or HTML subsets of a given page, given a selector (useful for sources that don't have an API).
Challenges we ran into
We spent our time on three main points:
- The evaluation of the technologies, the design of the architecture and how to implement it.
- The training of the AI and its intent comprehension.
- Of course, the implementation of the structure, design and learning of the different technologies.
Accomplishments that we're proud of
We are proud to have built such an app within the short interval that is the hackathon.
What we learned
We didn't know wit.ai before, neither tools to allow speech recognition.
What's next for Solus
There are a lot more features that can be added to the project, such as an extended configuration of each source and ways of parsing, history of the old results retrieved and an improvement on the search process.