Lexi

• 

  Lexi Logo

Inspiration

Vivian and I (Sanjana) both come from immigrant families, and being born in the US, we have always struggled with practicing our native languages. We built Lexi as a better way to practice writing our native languages, as an alternative to other platforms such as Duolingo or Babbel.

What it does

Lexi is a Snapchat Spectacles Lens. It listens to the users voice, translates their words from English to a target language, and then displays those characters in an augmented reality 3D space. Users can then trace over the words in order to practice their writing skills, and they can also generate new words by speaking new phrases. Lexi's strength over other platforms is that it allows the user customization over what they are learning-- there is no rigid structure to the platform, and the user can practice whatever words or phrases which they wish. However, this also means Lexi is better suited for people who have prior knowledge of a language and wish to practice it; or, Lexi can be used in conjunction with another language platform.

How we built it

We built this project using Snapchat Lens Studio, which is their platform for creating Lenses that work with Spectacles. In terms of tech stack, it mainly contained assets, scripts (written in TypeScript and JavaScript), and we also attempted to connected to a DeepL API service. Other tools we used were VSCode, and Git for version control.

Challenges we ran into

Some of the challenges we ran into were due to the fact that Spectacles is a relatively new product, so their documentation is sparse and unhelpful at times. For example, we were unable to use their in-built service to make http requests for API requests, but we had followed the documentation exactly, and only replaced the url. Another main challenge was that we were completely unfamiliar with the platform, and there is a steep learning curve to it, but luckily both of us had prior experience with Unity (a similar platform for game development).

Accomplishments that we're proud of

We are proud of ourselves for staying motivated and pushing through the learning curve of working in a new field (AR) with new technology (Spectacles), and being able to deliver a working product that can effectively translate and has a function to draw.

What we learned

We learned how to be more adaptable-- How to not burn out while debugging for hours-- How to stay resilient-- And on a more serious note, a big takeaway for both of us was learning and getting more familiar with the differences between TypeScript and JavaScript, and how they are used for scripting in AR.

What's next for Lexi

In the future, we would love to improve upon this MVP for Lexi. Currently, Lexi only supports English to Mandarin translation, but we would want to add more language support. Moreover, in languages such as Mandarin, Hindi, and many more, the stroke order matters, so we would like to incorporate that by maybe displaying the strokes one at a time, instead of displaying the whole character. A big feature we want to add is giving the user feedback on how well they are drawing their characters, so that they can use that to improve. Lastly, we would love to improve the drawing capabilities and overall improve the UI and user experience of using our product!

Built With

• deeplapi
• git
• javascript
• snap-lens-studio
• typescript
• vscode