Almost every company requires their employees to track time. Maintaining time sheets can be a pain, but fortunately a few years ago we came across Harvest.

It's a great tool, used by over 50,000 businesses worldwide, that allows you to track time as you work. It's way more efficient than filling time sheets later.

Recently, we started working from home more and more often. We joined over 4M employees in the US, who use latest technology to save time spent on commute and have more time for their families.

While it's great to save time, working from home also brings a lot of distractions. You know, all these moments when your dog wants to go out for a walk, your child needs attention or a FedEx courier rings at your door.

With such distractions, it's hard to keep accurate time entries, even in Harvest. A lot of people forgot to stop their running timers and realize that while away from their computer. To be fair with their employers and clients, they have to correct their time entries later.

Was I off for half an hour or more?

Wouldn't it be great if you could control time tracking from anywhere in your house?

"Alexa, tell time tracker to stop billing"

"Alexa, tell time tracker to start tracking project management for Upside"

Just tell it what you're doing and it will be reflected in your time sheets.

What it does

In our project we focused on three most important tasks that people use time tracking software for. That is:

  • starting tracking time for a task, as they get down to work
  • stopping, when taking a break or when they're done for the day
  • getting a summary of today's time entries

Our goal was to deliver the best possible experience for these core features.

How we built it

We contacted Harvest to ask if they're fine with us building an Alexa skill that would integrate with their product.

They loved the idea!

Designing the skill was the most important part

We built personas that would help us better understand the needs of the user and how they may interact with voice interfaces.

Our major persona was called Jane.

Jane works in marketing in a mid-sized agency. She lives in Midwest and regularly works from home.

Each morning, she makes herself a cup of coffee, sits on her couch and starts work. Sometimes she works without a computer, especially when writing articles. Using pen and paper helps her avoid distractions such as Facebook.

She often switches between projects for her company's clients.

She likes to do a bit of housework during her "working hours", to clean her mind before diving into creative work. She also has a dog that sometimes comes to her with its leash, waiting to go for a walk.

This gave us a sense of what features we should focus on and how they may be used.

Based on our experience with VUIs, we knew that simplicity is a must in such applications.

Without graphics that support conventional user interfaces, we needed to avoid complex dialogues and make interactions with our skill as simple as possible.

We prepared a large list of samples of how users may ask our skill to perform a task.

They were really similar to asking or telling someone what you're doing right now.

Then we moved into implementing the skill

We started with a simple proof of concept, written in Python and hosted using AWS Lambda. That was the moment when we discovered various challenges that we needed to overcome in order to build a fully functional skill (more about challenges in the next section).

After we got it running, we needed to build a layer for authenticating with Harvest and allowing to save user's preferences. At this point we needed to migrate from AWS Lambda to our own HTTPS service.

Fortunately, we were able to reuse all the code written before and within an hour we had our skill working in a service based on Django.

Testing the product

With fully functional product, we wanted to verify our assumptions so we tested it with a few Harvest users.

First thing that we noticed was that our skill missed a few sample utterances that people tried to use. Our guesses there were not 100% correct.

We also confirmed our assumption that the tasks that people ask our skill to track may differ from what's in Harvest. Good thing was that we prepared for it in advance.

Challenges we ran into

To start tracking time, user needs to provide name of the task and name of the project. Available tasks and projects are defined in Harvest.

One of the biggest challenges was dealing with situations when user's input differs slightly from what's in Harvest. When testing with users, we confirmed that people often shorten some names when asking Alexa.

They'll say "management" instead of "project management". They'll say "Facebook" instead of "Facebook Inc.".

Alexa has great capabilities when it comes to matching inputs to available options, however they work only when the list of options is constant.

For us, every Harvest user may have a different set of options available to them. In order to account for that, we needed our own algorithm for recognizing similar words.

Another challenge that we ran into was more technical, related to how Alexa's and Harvest's APIs are built. In order to start a time entry in Harvest API, we need to know user's timezone. Currently, there's no way to pull this information from Alexa's API (although looking at how fast Amazon adds new features to Alexa, it's a matter of time).

To get this information, we decided to ask user for their timezone during account linking process.

Accomplishments that we're proud of

We're really proud of how our system handles user's inputs.

Besides handling project and task names that don't exactly match what's in Harvest (see the section above), we were able to make the best out of Alexa's dialog model for prompting for missing information.

When one of the inputs is missing, we move from command-like interface into a simple dialogue.

[Alexa] Sorry, could you repeat thet name of the task


[Alexa] And what project do you want to start billing?

This kind of dialogue makes the interaction feel natural, like conversation with another person.

What we learned

Building Harvest for Alexa skill gave us a chance to experiment with latest Alexa capabilities (Dialogue delegation turned out to be an especially useful concept).

We had a chance to refresh our NLP knowledge.

We were also surprised by how little time it took Amazon to review our skill (and how constructive their feedback was).

What's next for Harvest Time Tracker for Alexa

We've just launched the skill for everyone. Our current goal is to get more Harvest users to give it a try.

We also have a few new features in mind. The most interesting ones are:

  • Recognizing synonyms in task/project names (sometimes people say "programming" instead of "development")
  • Adding push notifications to let users know that they've been working for too long (and help them avoid overtime)
Share this project: