LastMile

A voice-first Android assistant that finds real nearby businesses and with your permission, calls them to ask about what you need.

Comment

Inspiration

LastMile came from something pretty simple.

If I need something from a shop or pharmacy, I can search online and find places near me. But that still doesn't tell me whether they actually have what I need.

I still have to pick a place, find the number, call them and ask.

I kept thinking, why can't I just tell my phone what I'm looking for and let it help me do that?

That's where Miles came from.

What it does

LastMile is an Android app with a voice assistant called Miles.

You can talk to Miles and tell it what you're looking for. It uses your location and Google Places to find real businesses nearby and shows you the results.

For example, I can say:

"Hey Miles, I need CeraVe cream."

Miles can find nearby pharmacies.

But I didn't want Miles automatically calling every place it finds. The user has to choose who gets called.

So I can say:

"Call the first one."

That business is then passed to the CALL-E calling system together with what I originally asked for.

If that place doesn't have what I need, I can tell Miles:

"Try the next one."

Then Miles can move to the next result.

The idea is that I shouldn't have to search, copy phone numbers and make the same call several times myself.

How we built it

I built LastMile as a native Android app with a Python backend.

The Android app handles the phone side of things, including the microphone, location permissions, GPS and the Miles interface.

I used Google Places for finding real businesses instead of filling the app with fake sample stores.

The Python backend handles things like the request, conversations, business results and the logic around what Miles should do next.

For the actual calling part, I connected LastMile to CALL-E. There is also a secure gateway between LastMile and the calling service.

I built a browser version too, which has been useful for testing and for letting people try LastMile without installing the Android app.

The basic flow ended up being:

Voice request → location → find businesses → choose a business → call → get the result back

Challenges we ran into

Android permissions gave me quite a bit of trouble.

Miles needs the microphone and location for some of its main features, so I had to make sure a new user is actually guided through giving those permissions instead of just opening the app and finding that things don't work.

Getting all the different pieces to work together was another challenge. Voice recognition, GPS, Google Places and CALL-E are separate systems, but from the user's side they're supposed to feel like one conversation with Miles.

I also had to decide what Miles should be allowed to do automatically.

I didn't like the idea of someone asking for a product and Miles immediately starting to call businesses without asking them. So I made searching and calling separate.

Miles can find the places, but the user decides who gets called.

Testing the calling part is also tricky because a passing automated test doesn't mean somebody actually answered a phone call. Some parts of LastMile have to be tested in the real world.

Accomplishments that we're proud of

Getting the Android app working was a big one for me.

I didn't want LastMile to end up as just another chat box where you type something and get an answer back.

Miles can listen to a request, work with the phone's location, find actual businesses and then move into a calling workflow when the user asks it to.

I also like that the user stays in control of the calls. That became an important part of the project as I was building it.

And having both the Android app and a browser version means I can work on the mobile experience without making the project impossible for someone else to quickly try.

What we learned

One thing I learned while building LastMile is that finding information and actually helping someone finish something are very different problems.

If I ask where I can buy something, giving me five nearby stores is useful.

But I might still have to call all five.

That's the part I became more interested in while working on Miles.

I also learned how much more complicated an assistant becomes once it starts interacting with things outside the app. Location permissions matter. Microphone permissions matter. Calling the wrong place matters.

So giving Miles more ability also meant thinking more about when it should stop and ask the user first.

What's next for LastMile

I want to keep pushing Miles beyond business search and phone calls.

There are a lot of everyday tasks where I know what I want to happen, but I still have to open several apps and do all the small steps myself.

That's eventually where I want Miles to fit.

I want to be able to tell it what I need, let it handle more of those steps, and only come back to me when it needs a decision or permission to do something.

There is still a lot to build before Miles gets there, but LastMile is the start of that idea.

Built With

Share this project:

Updates

Submission history