Laguna

Inspiration

We heard a rumor that if you access a travel site from a a different country, it's often cheaper. Thinking this couldn't possibly be true, we tried a bunch of different combinations of flights using a VPN till finally we found an example where buying a flight from Colombia was $200 cheaper That's insane. We didn't want to tolerate that sort of abuse.

What it does

Laguna books your travel in a way that searches from every country. It finds the best deal on a trip not just from your own country, but from every country we could think of to end country based price discrimination. Moreover, we automatically deal with currency fluctuations so that if an airline doesn't update their price when a currency decreases in value you can take advantage of that.

How we built it

The primary user interface is a web app which is built in simple JavaScript/HTML/CSS. Where the magic happens is under the hood:

We wanted to allow users to type in airport codes or city names and get accurate predictions so they don't need to type in the whole thing. We looked into using an API for this, but found that when I type in a single letter, the APIs don't intelligently rank them–-it's often just alphabetical. We created a proprietary algorithm that ranks each prediction based on where the matches occur (the IATA, ICAO, or airport name) and the popularity of the given airport. This whole process was done by constructing tries for each parameter, searching each trie, and then ranking the results across each trie. This whole algorithm is super fast––noticeably faster than many website's own systems and works very well.
The data. We needed a way to access each country. Originally, we actually distributed 14 servers to handle 14 countries, but eventually discovered that Kayak allow us to just pass in a header value to change the region. But, we still have to find each country that exists and do currency conversion back to USD which required building a decently large DB––so lots of scraping.
We needed to be able to wait for JavaScrip to execute t on the Kayak site instead of just downloading the HTML (see challenges). We ended up using a modified version of Electron to do so.
We utilized micro-frameworks so that anything can easily be factored out and replaced. It's a fully sustainable code-base.
Sockets. Searching in so many countries is slow and would provide a bad UX if we waited till everything was done. Sockets let us push everything out quickly and magically.

Challenges we ran into

We wanted to deal with as many countries as possible which led us to make the choice to use Kayak as one of our main sources of data. However, it turns out Kayak operates a different website entirely for each country down to the structure of the HTML. We had to figure out a way to still take their data in a scaleable manner. Moreover, Kayak updates their webpage live and doesn't have a cross-country JSON API (we reverse engineered their iOS API, but it only worked for the United States), so we had to emulate a browser and run full JavaScript to scrape their data––and the only way to know when Kayak was finished searching was observing UI changes the same way a user does.

Accomplishments that we're proud of

Companies don't want us to take their data, but we did anyway. We also architected the system to use micro-frameworks and be distributed across a series of servers so that our project is an actually scalable hackathon project.

What we learned

We learned that sites can shock you in the greatest of ways. We expected to just reverse engineer an iOS API and call it a day on data acquisition, but no. We needed to adapt to using a different website for each country...