Inspiration

http://i.imgur.com/7jo97un.gif

The Devpost search is an ongoing project of mine that I've had a lot of fun building (even though it has No Business Value™)

In the past, when browsing through Devpost, I was very frustrated by the fact that I couldn't search for projects. I wanted to...

  • find projects built with my favorite technos (#elasticsearch!)
  • search through the project of the people I was following
  • find recent hackathon winners with videos
  • explore the most popular projects posted this week
  • and much more...

So I set out to create the best search I could.

What really got me started was the release of the new search feature of Trello.
It was fast, introduced useful search operators, and was just fun to use. I wanted Devpost to have a similarly badass search.
I also took some ideas from the facebook search, as well as GitHub's and Evernote's for the search operator's syntax.


How it works

Search operators

Operators are a huge part of what makes the CP Search fun.
You can find a complete list of all the supported search operators here: http://devpost.com/#search/help
Here are my favorites.

#tag

A classic. Search projects built with your favorite technos, apis, whatever...

http://i.imgur.com/RBHCBXd.gif

@following

Searches through the projects of your friends.

has:video

I really like this one in combination with the at: operator.
at:"Product Hunt Hackathon" has:video will allow you to watch all the videos made at the Product Hunt Hackathon for instance.

published:date

I find it very useful to keep up with all the cool projects published on CP.
published:week is:popular will show you the most popular projects of the week for instance.


Most of the search operators are highlighted: http://i.imgur.com/d7ttnR8.jpg

Suggestions

Gives the user ideas about interesting queries. A possible improvement would be to make them dynamic, and recommend searches based on the recent hackathons.

http://i.imgur.com/oQQ1Kjy.gif

Filters

Filters are just a way to make using operators a bit easier by allowing the user to pick from the most popular ones.

http://i.imgur.com/WIGlarV.jpg


Challenges I ran into

Javascript

I first started to build the search Javascript without any structure, using jQuery here and there. As I was adding features, I realized it wasn't going to work and started using Backbone.js. It really helped me write better and more structured code.

Next step, borrowing some code from Marionette.js, and using Backbone.radio to handle events.

Parsing search operators, with Parslet

My first approach to handle these was simplistic and regexp-based. It worked well for a while, but I quickly reached its limits when I decided to add more complex search operators, like published:week likes:>5.

Enter Parslet.

Parslet makes it dead simple to create parsers. So I basically just had to write the grammar defining a search query.

The issue is that a Parslet parser returns huge, nested hashes when parsing a complex query like '@"robin" #ruby is:winner published:year-1..2014-01-01'.

[
 user_operator: { string: "robin" }
 tag_operator: "ruby",
 is_operator: "winner",
 date_operator: {
   field: "published:",
   comparator: {
     comparison: {
       operator: nil,
       comparable: {
         date_range: {
           lb: {
             relative_date: {
               relative: "year",
               value: {
                 number: {
                   sign: "-",
                   natural: "1"
               }}
           ub: { absolute_date: "2014-01-01" }
  }}}}
]

Thankfully, they thought of everything, and Parslet comes with a hash transformation engine (Parslet::Transform). It provides a mechanism for simplifying and interpreting these nested hashes easily.

All in all, Parslet has been a crucial piece in the implementation of complex search operators, while keeping the code clean, easy to understand, maintainable and extensible.


Accomplishments that I'm proud of

I'm very happy with how this project turned out. It's fun to use, powerful, polished and full of details that are usually absent from other features that we have to release fast.

I also really like the fact that it makes other part of the application simpler by providing a "text api" for searching projects.

Instead of

Software.search do
  with :member_ids, current_user.followed_user_ids
  with :tags, ["ruby"]
  order_by :popularity
end

we can just do

SoftwareSearch.query("@following is:popular").tags("ruby")

What I learned

I learned a lot about Backbone.js, and Parslet.

What's next for Devpost Search

Moving from Solr to ElasticSearch. One day, it'll happen.

Built With

+ 2 more
Share this project:
×

Updates