We saw the implementation of both content-based recommendation and also recommendation based on collaborative filtering. In this chapter, we will discuss some more tricks that add more product search feature spice to the overall user experience. We will:
- Add a product search feature
- Add a recommendation listing
- Understand recommendation behavior
You may have noticed that with millions of users, and thousands of products, MongoDB is considerably slow for querying data on a single machine. Another point to note is that it stores most of the data which we may not want to query. And, to top it all, we want to be able to make free-text queries. For such a use case, an inverted-index based search engine is most appropriate. Please see this discussion for more details: https://stackoverflow.com/questions/12723239/elasticsearch-v-s-mongodb-for-filtering-application.
Elasticsearch is one of those search engines that we can conveniently set up, code and use in just a matter of hours. So, let's set it up.
First we download the ZIP archive:
$ wget -c https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.4.4.tar.gz $ tar zxf elasticsearch-1.4.4.tar.gz $ cd elasticsearch-1.4.4
We chose version 1.4.4 here, however you could choose a later version too.
Next, we will install a very handy plugin called head that will help us visualize the search indices and make queries.
$ bin/plugin -i mobz/elasticsearch-head
We are all set. Let's start the Elasticsearch server:
$ bin/elasticsearch
Now visit http://localhost:9200/_plugin/head/ to monitor your Elasticsearch collections and indices:
We have set up the Elasticsearch server, but it doesn't do us any good until we feed it some data. The product search feature will allow us to search through the title, group or categories of a product. This means we need to be able to add all these attributes into an Elasticsearch index. Once we have done that, we can query for products matching a criteria.
First, let's write some Scala code to push products data into the search server. Recall that we created a file called datasets/content-based-dataset-new.csv, for our content based recommender example. We will reuse that file to load product data. We can also read from MongoDB, but that's not the point. We are trying to quickly populate data into the search server. This code is implemented in the chapter07.ImportElasticSearch Scala object.
Now let's run the importer:
$ sbt "set fork := true" "run-main chapter07.ImportElasticSearch"
Once the import is complete, you can open Elasticsearch web UI, and explore the index. Open the UI:
Then go to the Browser tab and select products_index.
So far, so good. Now we must integrate the search server from your web application. For that we will use elastic4s Scala library to make queries. For that, we have to follow these steps:
Update build.sbt with the following line:
libraryDependencies += "com.sksamuel.elastic4s" %% "elastic4s" % "1.4.14"
Next we create Scala object ESClient, with a utility method to search for user queries:
Next we write update HTML, and some JavaScript in templates, and fetch search results using the SearchController class, which is routed to /search/ path (see the webapp-recommender/conf/routes file):
Finally, our search feature is working! Try searching for mount Kilimanjaro, which will probably show your following output:
This implementation is far from what you will use in a real website. However, it is an essential feature of most of the inventory-based websites, where you need to provide some way of narrowing down the search. Of course, this is minimal, but it will give you a head start on what technologies and glue code you will probably need. You may also want to make it much prettier. Next up, we will use our ALS model we created in the previous chapter. We will add recommendations to an individual customer's page.