CSC/ECE 517 Spring 2015/ch1b 18 AS

Apache Solr and Rails

The topic write up for this page can be found here.

Apache Solr<ref>http://lucene.apache.org/solr/</ref> is a standalone, open-source enterprise search server created by Yonik Seely. It has a REST-like API. It is a indexing and searching framework which could be deployed and used with many web frameworks like Rails, Drupal, Django etc. Indexing could be done using JSON, XML, CSV or binary over Hyper text transfer protocol. It could be then queried using HTTP with a GET method and receive the JSON, XML, CSV or binary results. It is a popular, scalable, blazing-fast, open source enterprise search platform built on Apache Lucene search library. Websites using Rails can take advantage of the Solr search engine to provide sophisticated and customizable search features. Ruby/Rails integrates with Solr search server using Sunspot<ref>https://rubygems.org/gems/sunspot_rails</ref><ref>https://github.com/sunspot/sunspot</ref> library and does a full text search.

Introduction

Apache Solr is a standalone enterprise search server with a REST-like API. Indexing could be done using JSON, XML, CSV or binary over Hyper text transfer protocol. It could be then queried using HTTP with a GET method and receive the JSON, XML, CSV or binary results. It is a popular, scalable, blazing-fast, open source enterprise search platform built on Apache Lucene. Websites using rails can take advantage of the Solr search engine to provide sophisticated and customizable search features.

Technology Stack<ref>http://www.slideshare.net/dkeener/rails-and-the-apache-solr-search-engine</ref>

Solr is built on top of Apache Lucene. It is a toolbox responsible for indexing, searching, spell-check and advance tokenization whereas Solr is a search server which inherits all the features of Lucene but also adds API integration, caching and most importantly a web admin interface. This feature makes Solr very easy to use in production environment. Both of these technology needs Java 1.4 or above.

Sunspot library interacts with Solr using a low-level interface called RSolr. It is a ruby client which integrates the Solr API's to Rails through the use of Sunspot gem. Sunspot has a drop-in ActiveRecord support. We can call Sunspot as the high level client of Solr. We will be talking about the installation and application of Solr using Sunspot gem in the section below.

Features<ref>http://lucene.apache.org/solr/features.html</ref>

Apache Solr has the following features

• Enables full-text matching, powered by Lucene software
• Built to handle high volume traffic
• Provides faceted searching
• Provides features like suggester, spellcheck, clustering, auto-complete, highlighting
• Extensible through plugins
• Supports statistical and aggregate processing of text
• Supports rich format data such as PDF, Word, Powerpoint

Rails uses the Sunspot Ruby library to integrate with the Solr search engine.

Installation

Sunspot makes it easy to do full text searching through Solr. Sunspot comes as a gem and is installed in the usual way by adding it to the Gemfile and running bundle.

gem 'sunspot_rails'
gem 'sunspot_solr' # optional pre-packaged Solr distribution for use in development

bundle install

Once the gem and its dependencies have installed we will need to generate Sunspot’s configuration file which we can do by running

$ rails g sunspot_rails:install

This command creates a YML file at /config/sunspot.yml. We don’t need to make any changes to the default settings in this file. Sunspot embeds Solr inside the gem so there’s no need to install it separately. This means that it works straight out of the box which makes it far more convenient to use in development. To get it up and running we run

$ rake sunspot:solr:start

If you’re running OS X Lion and you haven’t installed a Java runtime you’ll be prompted to do so when you run this command. You may also see a deprecation warning but this can be safely ignored. The command will also create some more configuration files for advanced configuration.

Usage and Examples<ref>http://outoftime.github.io/sunspot/docs/index.html</ref>

Indexing Objects

Add a searchable block to the objects you wish to index.

class Example < ActiveRecord::Base
  searchable do
    text :title, :body
  end
end

Text fields will be full-text searchable. Other fields which are outside the scope of searchable can be used to scope queries.

Searching Objects

Now searching can be done simply by passing the query to search method in the respective class.

Example.search do
  fulltext 'query'
  with :conditions
end

We can use many variations on the search now by changing the scope variables in the query.

Example of a Blog Search

Here we want to index the text fields. So, it is kept inside the searchable block.

class Blog < ActiveRecord::Base
  searchable do
    text :title, :body
    text :comments do
      comments.map { |comment| comment.body }
    end

    boolean :featured
    integer :blog_id
    integer :author_id
    integer :category_ids, :multiple => true
    double  :average_rating
    time    :published_at
    time    :expired_at

    string  :sort_title do
      title.downcase.gsub(/^(an?|the)/, '')
    end
  end
end

Now for searching the indexed objects we can use:

Blog.search do
  fulltext 'best author'

  with :blog_id, 1
  with(:published_at).less_than Time.now
  order_by :published_at, :desc
  paginate :page => 2, :per_page => 15
  facet :category_ids, :author_id
end

Here the text fields are full text searchable and other attributes like blog_id, page, author_id is used to scope the query.

Extensions to Sunspot

Though Sunspot is used primarily for indexing and searching, it could be further extended to support multiple features. Some of them are listed as:

1. Scoping scalar fields- We can put Positive and negative restrictions along with conjunctions or disjunctions to scope the query.

2. Pagination- All results from Solr are paginated.

3. Faceting- Faceting is a feature of Solr that determines the number of documents that match a given search and an additional criterion. This allows you to build powerful drill-down interfaces for search.

4. Ordering- By default, Sunspot orders results by "score": the Solr-determined relevancy metric. Sorting can be customized with the order_by method.

5. Highlighting-Highlighting allows you to display snippets of the part of the document that matched the query.The fields you wish to highlight must be stored.

6. Hits vs. Results-Sunspot simply stores the type and primary key of objects in Solr. When results are retrieved, those primary keys are used to load the actual object (usually from an SQL database).

7. Reindexing Objects- Objects are automatically indexed to Solr as a part of the save callbacks.

<ref>http://tech.favoritemedium.com/2010/01/full-text-search-in-rails-with-sunspot.html</ref><ref>http://www.linux-mag.com/id/7341/</ref>Other interesting reads on the topic and referral links are mentioned below.

References

<references/>