Sphinx and Ultrasphinx: And Eye on Search

Posted by Rob Kaufman
on Oct 15, 07

Sphinx Logo

Got into Sphinx this last week. It was cool to try the new Russian hotness of Rails search. The toughest part seems to be figuring out which of three Rails plugins to use. Your options are:

Ultrasphinx

This plugin by Evan Weaver seems very complete. It has field weighting and faciting, field merging and aliasing (allows you to have columns in associated models joined in the index). It is compatible with will_paginate, which is cool and has spell checking (always a plus for me).

ActsAsSphinx

Because ActsAs is cool ;-) This plugin doesn't generate the configuration files for Sphinx the way Ultra does, but if you decided you wanted to roll your own indexing config anyway it probably provides just enough with out being anywhere near as much code. Also it was this plugins main page that I used for installation instructions the first time. I'll put some below, but if you use these beware that they link to version 0.9.7-rc2 of Sphinx, which is not the most current stable version (and won't work with Ultrasphinx)

Sphincter This is Eric Hodel's entry into the Sphinx party. I'm not sure exactly how this compares to Ultrasphinx, and may try and get a project running with it in the future, just to know. It does look like testing the search portion of your code has been well thought out though, which I really like to see. Ok, the name grabs attention, giggle giggle whatever. Does a name like this make waves? Sure. Does it keep some people (especially clients) from liking it? Sure. I don't know if Eric thinks keeping the more PC people away is a feature or what, but to me picking a name just to be sensational isn't really worth much. I love puns and all, but it takes a way from people talking about the merits of your work.

Getting started is pretty easy (for OS X or Linux):

curl -O http://sphinxsearch.com/downloads/sphinx-0.9.7.tar.gz

You could also grab the latest from the Sphinx site. Replace the version number below if you do.

tar zxfv sphinx-0.9.7.tar.gz
cd sphinx-0.9.7

The next step is the configure. This needs to know where the libraries and header files for mysql live. Below is the line I used for OS X running a MacPorts based install of Mysql. We had some problems on another Mac getting to install with a Mysql installer based version, we finally ended up making a symlink in the /usr/local/lib and /usr/local/includes directory to /usr/local/mysql. The bottom line is that you'll need to change the two configure options to suite your install.

./configure --with-mysql-includes=/opt/local/include/mysql5/mysql --with-mysql-libs=/opt/local/lib/mysql5/mysql
make
sudo make install

Now we also need to install our chosen plugin into our Rails app. From your Rails root, using piston

piston import svn://rubyforge.org/var/svn/fauna/ultrasphinx/trunk vendor/plugins/ultrasphinx
cd vendor/plugins
ruby ultrasphinx/install.rb

Usage is also simple

All you have to do is open up your model and add an is_indexed line like this

is_indexed :fields => ['name', 'description']

A great thing about Ultrasphinx is that it is easy to get at multiple fields associated with a certain model. For instance this example has the relationships Bin has_many Services through BinsServices, Services has one Provider. We want the Bin name and description to be indexed, but we also want matches on Service name Service description and provider name to return the associated Bin. That ends up looking like this:

is_indexed :fields => ['name', 'description'],
       :include => [{:class_name => "Service", :field => 'description', :as => 'service_description', 
                     :association_sql => "JOIN (bins_services, services) ON (bins.id=bins_services.bin_id AND bins_services.service_id=services.id)"},
                    {:class_name => "Service", :field => 'name', :as => 'service_name', :association_sql => ""},
                    {:class_name => "Provider", :field => 'name', :as => 'provider_name', 
                     :association_sql => "JOIN (providers) ON (services.provider_id=providers.id)"}]

Notice that the second field for Service has to have its own line, and that the association_sql must be declaired, but be empty string (otherwise we JOIN twice and get an error). Also notice that the Provider JOIN starts off where the Service JOIN left off, it doesn't start at Bin, but goes from Service. It seems like this could be cleaned up a bit with some changes to the parser, but all in all it proved a workable solution for all the cases we had.

Next up we need to see what the search code looks like. We can just call Ultrasphinx::Search like so:

@search = Ultrasphinx::Search.new(:query => "My query string")
@search.run
@search.results

Now lets get the index run and the search daemon going. Note that these three commands can all be run at once with rake ultrasphinx:boot or its alias us:boot

rake ultrasphinx:configure
rake ultrasphinx:index
rake ultrasphinx:daemon:start

That about raps up our look at Ultrasphinx. One quick note, we ended up not using any of this code :-( The app in question is slated to run on Media Temple, and their grid structure precludes running background processes like this. We ended up moving everything to ActsAsSolr, a change that is worth looking at, but is the subject of another entry.

Comments

Leave a response

  1. Eric HodelOct 21 07 @ 06:23PM
    What's giggle-worthy or non-PC about sphincters? How do you think your food stays in your stomach?
  2. ScottOct 23 07 @ 11:28AM
    Yes, the body has many sphincters other than the anus. It's just a circular muscle ... get over it.
  3. NickOct 23 07 @ 12:05PM
    I don't see the file ultrasphinx/install.rb to run your instruction of "ruby ultrasphinx/install.rb"
  4. RobJan 16 08 @ 05:21AM
    Yeah, there has been a ton of trackback spam lately... guess I'll have to give up and disable them.
  5. MayankMar 26 08 @ 02:42PM
    @Eric Hodel -- IMO the biggest problem with the name is that it makes it difficult to find documentation on line. Have you tried googling "sphincter plug-in?" :-)
Comment