Sphinx and Ultrasphinx: And Eye on Search

Posted by Rob Kaufman
on Oct 15, 07

Sphinx Logo

Got into Sphinx this last week. It was cool to try the new Russian hotness of Rails search. The toughest part seems to be figuring out which of three Rails plugins to use. Your options are:

Ultrasphinx

This plugin by Evan Weaver seems very complete. It has field weighting and faciting, field merging and aliasing (allows you to have columns in associated models joined in the index). It is compatible with will_paginate, which is cool and has spell checking (always a plus for me).

ActsAsSphinx

Because ActsAs is cool ;-) This plugin doesn't generate the configuration files for Sphinx the way Ultra does, but if you decided you wanted to roll your own indexing config anyway it probably provides just enough with out being anywhere near as much code. Also it was this plugins main page that I used for installation instructions the first time. I'll put some below, but if you use these beware that they link to version 0.9.7-rc2 of Sphinx, which is not the most current stable version (and won't work with Ultrasphinx)

Sphincter This is Eric Hodel's entry into the Sphinx party. I'm not sure exactly how this compares to Ultrasphinx, and may try and get a project running with it in the future, just to know. It does look like testing the search portion of your code has been well thought out though, which I really like to see. Ok, the name grabs attention, giggle giggle whatever. Does a name like this make waves? Sure. Does it keep some people (especially clients) from liking it? Sure. I don't know if Eric thinks keeping the more PC people away is a feature or what, but to me picking a name just to be sensational isn't really worth much. I love puns and all, but it takes a way from people talking about the merits of your work.

Getting started is pretty easy (for OS X or Linux):

curl -O http://sphinxsearch.com/downloads/sphinx-0.9.7.tar.gz

You could also grab the latest from the Sphinx site. Replace the version number below if you do.

tar zxfv sphinx-0.9.7.tar.gz
cd sphinx-0.9.7

The next step is the configure. This needs to know where the libraries and header files for mysql live. Below is the line I used for OS X running a MacPorts based install of Mysql. We had some problems on another Mac getting to install with a Mysql installer based version, we finally ended up making a symlink in the /usr/local/lib and /usr/local/includes directory to /usr/local/mysql. The bottom line is that you'll need to change the two configure options to suite your install.

./configure --with-mysql-includes=/opt/local/include/mysql5/mysql --with-mysql-libs=/opt/local/lib/mysql5/mysql
make
sudo make install

Now we also need to install our chosen plugin into our Rails app. From your Rails root, using piston

piston import svn://rubyforge.org/var/svn/fauna/ultrasphinx/trunk vendor/plugins/ultrasphinx
cd vendor/plugins
ruby ultrasphinx/install.rb

Usage is also simple

All you have to do is open up your model and add an is_indexed line like this

is_indexed :fields => ['name', 'description']

A great thing about Ultrasphinx is that it is easy to get at multiple fields associated with a certain model. For instance this example has the relationships Bin has_many Services through BinsServices, Services has one Provider. We want the Bin name and description to be indexed, but we also want matches on Service name Service description and provider name to return the associated Bin. That ends up looking like this:

is_indexed :fields => ['name', 'description'],
       :include => [{:class_name => "Service", :field => 'description', :as => 'service_description', 
                     :association_sql => "JOIN (bins_services, services) ON (bins.id=bins_services.bin_id AND bins_services.service_id=services.id)"},
                    {:class_name => "Service", :field => 'name', :as => 'service_name', :association_sql => ""},
                    {:class_name => "Provider", :field => 'name', :as => 'provider_name', 
                     :association_sql => "JOIN (providers) ON (services.provider_id=providers.id)"}]

Notice that the second field for Service has to have its own line, and that the association_sql must be declaired, but be empty string (otherwise we JOIN twice and get an error). Also notice that the Provider JOIN starts off where the Service JOIN left off, it doesn't start at Bin, but goes from Service. It seems like this could be cleaned up a bit with some changes to the parser, but all in all it proved a workable solution for all the cases we had.

Next up we need to see what the search code looks like. We can just call Ultrasphinx::Search like so:

@search = Ultrasphinx::Search.new(:query => "My query string")
@search.run
@search.results

Now lets get the index run and the search daemon going. Note that these three commands can all be run at once with rake ultrasphinx:boot or its alias us:boot

rake ultrasphinx:configure
rake ultrasphinx:index
rake ultrasphinx:daemon:start

That about raps up our look at Ultrasphinx. One quick note, we ended up not using any of this code :-( The app in question is slated to run on Media Temple, and their grid structure precludes running background processes like this. We ended up moving everything to ActsAsSolr, a change that is worth looking at, but is the subject of another entry.

Test from mobile

Posted by Rob Kaufman
on Oct 05, 07

Test of s60 blog client blogplanet

it works!

Nested Resource Check List

Posted by Rob Kaufman
on Oct 04, 07

I want to get back to this and make a generator for it, but for now here is my checklist based mostly off of this post.

  1. Create resource scaffolding ruby script/generate scaffold_resource Nestee title:string body:text
  2. ruby script/generate scaffold_resource Nester body:text
  3. Setup relationships in models
  4. Modify Routes
  5. Modify controller of nested resource
    1. add before_filter to assign nestee
      1. include @nester = @nestee.nester.find(params[:id]) if params[:id]
    2. ammend the *_url calls with @parent
    3. remove all instances of Nester.find(params[:id])
    4. scope the nested class finder_all with the nestee
  6. Modify the views of nested resource
    1. edit.rhtml - formfor, linkto 'show'
    2. show.rhtml - link_to 'edit' (at bottom)
    3. index.rhtml - linkto 'edit' and linkto 'destroy'
  7. Provide links to nested resource