Amazon Cloudsearch With Ruby And Asari Moving Towards 1.0

When you need to offer advanced search in your Ruby based application you have a number of options. People often will look at using a Lucene based solution such as Solr or Elasticsearch. The only problem is that you will need to set up your own instances or spend a lot of money for someone else to provide a search index that may be in a different location from the rest of your servers.

If you use a lot of Amazon services the Amazon Cloudsearch offering might be a great solution for you. Up until recently Cloudsearch was based on a proprietary tech stack. Now Cloudearch also uses Lucene/Solr as it’s engine. Long term hopefully they will expose their services using the standard Solr interfaces and API’s which would make integrating as easy as dropping in the Sunspot Gem and using them like any other Solr server. In the meantime we are stuck using the Amazon Cloudsearch API’s.

Active Asari

As is the case with many Amazon Services, using the Cloudsearch API directly is not the most intuitive experience, especially as a Rubiest. Two years ago Tommy Morgan created the Asari gem to make life easier for us. While Asari makes things a lot easier there were some features that we needed for a project that went beyond the gem.

In Cloudsearch you create a domain that contains fields that you search against. We wanted to be able to easily create a dev, test and staging domain in an automated fashion without needing to create custom scripts for each change. In the end we decided to use yaml files that specify the names, types and options for each field in the domain index.

Because this was making Asari opinionated we decided to make it an add-on gem called Active Asari. While we had been in touch with Tommy while Active Asari was under development, it wasn’t until it was released and we talked some more that we both came to the conclusion that the two projects should merge.

Heading Towards 1.0

Two months ago I joined the Asari project as a full committer. After adding in Travis-CI support and going through the backlog of pull requests and issues I got to work on merging ActiveAsari with Asari. Now if you want to use the goodness of ActiveAsari to be able to specify fields to be indexed via a configuration file, or be able to update your Amazon index’s programmatically you can do it without needing to include another gem.

2013-01-01 API Support

The previous version of Asari only supported version 2011-02-01 of the Amazon Cloudsearch API. 1.0 introduces support for the 2013-01-01 version of the API which adds support for more data types, and a bunch of other new features. You can go here for a full list. To maintain backwards compatibility Asari defaults to using the 2011-02-01 version. To use the 2013-01-01 simply set and environment variable called CLOUDSEARCH_API_VERSION to the new version.

The Roadmap

At the time of this writing 1.0 has not been released, but can be used by referencing the repo and commit_id here. The only feature gap between the 2011-02-01 API support and 2013-01-01 is geo spacial searching. The old version had to use the INT data type and put logic around it to provide support for these kinds of searches. In the 2013-01-01 version Amazon added a latlon data type and built in proximity search functions. Asari currently supports the creation of a latlon field, along with creating and deleting data in the index. It currently does not support searching against these fields.

Faceting has also been requested. If we have time we would like to add that in, but a firm decision has not been made as to if this will make the cutoff or not. Pull requests are welcome. Either way, if you want to use Amazon Cloud Search and Ruby, Asari will make your life easier. It is actively being developed, well tested and being run in production today. A very special thanks to Playon Sports and Treehouse Software for contributing back to this project. You can find it over here.