With Kestrel, messages are pulled from a named queue, which can make it easier for a developer to think about problems in a way that can give them more flexability (including fairly easily changing the language/platform the work is being performed on).
Many of the benefits that you see with Kestrel often won't be evident until a high volume of work is being performed by the system.
Lets say I have a Ruby application. How hairy is Kestrel if you don't have Java experience? …
- Justin Leitgeb: In the Kitchen: What's behind Clean.ly
- Heroku: Background Jobs with DJ on Heroku
- RailsTips by John Nunemaker: Misleading Title About Queueing
- New Relic: "Extending New Relic's Custom Parameters," A Guest Post from Papertrail
One more deploy and we were queueing all track requests in Kestrel and popping them off in the track processors after which, the reports were updated in the primary database. This meant our track request now performed a single Kestrel set, instead of several queries and updates. As you would expect, response times dropped like a rock.
It is pretty obvious when Kestrel was rolled out as the graph went perfectly flat and dropped to ~4ms response times. BOOM .
You might say, yeah, your track …
- Justin Leitgeb: In the Kitchen: What's behind Clean.ly
- Heroku: Background Jobs with DJ on Heroku
- A Fresh Cup: Double Shot #797
- Trevor Turk: Links for 3-15-11
- Scout ~ The Blog: Monitoring HAProxy
…beanstalkd), Ruby ( Starling or Sparrow), Scala ( Kestrel) or Java ( ActiveMQ). A short overview can be found here
Sparrow
- written by Alex MacCaw
- Sparrow is a lightweight queue written in Ruby that "speaks memcache"
Starling
- written by Blaine Cook at Twitter
- Starling is a Message Queue Server based on MemCached
- written in Ruby
- stores jobs in memory (message queue)
- Ruby client: for instance Workling
- documentation: …
- ronin coder: Never do today what you can put off 'til tomorrow
- Brightbox Ruby Blog: Queues and Callbacks
- Scout ~ The Blog: RabbitMQ Monitoring Plugins
- Heroku: Background Jobs with DJ on Heroku
- igvita.com: Advanced Messaging & Routing with AMQP
- Opscode Blog: Chef 0.8.2 Release
…could accept data from different inputs without much trouble later. To meet these needs, I used a Kestrel queue to serialize HTTP retrievals from various providers.
Once I had the necessary data in place, Sphinx was an obvious choice to bring these data in front of the user on demand.
For those of you who have used the fulltext searching engine in Sphinx, it's easy to envision how restaurant searching based on name or "tag" (such as type of food) would work. A lesser …
- brant interactive: Project review: ifwinsight.com
- blog.webandy.com | Andy Atkinson: Rails Deployments
- Diego Carrion: ThinkingSphinx exits, enters ActsAsSolrReloaded
- MySQL Performance Blog: Getting around optimizer limitations with an IN() list
- Locomotivation: Mulling Over Our Ruby on Rails Full Text Search Options
- Developing with Style: Exact Matching with Sphinx