…experienced in a project. When I first started tweet hopper , I decided to use Starling for queue handling. Why not, twitter used it... Everything ran great as long as I had one box with the job running daemons and the job queue. Everything fell apart slowly as I moved to two boxes, even load balancing between two starling servers didn't help.
That is when I decided that RabbitMQ might be my best bet, I had heard great things about Erlang's capabilities in the scaling …
- There's no Place Like ::1: Twitter AMQP WebSocket Example (no polling)
- Brightbox Blog: Queues and Callbacks
- igvita.com: HTTP PubSub: Webhooks & PubSubHubbub
- 4 Lines of Code: RabbitMQ and AMQP in Ruby
- Open Sourcery: Driving Business Processes in Ruby
- High Scalability: Hot Holiday Scalability Links for 2009
- Starling is a Message Queue Server based on MemCached
- written in Ruby
- stores jobs in memory (message queue)
- Ruby client: for instance Workling
- documentation: some good tutorials, for example the railscast about starling and workling or this blog post about starling
Kestrel
- written by Robey Pointer
- Starling clone written in Scala (a port of Starling from Ruby to Scala)
- Queues are stored in memory, but logged on disk
RabbitMQ…
- Paul Dix Explains Nothing: Resources for Synchronous Reads, Asynchronous Writes at RubyConf 2009
- Plataforma Tecnologia Blog: Monitoring Delayed Job with Bluepill and Capistrano
- ronin coder: Never do today what you can put off 'til tomorrow
- Brightbox Blog: Queues and Callbacks
- Scout ~ The Blog: RabbitMQ Monitoring Plugins
- Heroku: Background Jobs with DJ on Heroku
…job), but we need a proper, queue based runner in production. Up until recently we were using the Starling runner, which worked pretty well for a small set of machines.
However, after migrating our infrastructure to Amazon EC2 and rapidly scaling up the number of app servers, we figured it would be great to take advantage of Amazon SQS (Simple Queue Service), rather than maintaining our own queue servers. Fortunately, Workling's plugin architecture makes it very easy to …
- 4 Lines of Code: Background Jobs in Ruby on Rails
- Railscasts: Episode 128: Starling and Workling
- Litany Against Fear: Messaging with Ruby on Rails Notes
…there already, and a cheap client since they're already implemented and easy to use. Looking at Starling as an example of this. 24% queue, 76% other shit LOC -wise with REST . Going through implementing a queue in Sinatra, which is really quite easy. Building apps on REST is good since there's plenty of servers and clients out there that can easy consume and use the services without a lot of code.
- Litany Against Fear: Professional Ruby Conference Notes
- Developing with Style: Whats new in Cake 1.2: Routing and Config
- Rubycon - All things geared towards web development: Sexy Resources
- SD Ruby Podcast: Episode 009: REST with Rails
- Andrzej on Software: MVC: How to write controllers
- processi: lunch with Keith Swenson