The Ruby Reflector

…actually consist of mongrel parser, EventMachine, and rack, but the latest EventMachine gem is not win32 binary release, that pose a lot of problem for the installation.

There are a few workarounds suggested on the net,

from Winston

install rack with the usual gem install

install EventMachine gem 0.8.1 as local gem

install thin gem install thin -ignore-dependencie

But this didn't quite workout for me, some suggested to install MinGW and MSYS , …

• Topics: Microsoft Windows, Duh.
• 4 Lines of Code: RabbitMQ and AMQP in Ruby
• Brightbox Blog: Queues and Callbacks
• A Fresh Cup: Double Shot #666
• Mike Perham: Scalable Ruby Processing with EventMachine
• oldmoe: A fast, simple, pure Ruby reactor library
• Engine Yard Ruby on Rails Blog: Using the Rubygems Bundler for Your App

A great example of a strong community would be EventMachine. They have a strong IRC channel (#eventmachine on freenode), a very active github project , and an active google group

How do you know all this?

Well, my friends, let me share a bit of growth pain I experienced in a project. When I first started tweet hopper , I decided to use Starling for queue handling. Why not, twitter used it... Everything ran great as long as I had one box with the job running …

• Topics: Advanced Message Queuing Protocol, RabbitMQ, Internet Relay Chat, Starling, TextMate
• There's no Place Like ::1: Twitter AMQP WebSocket Example (no polling)
• Brightbox Blog: Queues and Callbacks
• igvita.com: HTTP PubSub: Webhooks & PubSubHubbub
• 4 Lines of Code: RabbitMQ and AMQP in Ruby
• Article RSS Feed: August 2009 Meeting
• A Fresh Cup: Double Shot #692

…Ilya Grigorik's EM-Synchrony can then be used to make a collection of common EventMachine clients fiber-aware (for using Memcached, MongoDB, Beanstalk, and more). All of this works with any Rack app, not just in Rails.

This week, Aaron Gough has written an article called Improving application throughput 9x with asynchronous responses in Rails 3 that covers the concepts at a higher level and demonstrates how he ported an existing Rails app to use the …

• Topics: Ruby on Rails, Ruby, Rack, Ilya Grigorik, MongoDB
• upstream agile - software: Experimenting with node.js/mite
• RubyLearning Blog: RPCFN: Business Hours (#10)
• Jake Scruggs: Why I Left Obtiva
• Giles Bowkett: Code Review Service
• Brightbox Blog: Ruby Enterprise 1.8.7-2010.01 Packages for Ubuntu Hardy & Lucid
• Phusion Corporate Blog: Phusion Passenger 2.2.12 released

…Ruby process, even if you are somehow able to spawn thousands of Ruby threads. EventMachine works around this problem by bypassing Ruby's I/O code completely.

Naive native extensions and third party libraries

So just run a couple of multi-threaded Ruby processes, one process per core and multiple threads per process, and all is fine and we should be able to have a concurrency level of up to a couple hundred, right? Well not quite, there are a number of issues hindering this approach: …

• Topics: Ruby, Node.js, Ruby on Rails, Hypertext Transfer Protocol, Phusion Passenger
• Phusion Corporate Blog: Making Ruby threadable: properly handling context switching in native extensions
• The Queue Blog: Link dump for June 10th
• RubyLearning Blog: Paul Barry Winner RPCFN #8
• Intridea - Company Blog: In Memory of Zigzag Chen
• Ruby Inside: JRuby 1.5.0 Released: The Best Alternative Ruby Implementation Gets Even Better
• Luigi Montanez: Grassroots Fundraising for Open Source Software

…an evented I/O framework. You can think of it as Tornado, Twisted or EventMachine, simplified for sever-side JavaScript. Node doesn't require as much knowledge about event loops and non-blocking I/O, and the look and feel of such callbacks is very reminiscent of the type of AJAX code we've all seen before. Node can easily be used as a basic, ultra fast web server, to which one can delegate I/O callbacks for scalability and efficiency.

Recently Heroku announced beta support …

• Topics: JavaScript, Node, Ruby on Rails, Heroku, V8
• Simone Carletti's Blog: Unobtrusive JavaScript in Rails 3
• upstream agile - software: Capybara - Mix and match acceptance testing
• townx - tech: An introduction (of sorts) to JavaScript desktop application development with Gjs and Clutter
• Bamboo Blog: A JSON event-based convention for WebSockets
• the { buckblogs :here } - Home: Unobtrusive, yet explicit
• Dr Nic: Dead simple JavaScript Unit Testing in Rails

…walkthrough of creating a simple Web Sockets server in Ruby using EventMachine. There are plenty of other open source solutions for message pushing generally like Juggernaut and Cramp .

In many cases running your own standalone socket server gives you a lot of flexibility for more bespoke deployments. However, maintaining these socket servers, debugging the various issues that come up, and keeping up with the specs is not something everyone wants to be doing. Given …

• Topics: WebSockets, Ruby, Representational State Transfer, API, Pusher
• has_many :bugs, :through => :rails - Home: Websockets made easy with Cramp
• igvita.com: Ruby & WebSockets: TCP for the Browser
• Bamboo Blog: Brain Dump of Real Time Web(RTW) and WebSocket
• paperplanes: A Collection Of Redis Use Cases
• Luke Redpath: Pushing events to your iPhone using WebSockets and Pusher
• Rails Inside: Dragonfly: Image Handling For Champions

…Haskell. I think asynchronous IO/evented frameworks like Node.js and EventMachine provide a better model for building network servers. Technologies like XMPP and the WebSocket API that's part of HTML5 are also going to be key in the next generation of real-time, connected web and mobile applications.

Thank you Paul. In case you have any queries and/or questions, kindly post your questions here (as comments to this blog post) and Paul would be glad to answer.

• Topics: Ruby on Rails, Ruby, Paul, Satish, Baltimore
• Ruby Quicktips: Making sure that an options Hash contains certain keys only
• New Relic: Ruby developers: Get visbility into Resque, Authlogic, Paperclip and Camping with New Relic's rpm_contrib gem
• the evolving ultrasaurus: strong statements about testing
• Ruby on Rails, London - The Blog by Dynamic50: Installing a monitored Mongodb, Ruby 1.9, Nginx stack with Passenger on Ubuntu Linux
• Ruby Inside: Interesting Ruby Tidbits That Don't Need Separate Posts #30
• Giles Bowkett: Help Wanted: Rails Volunteer Work

…meticulously eliminating all blocking I/O from your code and dependencies, and using a library such as EventMachine, it's tedious and as Adam points out: "Libraries like eventmachine will never be truly intuitive to use, because event-driven I/O is enough of a fundamental shift that it requires deep language integration. JavaScript, it turns out, is a fundamentally event-driven language because of its origins in the browser"

Node.js is evented I/O for JavaScript…

• Topics: Ruby, JavaScript, Node.js, Node
• PaulBarry.com: Node.js Presentation
• GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS: Redis: Data Cheeseburgers
• jaikoo: Starting from scratch
• A Fresh Cup: Double Shot #672
• techno weenie: Node.js For My Tiny Ruby Brain: Keeping Promises
• High Scalability: Hot Holiday Scalability Links for 2009

…making every thread poll on a socket, we could pass each of those sockets to a single event loop ( EventMachine) library, and let it handle all the IO scheduling for us: gem install em-mysqlplus . Same API, in fact, it uses mysqlplus under the covers, but now every query has a callback for true non-blocking database access. Take a look at a few examples in the slides:

Non-blocking Rails with MySQL

Now we come around full circle. The downside of a true asynchronous library …

• Topics: Ruby on Rails, MySQL, VM, Drizzle, API
• Ruby Inside: Cramp: Asychronous Event-Driven Ruby Web App Framework
• has_many :bugs, :through => :rails - Home: Websockets made easy with Cramp
• Luigi Montanez: Presentation on Thin
• Engine Yard Ruby on Rails Blog: Ruby Scales, AND It's Fast - If You Do It Right!
• Softies on Rails - Home: What Did You Learn In 2008?
• codahale.com: When Formality Works

With that done, we can try some tests to see how we scale now. EventMachine works best when you have significant network latency. A simple test with database access over coffeeshop WiFi:

Without EventMachine:

Requests per second: 4.39 [#/sec] (mean)

With EventMachine:

Requests per second: 21.31 [#/sec] (mean)

That's it! There's no magic here: you can make your Rails app a "phat" app by following the same guidelines above. Fire up one thin instance …

• Topics: Ruby on Rails, Ruby, Web server, Rack, GIL
• A Fresh Cup: Double Shot #666
• igvita.com: Consuming XMPP PubSub in Ruby
• 4 Lines of Code: RabbitMQ and AMQP in Ruby
• Engine Yard Ruby on Rails Blog: Using the Rubygems Bundler for Your App
• oldmoe: Ruby Strikes Back
• Ruby Inside: EventMachine: Get Excited By Scalable Non-Blocking I/O