16 April 2014

The Ruby Reflector

Topic

AideRSS

  Source Favicon
By Ilya Grigorik of igvita.com over 2 years ago.
Email

…both em-http-request and Goliath for our stack at PostRank. A simple example in action:

require 'goliath' class Echo < Goliath::API use Goliath::Rack::Params use Goliath::Rack::Validation::RequiredParam , { :key => 'delay' } def response ( env ) EM::Synchrony . sleep params [ 'delay' ] [ 200 , { } , params [ 'delay' ] ] end end

require 'em-http-request' EM. run do conn = EM::HttpRequest . new ( 'http://localhost:9000/' …

igvita.com Read
  Source Favicon
On igvita.com over 2 years ago.
Email

…both em-http-request and Goliath for our stack at PostRank. A simple example in action:

require 'goliath' class Echo < Goliath :: API use Goliath :: Rack :: Params use Goliath :: Rack :: Validation :: RequiredParam , { :key => 'delay' } def response ( env ) EM :: Synchrony . sleep params [ 'delay' ] [ 200 , {}, params [ 'delay' ]] end end

require 'em-http-request' EM . run do conn = EM :: HttpRequest . new ( 'http://localhost:9000/' …

igvita.com Read
  Source Favicon
On PeepCode Products almost 3 years ago.
Email

…traditional procedural code. And you'll write a simple web service with Goliath, PostRank's evented replacement for Rack.

EventMachine is a powerful tool, but many features are difficult to figure out on their own or just by reading the API documentation. You'll benefit from Dan Sinclair's years of expertise and his current work with the latest EventMachine frameworks. Get started today!

Subscribe for your business, as an individual, or buy this screencast …

peepcode.com Read
  Source Favicon
On PeepCode Products almost 3 years ago.
Email

by Dan Sinclair, Developer at PostRank (recently acquired by Google)

The recent popularity of Node.js has overshadowed the existence of a mature, powerful tool available to Ruby developers: EventMachine. For over four years, some of the best and brightest Ruby developers have been deploying high performance applications with the EventMachine toolkit.

While the average single-threaded Rails application can serve a few hundred requests per second, EventMachine applications …

peepcode.com Read
  Source Favicon
By Ilya Grigorik of igvita.com almost 3 years ago.
Email

Goliath is an async web server and framework we developed at PostRank. While it does follow the Rack spec , it is also substantially different: it requires Ruby 1.9 runtime for Fiber support, and it uses an entirely different HTTP parser from Thin. Combined, these differences meant that up until now, deploying Goliath on any existing cloud environment was a no-go. However, with the new cedar stack, this is no longer an issue:

goliath ( Goliath is an async Ruby …

igvita.com Read
  Source Favicon
On igvita.com almost 3 years ago.
Email

Goliath is an async web server and framework we developed at PostRank. While it does follow the Rack spec , it is also substantially different: it requires Ruby 1.9 runtime for Fiber support, and it uses an entirely different HTTP parser from Thin. Combined, these differences meant that up until now, deploying Goliath on any existing cloud environment was a no-go. However, with the new cedar stack, this is no longer an issue:

source :gemcutter gem 'goliath' …

igvita.com Read
  Source Favicon
By Ilya Grigorik of igvita.com 3 years ago.
Email

…reimplemented at least half a dozen solutions in various languages and with various characteristics at PostRank, we arrived at the following requirements: we want a system that is able to scale to hundreds of millions of keys, we want it to be as space efficient as possible, have minimal maintenance, provide low latency access, and impose no language barriers . The tradeoff: we will accept a certain (customizable) degree of error, and we will not persist the keys forever.

Mneme: Duplicate …

igvita.com Read
  Source Favicon
By Ilya Grigorik of igvita.com 3 years ago.
Email

…complicated, non-linear execution models which are hard to maintain - we have experienced this firsthand at PostRank and hence made it a primary concern for Goliath.

To solve this, Goliath runs on Ruby 1.9 and leverages Fibers (coroutines) to allow us to transparently pause and resume the execution of our asynchronous codebase, while preserving the look and feel of a synchronous API!

Goliath: async GitHub proxy

To get started, simply "gem install goliath" …

igvita.com Read
  Source Favicon
By Ilya Grigorik of igvita.com over 3 years ago.
Email

…another notable user, said to be handling up to 800 queries/sec , and at PostRank we are currently managing a 1TB+ index (growing at ~40GB a week) with over 1.2 billion documents.

Lucene + HTTP: Solr Server

If Lucene is a low-level IR toolkit, then Solr is the fully-featured HTTP search server which wraps the Lucene library and adds a number of additional features: additional query parsers, HTTP caching, search faceting, highlighting, and many others …

igvita.com Read
  Source Favicon
By Ilya Grigorik of igvita.com almost 4 years ago.
Email

Beanstalk at PostRank: Chronos

At PostRank we have dozens of Beanstalk processes sprinkled throughout which are being used for job management within the same machine and coordination between entire clusters. The larger deployments, which are the front-line coordinators to our crawlers are serving 50+ million jobs on a daily basis (average job is several kb), without breaking a sweat. On average, each job is just several kilobytes, but the numbers add up, meaning that a pure memory system …

igvita.com Read