…and every one has 4 processes which every one has 25 workers. As we are all on EC2, we use spot instances for part of the cluster so we can add more workers as needed, so it could grow up from 400 to 800 or more.
The technology is nothing special but solid, Rails + Redis + Sharded MySQL.
Thanks Libin and everyone else who entered. I hope Sidekiq Pro makes your system even better!
…project you have to install new Rubies and bundle - plus you might need to install Redis, Solr or jam your public key on a server somewhere. Even worse, after bundling you notice this is a Rails 2.3 project not a Rails 3.2 and you haven't worked with Rails 2.3 for years and can't remember how to start a server.
Then you have to work out who is in charge of this damn project and bug them to find out why your PostgreSQL is complaining about some " PostGIS" thing …
…around it.
Related, the Big O cheatsheet .
§ Lisa Williams :
A scientific attitude toward one's pursuits is a great comfort: "It doesn't work" is a perfectly good experimental result.
§ Alastair Coote :
The new http://m.nytimes.com runs on Node, uses Coffeescript, LESS, and Redis for caching. How many hipster hacker points do we get?
…Redis. When a Worker starts processing a job, it adds a status entry to Redis. When the job is complete, the worker removes its entry from Redis. If you see one or more entries linger for much longer, there's two possibilities:
The worker thread has locked up or is blocked by some other issue
The Sidekiq process crashed
Log into the host noted in the list and look for the process with the PID. If the PID exists, the job is probably locked up somehow. Send the TTIN signal …
Experience with additional database platforms such as Postgres, Redis and Mongo in a live production environment
Able to work in a * nix/ Linux environment
Use source control exclusively, Git preferably
Use ftp to deploy applications
Experience programming iOS, Android or BlackBerry applications
Write tests with UnitTest, RSpec or some other test-driven framework
Additional Desirable Business Competencies
Comfortable working with business model canvases such as …
View the indexed ids using Redis smembers command: $ redis-cli smembers index:ba:movies 1) "1" 2) "2" 3) "3"
$ redis-cli smembers index:bat:movies 1) "2"
Note: The keys for the movie titles have been parameterized (lowercase and spaces replaced with dashes).
We'll use a Ruby class Autocomplete to wrap the Redis calls: # autocomplete.rb require 'redis'
class Autocomplete def initialize(partial_word) @partial_word …
jQuery 2.0 Released - Make sure you read about the supported browser changes before you switch.
Let Nunes Do It - Easily extend and consume Rails instrumentation.
Coffee-Rails - First piece of Rails 4.0 to put out a non-beta 4.0.0 version, I think.
Amanda - Blog entine that yets you write in Markdown, save on Dropbox, and store in Redis.
Pinterest has been riding an exponential growth curve, doubling every month and half. They've gone from 0 to 10s of billions of page views a month in two years, from 2 founders and one engineer to over 40 engineers, from one little MySQL server to 180 Web Engines, 240 API Engines, 88 MySQL DBs (cc2.8xlarge) + 1 slave each, 110 Redis Instances, and 200 Memcache Instances.
Stunning growth. So what's Pinterest's story? To tell their story we have our bards, Pinterest…
…relationships in data . Supports PostgreSQL, MySQL, SQLite, Redis, and MongoDB out-of-the-box.
§ Times Haiku, a collection of Haiku found in The New York Times . Love this one :
As an engineer,
I'm sort of a student of
how things fall apart.
§ Eric Bogs :
If Tetris has taught me anything, it is that errors pile up and accomplishments disappear.
§ Rich Rogers :
"A good programmer is someone who looks both …
Now, there are definitely other NoSQL options for persistent cache out there ( Redis, Couchbase, etc) , but they are outside the scope of this investigation and would require different client libraries and benchmark methodology.
My goal here was to compare a transparent switch (in terms of code) and experiment with the memcache plugin. Even the use of HandlerSocket would require coding changes (which is why it was also left out of the discussion).
The post MySQL 5.6 - InnoDB Memcached Plugin…