…Cassandra . We're using other open-source infrastructure to help us scale including Hadoop, RabbitMQ, Zookeeper, Thrift, HDFS and Lucene. We're rewriting Digg from the ground up and we need amazing developers to join our world-class team. If you think you are up for the challenge, or you know someone who might be, take a look at our jobs page for more information.
- Spyced: Cassandra in action
- AlternateIdea - Home: Programming Politics
- Simone Carletti's Blog: Introducing the Public Suffix List library for Ruby
- Software by Rob: "Startup Website Reviews" Screencast Series Has Moved to StartupLens.com
- Jason Seifer: 32 Rack Resources to Get You Started
- Litany Against Fear: Turning off the Firehose
That is when I decided that RabbitMQ might be my best bet, I had heard great things about Erlang's capabilities in the scaling department. After much deliberation and studying, I decided to make the leap and use AMQP and Eventmachine to interact with RabbitMQ. The choice of AMQP was a no brainer, we already used daemon-kit and it has native AMQP support. The switch took less than a weekend to get up and running!
In the end, we went from two starling queue servers …
- There's no Place Like ::1: Twitter AMQP WebSocket Example (no polling)
- Brightbox Blog: Queues and Callbacks
- igvita.com: HTTP PubSub: Webhooks & PubSubHubbub
- 4 Lines of Code: RabbitMQ and AMQP in Ruby
- Open Sourcery: Driving Business Processes in Ruby
- High Scalability: Hot Holiday Scalability Links for 2009
chef-solr/amqp_password - password for chef vhost in RabbitMQ.
chef-server-webui/admin_password - password for "admin" user in Chef Server WebUI, must be 6 characters.
Preseed settings can be specified with debconf-set-selections .
Reporting Issues
As with other Opscode Open Source projects, please report issues on our JIRA Ticketing system . Create new tickets in the CHEF project with the Packaging component. If you would like to contribute …
- English - AkitaOnRails.com: Cooking Solo with Chef
- Signal vs. Noise: Nuts & Bolts: Configuration management with Chef
- Labnotes: Necktie: dress to impress
- Brightbox Blog: The Brightbox Toolkit
- Hackido: Installing Git on a server (Ubuntu or Debian)
- MySQL Performance Blog: Percona Server version 10
excessive memory footprint ( ActiveMQ and RabbitMQ), both at startup (e.g. >120MB for ActiveMQ vs. <3 MB for ocamlmq) and as new queues are created or new topic subscriptions are added
bad performance with ActiveMQ's scalable (to thousands of queues) storage backends ( KahaDB, JDBC)
bad performance in RabbitMQ's topic message dispatch: RabbitMQ was doing a linear scan of the subscription table per dispatch
RabbitMQ did not guarantee that persistent messages …
- Litany Against Fear: Messaging with Ruby on Rails Notes
- 4 Lines of Code: Background Jobs in Ruby on Rails
- Jake Scruggs: Rails Conf 2010 Day 2
- Alexey Kovyrin's Blog: ActiveMQ Tips: Flow Control and Stalled Producers Problem
- ronin coder: Open Source Bridge presentation
Available from today, you can now choose either Ubuntu Hardy or Lucid as the base OS when building new Brightboxes.
Ubuntu Lucid is the latest Long Term Support ( LTS) version of Ubuntu with security fixes provided until April 2015. It brings a whole bunch of upgrades such as Ruby 1.8.7, Monit 5, Apache 2.2.14 and new packages like CouchDB, Sphinx, Chef, RabbitMQ, MongoDB, ejabberd and many more.
We've treated it to the usual Brightbox Ruby deployment tune-up, …
- MySQL Performance Blog: Pacemaker, please meet NDB Cluster or using Pacemaker/Heartbeat to start a NDB Cluster
- Slicehost - VPS Hosting - Blog: 32-bit Images Now Available
- Daddy Fixes Everything: Elk M1 Security System Control with Perl Script
- OmniNerd: Ubuntu 10.04 Accesses all iPhone Data by Default
- blog.webandy.com | Andy Atkinson: Rails Deployments
Get Your High Scalability Fix at Digg
Interested in working on cutting-edge high-scale infrastructure at Digg? We're making a big investment in scaling and have committed to the NoSQL (Not o nly SQL) path with Cassandra . We're using other open-source infrastructure to help us scale including Hadoop, RabbitMQ, Zookeeper, Thrift, HDFS and Lucene. We're rewriting Digg from the ground up and we need amazing developers to join our world-class team. …
- Spyced: Cassandra in action
- Axon Flux // A Ruby on Rails Blog: WTF is a SuperColumn? An Intro to the Cassandra Data Model (Arin Sarkissian explains)
- Jason Seifer: I Win
- Simone Carletti's Blog: Introducing the Public Suffix List library for Ruby
- Videos related to 'Ruby on Rails vs .NET - Commercial #5 of 9': Mac Spoof: Performance [Low Quality]
- Rails on Edge: Juggernaut Push Server
…processing system other than Resque and need to monitor it?
No Scout plugin for your background system? Be like Jesse and learn how to write your own Scout plugin . PS - We've added unit tests to plugins. Make your monitoring scripts rock-solid .
- Axon Flux // A Ruby on Rails Blog: Resque to the rescue. But a gotcha: Don't forget to refresh those ActiveRecord connections
- Intridea - Company Blog: Ten Keys to Building an Enduring Ruby User's Group
- zerosum dirt(nap) - Home: Resque Mailer
- Jason Seifer: Rake Tutorial
- A Fresh Cup: Double Shot #623
- ones zeros majors and minors: man what
When RabbitMQ is down during the creation of the initial Validation and Web UI keys, we now re-generate them once it is available.
Array's that contain hashes are now indexed properly.
I want to give a quick special thanks to the ton of people who have been helping the entire Chef community get up to speed on the changes in 0.8: by writing documentation, answering questions on the mailing list, and helping people in real-time in IRC . I'm proud to be around each and every …
- Opscode Blog: Amazon EC2 AMIs for Ubuntu 9.10 and Chef 0.8.10
- Riding Rails - home: Community Highlights
- Edd Dumbill's Weblog: Behind the Times: Speaking about Chef at RailsConf
- Signal vs. Noise: Nuts & Bolts: Configuration management with Chef
- A Fresh Cup: Double Shot #678
- BenCurtis.com: Sphinx, Chef and EY Cloud
Given all my work with Fibers and EventMachine over the last three months, it should come as no surprise that I've been working on infrastructure based on Fibers and EventMachine to get maximum scalability without the callback style of code which I dislike for many reasons. Watch my talk on scaling with EventMachine if you need more background on the problem.
Now that I have RabbitMQ, Cassandra, Solr and the Amazon AWS services evented, the only holdup was ActiveRecord…
…needs me to be aware: pick the right CPU architecture, find the AMI with the version of RabbitMQ I'm after etc.
A lot of other ‘cloud' providers (think Joyent Accelerators, Rackspace's CloudServers SliceHost etc.) are very similar. Although you can programmatically control instances, only pay for the time you use them for etc., you're still thinking at a relatively low systems level.
Amazon's Elastic MapReduce Service is an example of a higher-level …
- High Scalability: 10 Hot Scalability Links for January 13, 2010
- Rails Envy » Home: Rails Envy Podcast - Episode #093
- DigitalHobbit: Amazon EC2 Out Of Beta
- How Software is Built: Interview with Eliot Horowitz - CTO of 10gen / MongoDB
- Dead Programmer Society: PostgreSQL on Ubuntu on EC2: The Installation Guide
- elc technologies - Blog : Ruby on Rails, iPhone, Flash, Agile, Interactive Design: Amazon's MySQL in the Cloud (RDS)