Hey, it's HighScalability time:
( Earth sized solar flare , some more flair )
Google I/O to world: Just try to keep up with us. You can't. But go ahead and try. Nah na na na nah...
17 billion : Google Cloud Messaging messages per day with 60ms latency; 1B page views : 500px; 121 billion : edge graph using Titan; 4 billion hours : hours watched on Netflix per quarter; 4.5 trillion : BigTable transactions per month
- MySQL Performance Blog: Is Synchronous Replication right for your app?
As Spanner is a not so distant cousin of BigTable, the NoSQL component should be no surprise. Spanner is charged with spanning millions of machines inside any number of geographically distributed datacenters. What is surprising is how OldSQL has been embraced. In an earlier 2011 talk given by Alex at the HotStorage conference, the reason for embracing OldSQL was the desire to make it easier and faster for programmers to build applications. The main ideas will seem quite familiar: …
See the website .
See the slides .
He doesn't cover setting up a production cluster.
Using a schema is optional.
Cassandra is like a combination of Dynamo from Amazon and BigTable from Google.
It uses timestamps for conflict resolution. The clients determine the time. There are other approaches to conflict resolution as well.
Data in Cassandra looks like a multi-level dict.
By default, Cassandra eats 1/2 of your RAM. You might want to change that ;)
- Spyced: Cassandra in action
- Mike Perham: Cassandra Internals - Writing
- Ruby News: Red Dirt RubyConf 2010
- James on Software: Opinionated Modular Code
- High Scalability: NoSQL on the Microsoft Platform
- New Relic: Community News: Monitoring Alfresco with New Relic, slowing down the speed of change and more
…you get is a nice database engine for certain type of workloads. In fact, Google's BigTable, Hadoop's HBase, and Cassandra amongst others are all using a variant or a direct copy of this very architecture.
Simple on the surface, but as usual, implementation details matter a great deal. Thankfully, Jeff Dean and Sanjay Ghemawat , the original contributors to the SSTable and BigTable infrastructure at Google released LevelDB earlier last year , …
- Labnotes: Rounded Corners 330 - Bashed
- Trevor Turk: Links for 09-26-11
…can only assume DynamoDB either lets the last write win or has a scheme similar to BigTable, using timestamps for each attribute.
Writes don't allow you to specify something like a quorum, telling DynamoDB how consistent you'd like the write to be, it seems to be up to the system to decide when and how quickly replication to other datacenters is done. Alex Popescu's summary on DynamoDB and Werner Vogels' introduction suggest that writes are replicated …
- A Fresh Cup: Double Shot #800
- Vineet Gupta | Computer Programmer: NoSql Databases - Part 1 - Landscape
- Heroku: NoSQL, Heroku, and You
- Engine Yard Developer Blog: Key-Value Stores in Ruby: The Wrap Up
- carpeaqua by Justin Williams: Recipe for Baked WordPress
- Ruby Inside: Amazon Releases aws-sdk, An Official AWS SDK for Ruby Developers
…around this, even with a range-based key location. HBase (and Google's BigTable, for that matter) stores ranges of data in separate tablets. As tablets grow beyond their maximum size, they're split up and the remaining parts re-distributed. The advantage of this is that the original range is kept, even as you scale up.
Consistent Hashing Enables Partitioning
When you have a consistent hash, everything looks like a partition. The idea is simple. Consistent hashing forms a keyspace, …
- Trevor Turk: Links for 12-9-2011
- paperplanes: List All of the Riak Keys
- High Scalability: Is NoSQL a Premature Optimization that's Worse than Death? Or the Lady Gaga of the Database World?
- Axon Flux // A Ruby on Rails Blog: What if Foursquare wasn't using MongoDB and was using something else?
- Spyced: Cassandra: Fact vs fiction
- techno weenie: Nori: Node.js Riak wrapper
such as SimpleDB, BigTable, Cassandra, CouchDB, and MongoDB .
I recommend a few excellent podcasts on Cassandra , CouchDB and MongoDB to get a sense of what this is all about.
Recently, MongoDB has received a lot of attention due to the following factors:
availability on many platforms
rich language support: C, C++, C#, Java, Javascript, Perl, PHP, Python, Ruby
binary json for efficient storage
equivalent of …
- The "Tech. Arch.": Implementing Ruby Camping REST Services With RESTstop
- The "Tech. Arch.": Transform Your Ruby Camping Web App Into An OAuth Provider
- New Relic: Ruby developers: Get visbility into Resque, Authlogic, Paperclip and Camping with New Relic's rpm_contrib gem
- Practicing Ruby: Name that method: Symmetry
- SD Ruby Podcast: Episode 002: Camping
- EdgeCase - Home: From Camping to Sinatra + DataMapper + Mustache (part 1)
…Google and Amazon's broad use of their non-relational BigTable and Dynamo systems. We evaluated all the usual open source NoSQL suspects. After considerable debate, we decided to go with Cassandra.
We don't deserve anything. Publishers can do whatever they want. If you don't like it, don't send them nasty emails or browse their sites with ad-blockers: just don't support them. Don't read their content, don't link to them, and …
- Kalzumeus Software: Dashboard Design For Metrics-savvy Software Companies
- A Fresh Cup: Double Shot #657
- Rails Inside: Remail: Send and Receive E-mail Through Google App Engine for Free
- Katz Got Your Tongue?: The Blind Men and the Elephant: A Story of Noobs
- blog.mhartl | Michael Hartl's tech blog: Some rvm gotchas
- RubyLearning Blog: Showcasing RubyLearning's Awesome Rubyists
…engineer Stu Hood, who explained Cassandra's appeal: "Over the Bigtable clones, Cassandra has huge high-availability advantages, and no single point of failure. When compared to the Dynamo adherents, Cassandra has the advantage of a more advanced datamodel, allowing for a single row to contain billions of column/value pairs: enough to fill a machine. You also get efficient range queries for the top level key, and even within your values."
- Spyced: And now for something completely different
- Spyced: Cassandra: Fact vs fiction
- Mike Perham: Cassandra Internals - Writing
- Viget.com Blogs: Database Taxonomy
- High Scalability: Hot Scalability Links for March 19, 2010
- Axon Flux // A Ruby on Rails Blog: WTF is a SuperColumn? An Intro to the Cassandra Data Model (Arin Sarkissian explains)
…you've got a bunch of other data models in the middle: graph databases, tabular databases like BigTable, and document databases like Mongo.
There are a few different ways that you can think about document databases. One of the nice things about document databases is that they're closely mapped to how most developers are writing code, whereas SQL databases were designed for accounting and banking 30 or 40 years ago, prior to the advent of web applications and the rise of object-oriented …
- Locomotivation: TweetSaver.com Launched on MongoDB, MongoMapper