…hosted database service. On completion of that migration, we started a second migration, this time from SimpleDB to Cassandra. The first transition was key to our move from our own data center to AWS' cloud. The second was key to our expansion from one AWS Region to multiple geographically-distributed Regions -- today Netflix serves traffic out of two AWS Regions, one in Virginia, the other in Ireland ( F1 ). Both of these transitions have been successful, but have …
- Rails Envy » Home: Rails Envy Podcast - Episode #094: 10/02/2009
- paperplanes: NoSQL And You - Thoughts On Finding Right Partner In Chrime
- Engine Yard Developer Blog: MongoDB Best Practices
- James on Software: Most NoSQL DBs Are Not Scalable
- How Software is Built: Interview with Edd Dumbill - OSCON Co-Chair
- Dead Programmer Society: Happy Ada Lovelace Day 2010
…for storage and the maximum size for a single item always includes attribute names, just like with SimpleDB.
To give you an idea how pricing works out, here's a simple calculation. 100 GB of data, 1000 reads per second, 200 writes per second, item size is 4 KB on average. That's $ 1253.15 every month, not including traffic. Add 10 GB of data the second month and you're at $ 1263.15. You get the idea. Pricing is much more affected by read and write operations vs. item size. Make …
- A Fresh Cup: Double Shot #800
- Vineet Gupta | Computer Programmer: NoSql Databases - Part 1 - Landscape
- Heroku: NoSQL, Heroku, and You
- Dalibor Nasevic: Using Amazon S3 in Rails Applications
- New Relic: The Fallacies of Distributed Computing Reborn: The Cloud Era
- till's blog: Shopping for a CDN
SimpleDB (in the form of an ORM)
This covers the most used items of the AWS suite though some further investigation needs to be done to see if Elastic MapReduce and Elastic Block Store has been baked into the EC2 stuff. There's no Route 53 or Mechanical Turk support as yet, though Mark Percival's RTurk library is still being updated and supports the latter.
Getting Started With aws-sdk
Installing the new official AWS SDK gem is a piece of cake:
gem …
- Engine Yard Developer Blog: Offsite Backups with fog
- Zerosum Dirt(nap): Better AWS Access Control with IAM (and Fog)
- High Scalability: Stuff The Internet Says On Scalability For July 1, 2011
- MetaSkills.net: Authenticated S3 GETs For Private Objects Using Paperclip
- UK Specialists in Ruby on Rails, Exalead, AWS, Consultancy - Blog: Ooh la la: Paperclip et les European S3 buckets
- The Curiosity Project: Backing Up Your Data With Fog
So far, "database in the cloud" options have been pretty limited. You have Amazon's SimpleDB, which is slow, eventually consistent, and not really a database except in the loosest sense of the word; there's Amazon RDS, which is a managed MySQL instance and scales up, but not out; and of course you can run database [...]
- Merbist: Causality of scalability
- Vineet Gupta | Computer Programmer: NoSql Databases - Part 1 - Landscape
- Ruby Inside: Fog: A Powerful "Cloud Services" Gem
- OnRails.org: Amazon RDS: Amazon Relational Database Service or MySQL in the Cloud for Ruby On Rails.
- Trevor Turk: Links for 3-15-11
- MySQL Performance Blog: MySQL on Amazon RDS part 1: insert performance
…SQS (Simple Queuue Service) message queues, and Amazon SimpleDB domains.
blueimp/jQuery-File-Upload - GitHub
jQuery File Upload Plugin
Copycopter lets you edit live copy in your web app. Setup is simple and managing copy is simple, flexible and forgiving.
Cross browser testing with Selenium - Sauce Labs
Because you understand that test/behavior driven development and continuous integration make your web applications easier to develop …
- Lee Munroe: Goals for 2011
- Luke Redpath: Deploying your iOS ad-hoc builds with the BetaBuilder gem and TestFlight
- interblah.net: road-testing-the-eleven-inch-air
- Transcending Frontiers: Setting Up an iMac Pairing Station for Rails Development
- MetaSkills.net: Git Init XCode Projects
- AdminMyServer Blog: .gitignore for iPhone / XCode development
…they offer a custom, non-relational, data storage solution working around their own issues called SimpleDB. But why would they improve RDS and fix a really hard problem when they already offer an alternative solution? Easy. SimpleDB forces you to redesign your architecture to work with their custom solution and, guess what? You are now locked-in to that vendor!
So the answer is yes, you can offset scalability costs if you don't use AWS or any other providers with bad IO. Now you should …
- High Scalability: Scaling an AWS infrastructure - Tools and Patterns
- Ruby Inside: How to Cure Net::HTTP's Risky Default HTTPS Behavior
- Rails on Edge: The Ruby Hoedown
- Rails Envy » Home: Rails Envy Podcast - Episode #098
- blog.webandy.com | Andy Atkinson: Twin Cities Code Camp 2006
- Justin Leitgeb: NYC Amazon Web Services Meetup Talk
A giant step into the fully distributed future has been taken by the Google App Engine team with the release of their High Replication Datastore . The HRD is targeted at mission critical applications that require data replicated to at least three datacenters, full ACID semantics for entity groups , and lower consistency guarantees across entity groups.
This is a major accomplishment. Few organizations can implement a true multi-datacenter datastore. Other than SimpleDB…
fog is a Ruby gem by Wesley Beary to control a variety of cloud services through a unified API . It deals with both server cloud and storage based services and supports Amazon S3 and Rackspace Files; as well as servers and on Amazon EC2 , Rackspace Servers, Terremark vCloud and Slicehost. Support is also available for Amazon ELB and SimpleDB.
The beauty of fog is how the same code interacts with different cloud services. It uses models, an idea familiar …
- How Software is Built: Interview with Brian Gentile - CEO of Jaspersoft
- Opscode Blog: Chef Training Class in Seattle 4/26/2010
- RailsLab: Episode #21 - On The Edge - Part 3
- Dead Programmer Society: Configuring Hybrid Ruby on Rails Applications On EY-Cloud
- Software by Rob: Work Less, Get More Done, Convenient Open Source, What a Startup is Really Like, and more...
- till's blog: Thoughts on RightScale
such as SimpleDB, BigTable, Cassandra, CouchDB, and MongoDB .
I recommend a few excellent podcasts on Cassandra , CouchDB and MongoDB to get a sense of what this is all about.
Recently, MongoDB has received a lot of attention due to the following factors:
availability on many platforms
rich language support: C, C++, C#, Java, Javascript, Perl, PHP, Python, Ruby
binary json for efficient storage
equivalent of …
- The "Tech. Arch.": Implementing Ruby Camping REST Services With RESTstop
- The "Tech. Arch.": Transform Your Ruby Camping Web App Into An OAuth Provider
- New Relic: Ruby developers: Get visbility into Resque, Authlogic, Paperclip and Camping with New Relic's rpm_contrib gem
- Practicing Ruby: Name that method: Symmetry
- SD Ruby Podcast: Episode 002: Camping
- EdgeCase - Home: From Camping to Sinatra + DataMapper + Mustache (part 1)
…"document model" database formats, made popular by CouchDB, MongoDB and SimpleDB. For now, the W3C Working Group has officially declared SQL storage as being "at an impasse".
In the mean-time, there are some interesting alternatives. If SQL storage sounds suspiciously familiar, it might be because it's largely based on Google's browser plugin Gears . Gears has some issues - like the need to handle cases when it's not installed, or …
- Dead Programmer Society: LARubyConf 2009 - Jeremy Evans - "Sequel"
- oldmoe: Faster IO for Ruby with Postgres
- 4 Lines of Code: Background Jobs in Ruby on Rails
- Open Sourcery: Driving Business Processes in Ruby
- Litany Against Fear: Using Berkeley DB and Ruby for Large Data Sets Notes
- Luigi Montanez: Salesforce Tools of the Trade