In a great article, Amazon S3 Performance Tips & Tricks , Doug Grismore, Director of Storage Operations for AWS, has outed the secret arcana normally reserved for Premium Developer Support customers on how to really use S3:
Size matters . Workloads with less than 50-100 total requests per second don't require any special effort. Customers that routinely perform thousands of requests per second need a plan.
Automated partitioning . Automated systems scale …
- OnRails.org: MWRC 2010 - Notes from Day 1.
- thechrisoshow: How to host a static website on S3
- Nick Sieger: Polyglot Doesn't Fix Your Messy Architecture
- blog.webandy.com | Andy Atkinson: Twin Cities Code Camp 2006
- Rowan Hick: JungleDisk - online backup done right?
- p.latyp.us: morph - a new deployment platform
99designs is a crowdsourced design contest marketplace based out of Melbourne Australia. The idea is that if you have a design you need created you create a contest and designers compete to give you the best design within your budget.
If you are a medium sized commerce site this is a clean example architecture of a site that reliably supports a lot of users and a complex workflow on the cloud. Lars Yencken wrote a nicely written overview of the architecture behind 99designs in …
- New Relic: Community News: Monitoring Alfresco with New Relic, slowing down the speed of change and more
- Kalzumeus Software: Does Your Product Logo Actually Matter?
- ZURB: Difficulties Building Product: Q&A with Founder of SitePoint
- Ruby Inside: Practical Tips for Hiring Ruby Web Developers
- RubyLearning Blog: Dmitiry Nagirnyak Winner RPCFN #7
Jeremiah Peschka with a great a set of Notes on Scalability , just in case you do reach your wildest expectations of success:
Build it to Break . Plan for the fact that everything you make is going to break. Design in layers that are independent and redundant.
Everything is a Feature . Your application is a set of features created by a series of conscious choices made by considering trade-offs.
Scale Out, Not Up . Purchasing more hardware is easier than coding and managing horizontal resources.
Srinath Perera has put together a strong list of architecture patterns based on three meta patterns: distribution, caching, and asynchronous processing. He contends these three are the primal patterns and the following patterns are but different combinations:
LB ( Load Balancers) + Shared nothing Units . Units that do not share anything with each other fronted with a load balancer that routes incoming messages to a unit based on some criteria.
LB + entity_5 …
In How to launch in a month and scale to a million users , Joseph Perla , Former VP of Technology and founding team of Turntable.fm, shares techniques he used to build and quickly scale his startups. The post is very well written and a must read. Here are the essentials:
Keep it simple. Build API's before making the website or mobile apps. Keep interfaces small and single-purpose.
Get it right. Build in automated tests from the start. Create function tests, …
Sean Hull with two helpful posts:
5 Things That are Toxic to Scalability :
Object Relational Mappers. Create complex queries that hard to optimize and tweak.
Synchronous, Serial, Coupled or Locking Processes. Locks are like stop signs, traffic circles keep the traffic flowing. Row level locking is better than table level locking. Use async replication. Use eventual consistency for clusters.
One Copy of Your Database. A single database server is a choke point. …
Jared Rosoff concisely, effectively, entertainingly, and convincingly gives an 8 minute MongoDB tutorial on scaling MongoDB at Scale Out Camp . The ideas aren't just limited to MongoDB, they work for most any database: Optimize your queries; Know your working set size; Tune your file system; Choose the right disks; Shard. Here's an explanation of all 5 strategies:
- Trevor Turk: Weekly Digest, 3-24-10
- Viget.com Blogs: Getting Started with MongoDB & MongoMapper
- Effectif Development: Replication and Replica Sets - Dwight Merriman
- Obie Fernandez: A Farewell Shot of Jack for Durran
- Heroku: MongoHQ Add-on Public Beta
- Phil Burrows, Ruby on Rails Web Developer: Rails Logging With MongoDB
Sean Hall with some really good MySQL advice :
Tune those queries . Biggest bang for your buck. Enable the slow query log and watch it. Once you've found a heavy resource intensive query, optimize it! Tune what receives real-world traffic.
Employ Master-Master Replication . Immediately have a read-only slave for your application to hit as well.
Use Your Memory . Set innodb_buffer_pool_size, key_buffer_size and other key options.
In How to take advantage of Redis just adding it to your stack Salvatore ' antirez' Sanfilippo shows how to solve some common problems in Redis by taking advantage of its unique data structure handling capabilities. Common Redis primitives like LPUSH, and LTRIM, and LREM are used to accomplish tasks programmers need to get done, but that can be hard or slow in more traditional stores. A very useful and practical article. How would you accomplish these tasks in your framework?
- paperplanes: When To Redis
- Slicehost VPS Hosting is now Rackspace Cloud Servers hosting - Blog: Redis for your slice?
- Axon Flux // A Ruby on Rails Blog: How Superfeedr built Analytics using MongoDB
- Merbist: How and why I joined the "suit people"
- Plataformatec Blog: Best Ruby Open Source Test Suites Awards
- Labnotes: 2011 is year of the Server-Side JavaScript
The most common complaint against NoSQL is that if you know how to write good SQL queries then SQL works fine. If SQL is slow you can always tune it and make it faster. A great example of this incremental improvement process was written up by StackExchange's Sam Saffron, in A day in the life of a slow page at Stack Overflow , where he shows through profiling and SQL tuning it was possible to reduce page load times from 630ms to 40ms for some pages and for other pages the improvement was 100x.
- paperplanes: Relational Data, Document Databases and Schema Design
- igvita.com: HandlerSocket: The NoSQL MySQL & Ruby
- Jake Scruggs: Rails Conf 2010 Day 3
- oldmoe: Faster IO for Ruby with Postgres
- Pragmatic Bookshelf: SQL Antipatterns: Avoiding the Pitfalls of Database Programming now in print
- Ruby Quicktips: How to check if objects or relations exist