…Flickr, uploads and updates (around 100/s depending on the time of day), all of them inserting tasks. Implemented with Cache, Tasks, & Queues: PubSubHubbub; Async task system Gearman; use async EVERYWHERE; use Redis Lists for queues; cron to consume events off the queue; To read even more Stuff the Internet has on Scalability, please click below...
- Engine Yard Blog: Concurrency in JRuby
- MySQL Performance Blog: InnoDB Flushing: Theory and solutions
- Mental Paging Space: RealTime and Rails
- A Rubyist Railstastic Adventure: Multithreaded Rails is generally better than Async Rails
- Merbist: Ruby concurrency explained
- igvita.com: HTML5 Visibility API & Page Pre-Rendering
Mobile Activity Feeds via PubSubHubbub
Maintaining direct tethered links to each individual mobile device is obviously an expensive and a technically challenging proposition, especially if we are after real-time updates. But the good news is, we have already solved this problem in a different context. WebHooks allow us to establish callback (push) semantics between web-services, and PubSubHubbub solves the problem of efficiently delivering real-time notifications from …
- Luigi Montanez: Real Time Disclosure, Technically Speaking
WebHooks, PubSubHubbub, WebSockets, ...
Of course, WebSockets are not the panacea to every problem. WebHooks and PubSubHubbub are great protocols for intermittent push updates where a long-lived TCP connection may prove to be inefficient. Likewise, if you require non-trivial routing then AMQP is a powerful tool , and there is little reason to reinvent the powerful presence model built into XMPP . Right tool for the right job, but WebSockets are without …
…data that's being transferred is actual data and there is no useless polling going on. Although PubSubHubbub is one solution to the polling problem , it's not the only one. Others are using the IM standard, XMPP , to also distribute information to Subscribers.
If you're interested in PubSubHubbub, I strongly suggest that you read the documentation at Google's Code page .
- igvita.com: Smart Clients: ReverseHTTP & WebSockets