high_scalability high_scalability-2010 high_scalability-2010-771 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Lots of cool stuff happening this week... Voldemort gets rebalancing. It's one thing to shard data to scale, it's a completely different level of functionality to manage those shards intelligently. Voldemort has stepped up by adding advanced rebalancing functionality: Dynamic addition of new nodes to the cluster; Deletion of nodes from cluster; Load balancing of data inside a cluster. Microsoft Finally Opens Azure for Business. Out of the blue Microsoft opens up their platform as a service service. Good to have more competition and we'll keep an eye out for experience reports. New details on LinkedIn architecture by Greg Linden. LinkedIn appears to only use caching minimally, preferring to spend their efforts and machine resources on making sure they can recompute computations quickly than on hiding poor performance behind caching layers . The end of SQL and relational databases? by David Intersimone . For new projects, I believe, we have genuine non-relational a
sentIndex sentText sentNum sentScore
1 It's one thing to shard data to scale, it's a completely different level of functionality to manage those shards intelligently. [sent-5, score-0.105]
2 Voldemort has stepped up by adding advanced rebalancing functionality: Dynamic addition of new nodes to the cluster; Deletion of nodes from cluster; Load balancing of data inside a cluster. [sent-6, score-0.317]
3 Out of the blue Microsoft opens up their platform as a service service. [sent-8, score-0.278]
4 Good to have more competition and we'll keep an eye out for experience reports. [sent-9, score-0.081]
5 LinkedIn appears to only use caching minimally, preferring to spend their efforts and machine resources on making sure they can recompute computations quickly than on hiding poor performance behind caching layers . [sent-11, score-0.515]
6 For new projects, I believe, we have genuine non-relational alternatives on the table (pun intended) . [sent-14, score-0.189]
7 When you make millions of widgets saving pennies per widget quickly adds up to real money. [sent-16, score-0.436]
8 Facebook released HipHop, a PHP compiler, aimed at shaving off cycle of CPU and bytes of memory in production of their social widgets. [sent-17, score-0.388]
9 What Second Life can teach your datacenter about scaling Web apps By Ian Wilkes . [sent-18, score-0.091]
10 This article will lay out some pragmatic advice for getting past this point in the real world of limited foresight and budgets . [sent-20, score-0.229]
11 A nice explanation of why DNS sucks for failover and Google's proposal to fix it (and get location information). [sent-23, score-0.191]
12 Learn all about the esoterica of write-ahead-logging and how HBase uses it to survive server crashes. [sent-25, score-0.083]
13 Plurk released instant notifications which allows hundred thousands of active users , who have post posted over 1 billion unique messages, to converse with only sub-second delays. [sent-27, score-0.396]
14 Not only is this impressive, but as impressive is that they use Node. [sent-28, score-0.152]
15 If you are a worried about how to scale the future, this sounds pretty good to me. [sent-33, score-0.098]
16 But yesterday's organizations — from carmakers, to investment banks, to the healthcare system, to the energy industry, to the Senate itself — are broken. [sent-36, score-0.328]
17 Today's biggest human challenge isn't leading broken organizations slightly better. [sent-37, score-0.213]
18 It isn't about leadership: it's about "buildership", or what I often refer to as Constructivism. [sent-39, score-0.095]
wordName wordTfidf (topN-words)
[('organizations', 0.213), ('yesterday', 0.193), ('opens', 0.189), ('impressive', 0.152), ('shaving', 0.142), ('recompute', 0.133), ('converse', 0.133), ('pennies', 0.133), ('instant', 0.132), ('released', 0.131), ('minimally', 0.127), ('foresight', 0.127), ('architectureby', 0.127), ('gslb', 0.127), ('pun', 0.123), ('preferring', 0.123), ('stepped', 0.115), ('aimed', 0.115), ('healthcare', 0.115), ('deletion', 0.113), ('widget', 0.11), ('hiphop', 0.11), ('lars', 0.11), ('widgets', 0.108), ('proposal', 0.108), ('banks', 0.108), ('rebalancing', 0.108), ('functionality', 0.105), ('genuine', 0.104), ('builders', 0.102), ('lay', 0.102), ('worried', 0.098), ('conversations', 0.098), ('leadership', 0.097), ('php', 0.096), ('refer', 0.095), ('balancing', 0.094), ('leaders', 0.094), ('voldemort', 0.094), ('hiding', 0.093), ('teach', 0.091), ('blue', 0.089), ('quickly', 0.085), ('alternatives', 0.085), ('compiler', 0.084), ('sucks', 0.083), ('survive', 0.083), ('eye', 0.081), ('computations', 0.081), ('intended', 0.08)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999976 771 high scalability-2010-02-04-Hot Scalability Links for February 4, 2010
Introduction: Lots of cool stuff happening this week... Voldemort gets rebalancing. It's one thing to shard data to scale, it's a completely different level of functionality to manage those shards intelligently. Voldemort has stepped up by adding advanced rebalancing functionality: Dynamic addition of new nodes to the cluster; Deletion of nodes from cluster; Load balancing of data inside a cluster. Microsoft Finally Opens Azure for Business. Out of the blue Microsoft opens up their platform as a service service. Good to have more competition and we'll keep an eye out for experience reports. New details on LinkedIn architecture by Greg Linden. LinkedIn appears to only use caching minimally, preferring to spend their efforts and machine resources on making sure they can recompute computations quickly than on hiding poor performance behind caching layers . The end of SQL and relational databases? by David Intersimone . For new projects, I believe, we have genuine non-relational a
2 0.12233895 561 high scalability-2009-04-08-N+1+caching is ok?
Introduction: Hibernate and iBATIS and other similar tools have documentation with recommendations for avoiding the "N+1 select" problem. The problem being that if you wanted to retrieve a set of widgets from a table, one query would be used to to retrieve all the ids of the matching widgets (select widget_id from widget where ...) and then for each id, another select is used to retrieve the details of that widget (select * from widget where widget_id = ?). If you have 100 widgets, it requires 101 queries to get the details of them all. I can see why this is bad, but what if you're doing entity caching? i.e. If you run the first query to get your list of ids, and then for each widget you retrive it from the cache. Surely in that case, N+1(+caching) is good? Assuming of course that there is a high probability of all of the matching entities being in the cache. I may be asking a daft question here - one whose answer is obviously implied by the large scalable mechanisms for storing data th
3 0.10833878 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
Introduction: For everything given something seems to be taken. Caching is a great scalability solution, but caching also comes with problems . Sharding is a great scalability solution, but as Foursquare recently revealed in a post-mortem about their 17 hours of downtime, sharding also has problems. MongoDB, the database Foursquare uses, also contributed their post-mortem of what went wrong too. Now that everyone has shared and resharded, what can we learn to help us skip these mistakes and quickly move on to a different set of mistakes? First, like for Facebook , huge props to Foursquare and MongoDB for being upfront and honest about their problems. This helps everyone get better and is a sign we work in a pretty cool industry. Second, overall, the fault didn't flow from evil hearts or gross negligence. As usual the cause was more mundane: a key system, that could be a little more robust, combined with a very popular application built by a small group of people, under immense pressure
Introduction: If you code in PHP have you ever wondered about moving to Facebook's HipHop JIT Virtual Machine for PHP ? With HipHop Facebook achieved over a 9x increase in web request throughput and over a 5x reduction in memory consumption compared to Zend PHP 5.2 engine + APC. But will HipHop really work for you? Is it really drop-in compatible? Is it really as fast as they say? To answer questions like this nothing beats a good experience report and here's a great one: Adventures in Configuring and Running Facebook's HipHopVM (hhvm) JIT Compiler for PHP by Yermo Lamers. Yermo selected PHP to implement a number of content web sites. He took an interesting approach, he created a forms, views, validation, and business logic description language to remove the drudgery of creating the same code over and over again for each page. Having done this in Perl I think it's a great a approach. The problem is it can be slow. PHP's slow string handling makes dynamically evaluating a description templat
5 0.099524245 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
7 0.09929017 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
8 0.095431328 152 high scalability-2007-11-13-Flickr Architecture
9 0.094967216 943 high scalability-2010-11-16-Facebook's New Real-time Messaging System: HBase to Store 135+ Billion Messages a Month
10 0.094889708 1404 high scalability-2013-02-11-At Scale Even Little Wins Pay Off Big - Google and Facebook Examples
11 0.093922995 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
12 0.091664836 1180 high scalability-2012-01-24-The State of NoSQL in 2012
13 0.089496821 570 high scalability-2009-04-15-Implementing large scale web analytics
14 0.087052107 667 high scalability-2009-07-31-NSFW: Hilarious Fault-Tolerance Cartoon
15 0.086969957 427 high scalability-2008-10-22-Server load balancing architectures, Part 2: Application-level load balancing
16 0.085238397 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
17 0.08425726 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
18 0.083622999 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
19 0.083468273 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
20 0.081592001 672 high scalability-2009-08-06-An Unorthodox Approach to Database Design : The Coming of the Shard
topicId topicWeight
[(0, 0.173), (1, 0.08), (2, 0.006), (3, -0.004), (4, 0.033), (5, 0.001), (6, -0.041), (7, -0.013), (8, -0.002), (9, 0.039), (10, -0.004), (11, -0.001), (12, 0.012), (13, 0.015), (14, -0.014), (15, 0.003), (16, 0.015), (17, -0.031), (18, 0.013), (19, 0.032), (20, 0.033), (21, 0.036), (22, -0.013), (23, -0.057), (24, 0.01), (25, 0.013), (26, -0.024), (27, -0.035), (28, -0.064), (29, 0.031), (30, -0.021), (31, -0.012), (32, 0.027), (33, -0.034), (34, 0.066), (35, 0.014), (36, 0.033), (37, -0.004), (38, -0.036), (39, 0.001), (40, -0.046), (41, 0.022), (42, 0.043), (43, -0.018), (44, 0.044), (45, 0.049), (46, 0.024), (47, 0.049), (48, 0.034), (49, -0.042)]
simIndex simValue blogId blogTitle
same-blog 1 0.96233273 771 high scalability-2010-02-04-Hot Scalability Links for February 4, 2010
Introduction: Lots of cool stuff happening this week... Voldemort gets rebalancing. It's one thing to shard data to scale, it's a completely different level of functionality to manage those shards intelligently. Voldemort has stepped up by adding advanced rebalancing functionality: Dynamic addition of new nodes to the cluster; Deletion of nodes from cluster; Load balancing of data inside a cluster. Microsoft Finally Opens Azure for Business. Out of the blue Microsoft opens up their platform as a service service. Good to have more competition and we'll keep an eye out for experience reports. New details on LinkedIn architecture by Greg Linden. LinkedIn appears to only use caching minimally, preferring to spend their efforts and machine resources on making sure they can recompute computations quickly than on hiding poor performance behind caching layers . The end of SQL and relational databases? by David Intersimone . For new projects, I believe, we have genuine non-relational a
2 0.7227723 1223 high scalability-2012-04-06-Stuff The Internet Says On Scalability For April 6, 2012
Introduction: It's HighScalability Time: Exascale Supercomputer : how IBM plans to understand data from a universe of light; 905 Billion Objects and 650,000 Requests/Second : S3; 64-cores : PostgreSQL shows linear read scalability; Quotable quotes: pkaler : Programming is hard. Scaling is harder. @crucially : As far as I can tell, openstack is what happens when ops people write code. @DEVOPS_BORAT : Goal of sysadmin is replace itself with small shell script. Goal of devops is replace itself with small REST API. @fowlduck : ec2, where dynamic scalability means them running out of instances :( hcarvalhoalves : You know what is amazing? Is that as soon you hit bigger or more general problems, you always face the compromise of "trading X resource for accuracy". Which leads me to believe that software, so far, has only been deterministic by pure accident. Kyle Lemmons : Clearly Go is a superior weapon if the goal is to shoot everyone in the > foot at the same ti
3 0.67845702 1487 high scalability-2013-07-05-Stuff The Internet Says On Scalability For July 5, 2013
Introduction: Hey, it's HighScalability time: (Dolls nerds can nest with ) Quotable Quotes: @Carnage4Life : "Google uses Bayesian filtering the way Microsoft uses the if statement" - http://www.joelonsoftware.com/items/2005/10/17.html … <= finally at the point where I get this @etherealmind : You can dramatically improve blog performance by blocking Amazon IP address ranges. Tells you how much information mining is occurring. Randy Bias : Choice is possible only when there’s architectural consistency between public and private cloud infrastructure. Those who focus only on API compatibility are either confused or intentionally misleading people. There is NO API COMPATIBILITY without architectural compatibility. Nassim Nicholas Taleb : Everything that is fragile and still in existence (that is, unbroken), will be harmed more by a certain stressor of intensity X than by k times a stressor of intensity X/k, up to the point of breaking.
4 0.67777729 1252 high scalability-2012-05-25-Stuff The Internet Says On Scalability For May 25, 2012
Introduction: It's HighScalability Time: 15 million 64 GB iPods every day : IBM on the Big Bang; Billions of msgs/day, 75Bn ops/day, 1.5M ops/sec peak. 250TB new data/mo and growing : Facebook HBase stats; 72 hours of video uploaded every minute : YouTube Quotable Quotes: @ammeep : Great discussion on data scalability and caching approaches at #yellowcamp "a cache is like your best mate you can never trust" @adrianco : Why do OpenStack talks I go to always focus on the story of how they did it rather than how it works and what it does for me? #Gluecon. Cliff Moon : Incuriosity killed the infrastructure. @seldo2 : "NoSQL is a land grab against the DBAs the prevent you from doing shit" - @monkchips #gluecon @mjasay : Netflix's @atseitlin: Build for fail, not for scale. Focus on service-level metrics, not individual servers, when monitoring system health @peakscale : I liked the part about "graph DBs are a niche data model... that only covers 95% of situation
5 0.67370403 1122 high scalability-2011-09-23-Stuff The Internet Says On Scalability For September 23, 2011
Introduction: I'd walk a mile for HighScalability : 1/12th the World Population on Facebook in One Day ; 1.8 ZettaBytes of data in 2011; 1 Billion Foursquare Checkins ; 2 million on Spotify ; 1 Million on GitHub ; $1,279-per-hour, 30,000-core cluster built on EC2 ; Patent trolls cost .5 trillion dollars ; 235 terabytes of data collected by the U.S. Library of Congress in April . Potent quotables: @jstogdill : Corporations over protect low value info assets (which screws up collaboration) and under protects high value assets. #strataconf @sbtourist : I think BigMemory-like approaches based on large put-and-forget memory cans, are rarely a solution to performance/scalability problems. 1 Million TCP Connections . Remember when 10K was a real limit and you had to build out boxes just to handle the load? Amazing. We don't know how much processing can be attached to these connections, how much memory the apps use, or what the response latency is to
6 0.66984355 1645 high scalability-2014-05-09-Stuff The Internet Says On Scalability For May 9th, 2014
7 0.66917521 1214 high scalability-2012-03-23-Stuff The Internet Says On Scalability For March 23, 2012
8 0.66885448 1451 high scalability-2013-05-03-Stuff The Internet Says On Scalability For May 3, 2013
9 0.66861939 288 high scalability-2008-03-25-Paper: On Designing and Deploying Internet-Scale Services
10 0.66834772 519 high scalability-2009-02-23-Database Sharding at Netlog, with MySQL and PHP
11 0.66761655 1089 high scalability-2011-07-29-Stuff The Internet Says On Scalability For July 29, 2011
12 0.66740805 1494 high scalability-2013-07-19-Stuff The Internet Says On Scalability For July 19, 2013
13 0.66697294 835 high scalability-2010-06-03-Hot Scalability Links for June 3, 2010
14 0.66679686 1596 high scalability-2014-02-14-Stuff The Internet Says On Scalability For February 14th, 2014
15 0.66478592 1476 high scalability-2013-06-14-Stuff The Internet Says On Scalability For June 14, 2013
16 0.66395468 1297 high scalability-2012-08-03-Stuff The Internet Says On Scalability For August 3, 2012
17 0.66364688 1158 high scalability-2011-12-16-Stuff The Internet Says On Scalability For December 16, 2011
18 0.66283137 1389 high scalability-2013-01-18-Stuff The Internet Says On Scalability For January 18, 2013
19 0.66158783 1424 high scalability-2013-03-15-Stuff The Internet Says On Scalability For March 15, 2013
20 0.66150302 1336 high scalability-2012-10-09-Batoo JPA - The new JPA Implementation that runs over 15 times faster...
topicId topicWeight
[(1, 0.145), (2, 0.184), (10, 0.042), (11, 0.194), (43, 0.016), (47, 0.041), (56, 0.03), (61, 0.035), (77, 0.056), (79, 0.097), (85, 0.039), (94, 0.043)]
simIndex simValue blogId blogTitle
1 0.90004236 668 high scalability-2009-08-01-15 Scalability and Performance Best Practices
Introduction: These are from Laura Thomson of OmniTi : Profile early, profile often. Pick a profiling tool and learn it in and out. Dev-ops cooperation is essential. The most critical difference in organizations that handles crises well. Test on production data. Code behavior (especially performance) is often data driven. Track and trend. Understanding your historical performance characteristics is essential for spotting emerging problems. Assumptions will burn you. Systems are complex and often break in unexpected ways. Decouple. Isolate performance failures. Cache. Caching is the core of most optimizations. Federate. Data federation is taking a single data set and spreading it across multiple database/application servers. Replicate. Replication is making synchronized copies of data available in more than one place. Avoid straining hard-to-scale resources. Some resources are inherently hard to scale: Uncacheable’ data, Data with a very high read+write rate
2 0.89314181 134 high scalability-2007-10-26-Paper: Wikipedia's Site Internals, Configuration, Code Examples and Management Issues
Introduction: Wikipedia and Wikimedia have some of the best, most complete real-world documentation on how to build highly scalable systems. This paper by Domas Mituzas covers a lot of details about how Wikipedia works, including: an overview of the different packages used (Linux, PowerDNS, LVS, Squid, lighttpd, Apache, PHP5, Lucene, Mono, Memcached), how they use their CDN, how caching works, how they profile their code, how they store their media, how they structure their database access, how they handle search, how they handle load balancing and administration. All with real code examples and examples of configuration files. This is a really useful resource. Related Articles Wikimedia Architecture Domas Mituzas' Blog
same-blog 3 0.89191258 771 high scalability-2010-02-04-Hot Scalability Links for February 4, 2010
Introduction: Lots of cool stuff happening this week... Voldemort gets rebalancing. It's one thing to shard data to scale, it's a completely different level of functionality to manage those shards intelligently. Voldemort has stepped up by adding advanced rebalancing functionality: Dynamic addition of new nodes to the cluster; Deletion of nodes from cluster; Load balancing of data inside a cluster. Microsoft Finally Opens Azure for Business. Out of the blue Microsoft opens up their platform as a service service. Good to have more competition and we'll keep an eye out for experience reports. New details on LinkedIn architecture by Greg Linden. LinkedIn appears to only use caching minimally, preferring to spend their efforts and machine resources on making sure they can recompute computations quickly than on hiding poor performance behind caching layers . The end of SQL and relational databases? by David Intersimone . For new projects, I believe, we have genuine non-relational a
4 0.88819522 25 high scalability-2007-07-25-Paper: Designing Disaster Tolerant High Availability Clusters
Introduction: A very detailed (339 pages) paper on how to use HP products to create a highly available cluster. It's somewhat dated and obviously concentrates on HP products, but it is still good information. Table of contents: 1. Disaster Tolerance and Recovery in a Serviceguard Cluster 2. Building an Extended Distance Cluster Using ServiceGuard 3. Designing a Metropolitan Cluster 4. Designing a Continental Cluster 5. Building Disaster-Tolerant Serviceguard Solutions Using Metrocluster with Continuous Access XP 6. Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with EMC SRDF 7. Cascading Failover in a Continental Cluster Evaluating the Need for Disaster Tolerance What is a Disaster Tolerant Architecture? Types of Disaster Tolerant Clusters Extended Distance Clusters Metropolitan Cluster Continental Cluster Continental Cluster With Cascading Failover Disaster Tolerant Architecture Guidelines Protecting Nodes through Geographic Dispersion Protecting Data th
Introduction: Snooze is an open-source, scalable, autonomic, and energy-efficient virtual machine (VM) management framework for private clouds. Similarly to other VM management frameworks such as Nimbus, OpenNebula, Eucalyptus, and OpenStack it allows to build compute infrastructures from virtualized resources. Particularly, once installed and configured users can submit and control the life-cycle of a large number of VMs. However, contrary to existing frameworks for scalability and fault tolerance, Snooze employs a self-organizing and healing (based on Apache ZooKeeper) hierarchical architecture. Moreover, it performs distributed VM management and is designed to be energy efficient. Therefore, it implements features to monitor and estimate VM resource (CPU, memory, network Rx, network Tx) demands, detect and resolve overload/underload situations, perform dynamic VM consolidation through live migration, and finally power management to save energy. Last but not least, it integrates a g
6 0.85611987 1055 high scalability-2011-06-08-Stuff to Watch from Google IO 2011
7 0.85253876 1236 high scalability-2012-04-30-Masstree - Much Faster than MongoDB, VoltDB, Redis, and Competitive with Memcached
8 0.85093182 105 high scalability-2007-10-01-Statistics Logging Scalability
9 0.84243083 457 high scalability-2008-12-01-Sun FireTM X4540 Server as Backup Server for Zmanda's Amanda Enterprise 2.6 Software
10 0.83062851 908 high scalability-2010-09-28-6 Strategies for Scaling BBC iPlayer
11 0.823614 136 high scalability-2007-10-28-Scaling Early Stage Startups
12 0.82281792 442 high scalability-2008-11-13-Plenty of Fish Says Scaling for Free Doesn't Pay
13 0.79772019 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
14 0.79544455 72 high scalability-2007-08-22-Wikimedia architecture
15 0.79029769 303 high scalability-2008-04-18-Scaling Mania at MySQL Conference 2008
16 0.78919655 279 high scalability-2008-03-17-Microsoft's New Database Cloud Ready to Rumble with Amazon
17 0.7890445 1537 high scalability-2013-10-25-Stuff The Internet Says On Scalability For October 25th, 2013
18 0.78683907 459 high scalability-2008-12-03-Java World Interview on Scalability and Other Java Scalability Secrets
19 0.78680193 1377 high scalability-2012-12-26-Ask HS: What will programming and architecture look like in 2020?
20 0.7866801 1567 high scalability-2013-12-20-Stuff The Internet Says On Scalability For December 20th, 2013