high_scalability high_scalability-2011 high_scalability-2011-981 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: If a large number of leaf node machines send requests to a central root node then that root node can become overwhelmed: The CPU becomes a bottleneck, for either processing requests or sending replies, because it can't possibly deal with the flood of requests. The network interface becomes a bottleneck because a wide fan-in causes TCP drops and retransmissions, which causes latency. Then clients start retrying requests which quickly causes a spiral of death in an undisciplined system. One solution to this problem is a strategy given by Dr. Jeff Dean , Head of Google's School of Infrastructure Wizardry, in this Stanford video presentation : Tree Distribution of Requests and Responses . Instead of having a root node connected to leaves in a flat topology, the idea is to create a tree of nodes. So a root node talks to a number of parent nodes and the parent nodes talk to a number of leaf nodes. Requests are pushed down the tree through the parents and only hit a subset
sentIndex sentText sentNum sentScore
1 If a large number of leaf node machines send requests to a central root node then that root node can become overwhelmed: The CPU becomes a bottleneck, for either processing requests or sending replies, because it can't possibly deal with the flood of requests. [sent-1, score-2.306]
2 The network interface becomes a bottleneck because a wide fan-in causes TCP drops and retransmissions, which causes latency. [sent-2, score-0.659]
3 Then clients start retrying requests which quickly causes a spiral of death in an undisciplined system. [sent-3, score-0.641]
4 One solution to this problem is a strategy given by Dr. [sent-4, score-0.053]
5 Instead of having a root node connected to leaves in a flat topology, the idea is to create a tree of nodes. [sent-6, score-1.104]
6 So a root node talks to a number of parent nodes and the parent nodes talk to a number of leaf nodes. [sent-7, score-1.936]
7 Requests are pushed down the tree through the parents and only hit a subset of the leaf nodes. [sent-8, score-0.888]
8 With this solution: Fan-in at each level of the tree is manageable . [sent-9, score-0.289]
9 The CPU cost of processing requests and responses is spread out across all the parents, which reduces the CPU and network bottlenecks. [sent-10, score-0.529]
10 Ideally the parent can provide a level of response filtering so the root only sees a subset of the response data. [sent-12, score-1.27]
11 This further reduces the network and CPU needed by the root. [sent-13, score-0.058]
12 The parent can be collocated with leaves on one rack, which keeps all that traffic off your datacenter networks. [sent-15, score-0.79]
13 Parents return the best 20-30 responses out of the 30 leaves the parent is responsible for. [sent-17, score-0.862]
14 This is a large degree of data reduction compared to the case the root had to process all that data directly. [sent-18, score-0.383]
wordName wordTfidf (topN-words)
[('parent', 0.44), ('root', 0.321), ('leaves', 0.294), ('leaf', 0.286), ('tree', 0.235), ('parents', 0.186), ('node', 0.183), ('requests', 0.183), ('causes', 0.171), ('filtering', 0.132), ('responses', 0.128), ('subset', 0.123), ('wizardry', 0.12), ('retrying', 0.12), ('retransmissions', 0.113), ('cpu', 0.111), ('bottleneck', 0.105), ('reduces', 0.099), ('spiral', 0.098), ('collocated', 0.098), ('flood', 0.09), ('overwhelmed', 0.087), ('replies', 0.083), ('ideally', 0.083), ('becomes', 0.082), ('dean', 0.076), ('stanford', 0.076), ('school', 0.075), ('topology', 0.074), ('drops', 0.072), ('flat', 0.071), ('death', 0.069), ('nodes', 0.067), ('response', 0.067), ('reduction', 0.067), ('number', 0.066), ('sees', 0.066), ('rack', 0.065), ('head', 0.063), ('degree', 0.062), ('processing', 0.061), ('jeff', 0.061), ('pushed', 0.058), ('network', 0.058), ('possibly', 0.058), ('keeps', 0.056), ('sending', 0.054), ('level', 0.054), ('solution', 0.053), ('central', 0.052)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 981 high scalability-2011-02-01-Google Strategy: Tree Distribution of Requests and Responses
Introduction: If a large number of leaf node machines send requests to a central root node then that root node can become overwhelmed: The CPU becomes a bottleneck, for either processing requests or sending replies, because it can't possibly deal with the flood of requests. The network interface becomes a bottleneck because a wide fan-in causes TCP drops and retransmissions, which causes latency. Then clients start retrying requests which quickly causes a spiral of death in an undisciplined system. One solution to this problem is a strategy given by Dr. Jeff Dean , Head of Google's School of Infrastructure Wizardry, in this Stanford video presentation : Tree Distribution of Requests and Responses . Instead of having a root node connected to leaves in a flat topology, the idea is to create a tree of nodes. So a root node talks to a number of parent nodes and the parent nodes talk to a number of leaf nodes. Requests are pushed down the tree through the parents and only hit a subset
Introduction: This is a guest post by Ron Pressler, the founder and CEO of Parallel Universe , a Y Combinator company building advanced middleware for real-time applications. A little over a month ago, we open-sourced a new in-memory data grid called Galaxy . An in-memory data grid, or IMDG, is a clustered data storage and processing middleware that uses RAM as the authoritative and primary storage, and distributes data over a cluster for purposes of data and processing scalability and high-availability. A common feature of IMDGs is co-location of code and data, meaning that application code runs on all cluster nodes, each instance processing those data items residing in the local node's RAM. While quite a few commercial and open-source IMDGs are available (like Terracotta, Gigaspaces, Oracle Coherence, GemFire, Websphere eXtreme Scale, Infinispan and Hazelcast), Galaxy has adopted a completely different architecture from all other IMDGs, to service some usage scenarios ill-fitted to the othe
3 0.25627047 1207 high scalability-2012-03-12-Google: Taming the Long Latency Tail - When More Machines Equals Worse Results
Introduction: Likewise the current belief that, in the case of artificial machines the very large and the very small are equally feasible and lasting is a manifest error. Thus, for example, a small obelisk or column or other solid figure can certainly be laid down or set up without danger of breaking, while the large ones will go to pieces under the slightest provocation, and that purely on account of their own weight. -- Galileo Galileo observed how things broke if they were naively scaled up. Interestingly, Google noticed a similar pattern when building larger software systems using the same techniques used to build smaller systems. Luiz André Barroso , Distinguished Engineer at Google, talks about this fundamental property of scaling systems in his fascinating talk, Warehouse-Scale Computing: Entering the Teenage Decade . Google found the larger the scale the greater the impact of latency variability. When a request is implemented by work done in parallel, as is common with today's service
4 0.18633582 1236 high scalability-2012-04-30-Masstree - Much Faster than MongoDB, VoltDB, Redis, and Competitive with Memcached
Introduction: The EuroSys 2012 system conference has an excellent live blog summary of their talks for: Day 1 , Day 2 , Day 3 (thanks Henry at the Paper Trail blog ). Summaries for each of the accepted papers are here . One of the more interesting papers from a NoSQL perspective was Cache Craftiness for Fast Multicore Key-Value Storage , a wonderfully detailed description of the low level techniques used to implement Masstree: A storage system specialized for key-value data in which all data fits in memory, but must persist across server restarts. It supports arbitrary, variable-length keys. It allows range queries over those keys: clients can traverse subsets of the database, or the whole database, in sorted order by key. On a 16-core machine Masstree achieves six to ten million operations per second on parts A–C of the Yahoo! Cloud Serving Benchmark benchmark, more than 30 as fast as VoltDB [5] or MongoDB [2]. If you are looking for innovative detailed high performance design, t
5 0.12213776 1138 high scalability-2011-11-07-10 Core Architecture Pattern Variations for Achieving Scalability
Introduction: Srinath Perera has put together a strong list of architecture patterns based on three meta patterns: distribution, caching, and asynchronous processing. He contends these three are the primal patterns and the following patterns are but different combinations: LB (Load Balancers) + Shared nothing Units . Units that do not share anything with each other fronted with a load balancer that routes incoming messages to a unit based on some criteria. LB + Stateless Nodes + Scalable Storage . Several stateless nodes talking to a scalable storage, and a load balancer distributes load among the nodes. Peer to Peer Architectures (Distributed Hash Table (DHT) and Content Addressable Networks (CAN)) . Algorithm for scaling up logarithmically. Distributed Queues . Queue implementation (FIFO delivery) implemented as a network service. Publish/Subscribe Paradigm . Network publish subscribe brokers that route messages to each other. Gossip and Nature-inspired Architectures . Each
7 0.11376063 1421 high scalability-2013-03-11-Low Level Scalability Solutions - The Conditioning Collection
9 0.10337118 1529 high scalability-2013-10-08-F1 and Spanner Holistically Compared
10 0.10008232 1413 high scalability-2013-02-27-42 Monster Problems that Attack as Loads Increase
11 0.094012231 249 high scalability-2008-02-16-S3 Failed Because of Authentication Overload
12 0.089671247 1051 high scalability-2011-06-01-Why is your network so slow? Your switch should tell you.
13 0.087748878 666 high scalability-2009-07-30-Learn How to Think at Scale
14 0.087712891 1622 high scalability-2014-03-31-How WhatsApp Grew to Nearly 500 Million Users, 11,000 cores, and 70 Million Messages a Second
15 0.084974825 728 high scalability-2009-10-26-Facebook's Memcached Multiget Hole: More machines != More Capacity
16 0.083124697 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
17 0.076651685 1591 high scalability-2014-02-05-Little’s Law, Scalability and Fault Tolerance: The OS is your bottleneck. What you can do?
18 0.076542929 1276 high scalability-2012-07-04-Top Features of a Scalable Database
19 0.075582527 1038 high scalability-2011-05-11-Troubleshooting response time problems – why you cannot trust your system metrics
20 0.074141257 960 high scalability-2010-12-20-Netflix: Use Less Chatty Protocols in the Cloud - Plus 26 Fixes
topicId topicWeight
[(0, 0.101), (1, 0.082), (2, -0.006), (3, -0.002), (4, -0.033), (5, 0.017), (6, 0.093), (7, 0.065), (8, -0.06), (9, 0.001), (10, 0.016), (11, -0.007), (12, -0.009), (13, -0.026), (14, 0.028), (15, 0.031), (16, 0.018), (17, -0.013), (18, 0.031), (19, -0.022), (20, 0.037), (21, 0.038), (22, 0.038), (23, -0.029), (24, 0.026), (25, 0.043), (26, 0.023), (27, 0.016), (28, 0.006), (29, -0.022), (30, 0.076), (31, -0.031), (32, 0.026), (33, 0.028), (34, 0.039), (35, 0.065), (36, 0.025), (37, -0.064), (38, -0.057), (39, -0.002), (40, 0.036), (41, -0.038), (42, 0.028), (43, -0.027), (44, -0.023), (45, 0.004), (46, -0.0), (47, -0.012), (48, -0.012), (49, 0.044)]
simIndex simValue blogId blogTitle
same-blog 1 0.97919911 981 high scalability-2011-02-01-Google Strategy: Tree Distribution of Requests and Responses
Introduction: If a large number of leaf node machines send requests to a central root node then that root node can become overwhelmed: The CPU becomes a bottleneck, for either processing requests or sending replies, because it can't possibly deal with the flood of requests. The network interface becomes a bottleneck because a wide fan-in causes TCP drops and retransmissions, which causes latency. Then clients start retrying requests which quickly causes a spiral of death in an undisciplined system. One solution to this problem is a strategy given by Dr. Jeff Dean , Head of Google's School of Infrastructure Wizardry, in this Stanford video presentation : Tree Distribution of Requests and Responses . Instead of having a root node connected to leaves in a flat topology, the idea is to create a tree of nodes. So a root node talks to a number of parent nodes and the parent nodes talk to a number of leaf nodes. Requests are pushed down the tree through the parents and only hit a subset
Introduction: In Taming The Long Latency Tail we covered Luiz Barroso ’s exploration of the long tail latency (some operations are really slow) problems generated by large fanout architectures (a request is composed of potentially thousands of other requests). You may have noticed there weren’t a lot of solutions. That’s where a talk I attended, Achieving Rapid Response Times in Large Online Services ( slide deck ), by Jeff Dean , also of Google, comes in: In this talk, I’ll describe a collection of techniques and practices lowering response times in large distributed systems whose components run on shared clusters of machines, where pieces of these systems are subject to interference by other tasks, and where unpredictable latency hiccups are the norm, not the exception. The goal is to use software techniques to reduce variability given the increasing variability in underlying hardware, the need to handle dynamic workloads on a shared infrastructure, and the need to use lar
Introduction: Likewise the current belief that, in the case of artificial machines the very large and the very small are equally feasible and lasting is a manifest error. Thus, for example, a small obelisk or column or other solid figure can certainly be laid down or set up without danger of breaking, while the large ones will go to pieces under the slightest provocation, and that purely on account of their own weight. -- Galileo Galileo observed how things broke if they were naively scaled up. Interestingly, Google noticed a similar pattern when building larger software systems using the same techniques used to build smaller systems. Luiz André Barroso , Distinguished Engineer at Google, talks about this fundamental property of scaling systems in his fascinating talk, Warehouse-Scale Computing: Entering the Teenage Decade . Google found the larger the scale the greater the impact of latency variability. When a request is implemented by work done in parallel, as is common with today's service
Introduction: You go through the expense of installing CDNs all over the globe to make sure users always have a node close by and you notice something curious and furious: clients still experience poor latencies. What's up with that? What do you do to find the problem? If you are Google you build a tool (WhyHigh) to figure out what's up. This paper is about the tool and the unexpected problem of high latencies on CDNs. The main problems they found: inefficient routing to nearby nodes and packet queuing. But more useful is the architecture of WhyHigh and how it goes about identifying bottle necks. And even more useful is the general belief in creating sophisticated tools to understand and improve your service. That's what professionals do. From the abstract: Replicating content across a geographically distributed set of servers and redirecting clients to the closest server in terms of latency has emerged as a common paradigm for improving client performance. In this paper, we analyze latenc
5 0.715132 1558 high scalability-2013-12-04-How Can Batching Requests Actually Reduce Latency?
Introduction: Jeremy Edberg gave a talk on Scaling Reddit from 1 Million to 1 Billion–Pitfalls and Lessons and one of the issues they had was that they: Did not account for increased latency after moving to EC2. In the datacenter they had submillisecond access between machines so it was possible to make a 1000 calls to memache for one page load. Not so on EC2. Memcache access times increased 10x to a millisecond which made their old approach unusable. Fix was to batch calls to memcache so a large number of gets are in one request. Dave Pacheco had an interesting question about batching requests and its impact on latency: I was confused about the memcached problem after moving to the cloud. I understand why network latency may have gone from submillisecond to milliseconds, but how could you improve latency by batching requests? Shouldn't that improve efficiency, not latency, at the possible expense of latency (since some requests will wait on the client as they get batched)?
6 0.70332742 661 high scalability-2009-07-25-Latency is Everywhere and it Costs You Sales - How to Crush it
8 0.67955065 946 high scalability-2010-11-22-Strategy: Google Sends Canary Requests into the Data Mine
9 0.67530221 960 high scalability-2010-12-20-Netflix: Use Less Chatty Protocols in the Cloud - Plus 26 Fixes
10 0.66919851 1421 high scalability-2013-03-11-Low Level Scalability Solutions - The Conditioning Collection
12 0.65495205 1413 high scalability-2013-02-27-42 Monster Problems that Attack as Loads Increase
13 0.65365165 772 high scalability-2010-02-05-High Availability Principle : Concurrency Control
14 0.65220344 941 high scalability-2010-11-15-How Google's Instant Previews Reduces HTTP Requests
15 0.64919025 1622 high scalability-2014-03-31-How WhatsApp Grew to Nearly 500 Million Users, 11,000 cores, and 70 Million Messages a Second
17 0.62984699 1116 high scalability-2011-09-15-Paper: It's Time for Low Latency - Inventing the 1 Microsecond Datacenter
18 0.62228882 1001 high scalability-2011-03-09-Google and Netflix Strategy: Use Partial Responses to Reduce Request Sizes
19 0.62096369 1221 high scalability-2012-04-03-Hazelcast 2.0: Big Data In-Memory
20 0.6178413 1415 high scalability-2013-03-04-7 Life Saving Scalability Defenses Against Load Monster Attacks
topicId topicWeight
[(1, 0.117), (2, 0.161), (10, 0.045), (14, 0.255), (56, 0.043), (61, 0.087), (77, 0.017), (79, 0.072), (83, 0.01), (85, 0.017), (94, 0.068)]
simIndex simValue blogId blogTitle
1 0.88876134 405 high scalability-2008-10-07-Help a Scoble out. What should Robert ask in his scalability interview?
Introduction: One of the cool things about Mr. Scoble is he doesn't pretend to know everything, which can be an deadly boring affliction in this field. In this case Robert is asking for help in an upcoming interview. Maybe we can help? Here's Robert's plight: I’m really freaked out. I have one of the biggest interviews of my life coming up and I’m way under qualified to host it. It’s on Thursday and it’s about Scalability and Performance of Web Services. Look at who will be on. Matt Mullenweg, founder of Automattic, the company behind WordPress (and behind this blog). Paul Bucheit, one of the founders of FriendFeed and the creator of Gmail (he’s also the guy who gave Google the “don’t be evil” admonishion). Nat Brown, CTO of iLike, which got six million users on Facebook in about 10 days. What would you ask?
2 0.8757391 725 high scalability-2009-10-21-Manage virtualized sprawl with VRMs
Introduction: The essence of my work is coming into daily contact with innovative technologies. A recent example was at the request of a partner company who wanted to answer- which one of these tools will best solve my virtualized datacenter headache? After initial analysis all the products could be classified as tools that troubleshoot VM sprawl, but there was no universally accepted term for them. The most descriptive term that I found was Virtual Resource Manager (VRM) from DynamicOps . As I delved deeper into their workings, the distinction between VRMs and Private Clouds became blurred. What are the differences? Read more at: http://bigdatamatters.com/bigdatamatters/2009/10/cloud-vs-vrm.html
same-blog 3 0.87440652 981 high scalability-2011-02-01-Google Strategy: Tree Distribution of Requests and Responses
Introduction: If a large number of leaf node machines send requests to a central root node then that root node can become overwhelmed: The CPU becomes a bottleneck, for either processing requests or sending replies, because it can't possibly deal with the flood of requests. The network interface becomes a bottleneck because a wide fan-in causes TCP drops and retransmissions, which causes latency. Then clients start retrying requests which quickly causes a spiral of death in an undisciplined system. One solution to this problem is a strategy given by Dr. Jeff Dean , Head of Google's School of Infrastructure Wizardry, in this Stanford video presentation : Tree Distribution of Requests and Responses . Instead of having a root node connected to leaves in a flat topology, the idea is to create a tree of nodes. So a root node talks to a number of parent nodes and the parent nodes talk to a number of leaf nodes. Requests are pushed down the tree through the parents and only hit a subset
4 0.844091 599 high scalability-2009-05-14-Who Has the Most Web Servers?
Introduction: An interesting post on DataCenterKnowledge! 1&1 Internet: 55,000 servers Rackspace: 50,038 servers The Planet: 48,500 servers Akamai Technologies: 48,000 servers OVH: 40,000 servers SBC Communications: 29,193 servers Verizon: 25,788 servers Time Warner Cable: 24,817 servers SoftLayer: 21,000 servers AT&T;: 20,268 servers iWeb: 10,000 servers How about Google , Microsoft, Amazon , eBay , Yahoo, GoDaddy, Facebook? Check out the post on DataCenterKnowledge and of course here on highscalability.com!
5 0.83229029 441 high scalability-2008-11-13-CloudCamp London 2: private clouds and standardisation
Introduction: CloudCamp returned to London yesterday, organised with the help of Skills Matter at the Crypt on the Clarkenwell green. The main topics of this cloud/grid computing community meeting were service-level agreements, connecting private and public clouds and standardisation issues.
6 0.82037663 1253 high scalability-2012-05-28-The Anatomy of Search Technology: Crawling using Combinators
7 0.80460775 694 high scalability-2009-09-04-Hot Links for 2009-9-4
8 0.79359543 495 high scalability-2009-01-17-Intro to Caching,Caching algorithms and caching frameworks part 1
9 0.78702921 537 high scalability-2009-03-12-QCon London 2009: Database projects to watch closely
10 0.77237201 128 high scalability-2007-10-21-Paper: Standardizing Storage Clusters (with pNFS)
11 0.76191062 487 high scalability-2009-01-08-Paper: Sharding with Oracle Database
12 0.75396991 744 high scalability-2009-11-24-Hot Scalability Links for Nov 24 2009
13 0.74381959 1278 high scalability-2012-07-06-Stuff The Internet Says On Scalability For July 6, 2012
14 0.68891102 1233 high scalability-2012-04-25-The Anatomy of Search Technology: blekko’s NoSQL database
15 0.68060929 659 high scalability-2009-07-20-A Scalability Lament
16 0.67896318 1180 high scalability-2012-01-24-The State of NoSQL in 2012
18 0.6732825 1334 high scalability-2012-10-04-Stuff The Internet Says On Scalability For October 5, 2012
19 0.67232561 711 high scalability-2009-09-22-How Ravelry Scales to 10 Million Requests Using Rails
20 0.67151463 1329 high scalability-2012-09-26-WordPress.com Serves 70,000 req-sec and over 15 Gbit-sec of Traffic using NGINX