high_scalability high_scalability-2010 high_scalability-2010-897 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Jesper Söderlund put together an excellent list of four general scalability patterns and four subpatterns in his post Scalability patterns and an interesting story : Load distribution - Spread the system load across multiple processing units Load balancing / load sharing - Spreading the load across many components with equal properties for handling the request Partitioning - Spreading the load across many components by routing an individual request to a component that owns that data specific Vertical partitioning - Spreading the load across the functional boundaries of a problem space, separate functions being handled by different processing units Horizontal partitioning - Spreading a single type of data element across many instances, according to some partitioning key, e.g. hashing the player id and doing a modulus operation, etc. Quite often referred to as sharding. Queuing and batch - Achieve efficiencies of scale by
sentIndex sentText sentNum sentScore
1 hashing the player id and doing a modulus operation, etc. [sent-3, score-0.457]
wordName wordTfidf (topN-words)
[('spreading', 0.479), ('processing', 0.243), ('across', 0.205), ('immediacy', 0.19), ('load', 0.185), ('modulus', 0.179), ('soderlund', 0.179), ('jesper', 0.17), ('regards', 0.159), ('four', 0.156), ('patterns', 0.151), ('amortized', 0.151), ('efficiencies', 0.145), ('operation', 0.141), ('owns', 0.135), ('referred', 0.131), ('explanations', 0.131), ('batches', 0.126), ('boundaries', 0.119), ('components', 0.117), ('element', 0.114), ('multiple', 0.107), ('constraints', 0.106), ('hashing', 0.105), ('fall', 0.103), ('player', 0.103), ('equal', 0.103), ('many', 0.096), ('properties', 0.092), ('according', 0.09), ('functional', 0.083), ('handled', 0.078), ('overhead', 0.077), ('task', 0.077), ('original', 0.077), ('spread', 0.077), ('batch', 0.077), ('partitioning', 0.076), ('component', 0.076), ('functions', 0.075), ('routing', 0.074), ('story', 0.073), ('achieve', 0.073), ('data', 0.071), ('id', 0.07), ('storing', 0.067), ('individual', 0.067), ('techniques', 0.067), ('usually', 0.064), ('balancing', 0.063)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 897 high scalability-2010-09-08-4 General Core Scalability Patterns
Introduction: Jesper Söderlund put together an excellent list of four general scalability patterns and four subpatterns in his post Scalability patterns and an interesting story : Load distribution - Spread the system load across multiple processing units Load balancing / load sharing - Spreading the load across many components with equal properties for handling the request Partitioning - Spreading the load across many components by routing an individual request to a component that owns that data specific Vertical partitioning - Spreading the load across the functional boundaries of a problem space, separate functions being handled by different processing units Horizontal partitioning - Spreading a single type of data element across many instances, according to some partitioning key, e.g. hashing the player id and doing a modulus operation, etc. Quite often referred to as sharding. Queuing and batch - Achieve efficiencies of scale by
2 0.46164829 906 high scalability-2010-09-22-Applying Scalability Patterns to Infrastructure Architecture
Introduction: Too often software design patterns are overlooked by network and application delivery network architects but these patterns are often equally applicable to addressing a broad range of architectural challenges in the application delivery tier of the data center. By Lori Mac Vittie, F5 Networks The “ High Scalability ” blog is fast becoming one of my favorite reads. Last week did not disappoint with a post highlighting a set of scalability design patterns that was, apparently, inspired by yet another High Scalability post on “ 6 Ways to Kill Your Servers: Learning to Scale the Hard Way. ” Credit:Michael Chow/azcentral.com This particular post caught my attention primarily because although I’ve touched on many of these patterns in the past, I’ve never thought to call them what they are: scalability patterns. That’s probably a side-effect of forgetting that building an architecture of any kind is at its core computer science and thus
3 0.19930054 898 high scalability-2010-09-09-6 Scalability Lessons
Introduction: Jesper Söderlund not only put together a few interesting scalability patterns , he also came up with a few interesting scalability lessons : Lesson #1 . Put Smarty compile and template caches on an active-active DRBD cluster with high load and your servers will DIE! Lesson #2 . Don't use out-of-the-box configurations. Lesson #3 . Single points of contention will eventually become a bottleneck. Lesson #4 . Plan in advance. Lesson #5 . Offload your databases as much as possible. Lesson #6 . File systems matter and can run out of space / inodes. For more details and explanations see the original post.
4 0.18034691 1325 high scalability-2012-09-19-The 4 Building Blocks of Architecting Systems for Scale
Introduction: If you are looking for an excellent overview of general architecture principles then take a look at Will Larson's Introduction to Architecting Systems for Scale . Based on his experiences at Yahoo! and Digg, Will covers key concepts in some depth. A quick gloss on the building blocks: Load Balancing: Scalability & Redundancy . Horizontal scalability and redundancy are usually achieved via load balancing, the spreading of requests across multiple resources. Smart Clients . The client has a list of hosts and load balances across that list of hosts. Upside is simple for programmers. Downside is it's hard to update and change. Hardware Load Balancers . Targeted at larger companies, this is dedicated load balancing hardware. Upside is performance. Downside is cost and complexity. Software Load Balancers . The recommended approach, it's software that handles load balancing, health checks, etc. Caching . Make better use of resources you already have. Pr
5 0.16719547 1313 high scalability-2012-08-28-Making Hadoop Run Faster
Introduction: Making Hadoop Run Faster One of the challenges in processing data is that the speed at which we can input data is quite often much faster than the speed at which we can process it. This problem becomes even more pronounced in the context of Big Data, where the volume of data keeps on growing, along with a corresponding need for more insights, and thus the need for more complex processing also increases. Batch Processing to the Rescue Hadoop was designed to deal with this challenge in the following ways: 1. Use a distributed file system: This enables us to spread the load and grow our system as needed. 2. Optimize for write speed: To enable fast writes the Hadoop architecture was designed so that writes are first logged, and then processed. This enables fairly fast write speeds. 3. Use batch processing (Map/Reduce) to balance the speed for the data feeds with the processing speed. Batch Processing Challenges The challenge with batch-processing is that it assumes
6 0.12619784 891 high scalability-2010-09-01-Scale-out vs Scale-up
7 0.099094152 770 high scalability-2010-02-03-NoSQL Means Never Having to Store Blobs Again
8 0.09765397 400 high scalability-2008-10-01-The Pattern Bible for Distributed Computing
9 0.09735778 435 high scalability-2008-10-30-The case for functional decomposition
10 0.091611259 427 high scalability-2008-10-22-Server load balancing architectures, Part 2: Application-level load balancing
11 0.091558918 668 high scalability-2009-08-01-15 Scalability and Performance Best Practices
12 0.089414023 674 high scalability-2009-08-07-The Canonical Cloud Architecture
13 0.087407425 1514 high scalability-2013-09-09-Need Help with Database Scalability? Understand I-O
14 0.085584715 1216 high scalability-2012-03-27-Big Data In the Cloud Using Cloudify
15 0.082357541 21 high scalability-2007-07-23-GoogleTalk Architecture
16 0.081328981 1138 high scalability-2011-11-07-10 Core Architecture Pattern Variations for Achieving Scalability
17 0.080947809 448 high scalability-2008-11-22-Google Architecture
18 0.080843128 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
19 0.080349773 892 high scalability-2010-09-02-Distributed Hashing Algorithms by Example: Consistent Hashing
20 0.079264581 1509 high scalability-2013-08-30-Stuff The Internet Says On Scalability For August 30, 2013
topicId topicWeight
[(0, 0.122), (1, 0.089), (2, -0.025), (3, -0.017), (4, -0.006), (5, 0.039), (6, 0.065), (7, -0.052), (8, -0.086), (9, 0.014), (10, 0.043), (11, 0.056), (12, -0.034), (13, -0.064), (14, -0.0), (15, -0.002), (16, 0.056), (17, -0.021), (18, 0.02), (19, 0.062), (20, 0.001), (21, 0.036), (22, -0.003), (23, -0.038), (24, 0.051), (25, -0.02), (26, 0.0), (27, 0.072), (28, 0.0), (29, 0.077), (30, 0.046), (31, 0.038), (32, 0.001), (33, 0.07), (34, -0.056), (35, -0.07), (36, 0.02), (37, -0.066), (38, -0.007), (39, -0.082), (40, 0.011), (41, -0.015), (42, -0.074), (43, 0.021), (44, 0.063), (45, 0.039), (46, 0.015), (47, -0.055), (48, 0.036), (49, -0.002)]
simIndex simValue blogId blogTitle
same-blog 1 0.96304893 897 high scalability-2010-09-08-4 General Core Scalability Patterns
Introduction: Jesper Söderlund put together an excellent list of four general scalability patterns and four subpatterns in his post Scalability patterns and an interesting story : Load distribution - Spread the system load across multiple processing units Load balancing / load sharing - Spreading the load across many components with equal properties for handling the request Partitioning - Spreading the load across many components by routing an individual request to a component that owns that data specific Vertical partitioning - Spreading the load across the functional boundaries of a problem space, separate functions being handled by different processing units Horizontal partitioning - Spreading a single type of data element across many instances, according to some partitioning key, e.g. hashing the player id and doing a modulus operation, etc. Quite often referred to as sharding. Queuing and batch - Achieve efficiencies of scale by
2 0.81311357 906 high scalability-2010-09-22-Applying Scalability Patterns to Infrastructure Architecture
Introduction: Too often software design patterns are overlooked by network and application delivery network architects but these patterns are often equally applicable to addressing a broad range of architectural challenges in the application delivery tier of the data center. By Lori Mac Vittie, F5 Networks The “ High Scalability ” blog is fast becoming one of my favorite reads. Last week did not disappoint with a post highlighting a set of scalability design patterns that was, apparently, inspired by yet another High Scalability post on “ 6 Ways to Kill Your Servers: Learning to Scale the Hard Way. ” Credit:Michael Chow/azcentral.com This particular post caught my attention primarily because although I’ve touched on many of these patterns in the past, I’ve never thought to call them what they are: scalability patterns. That’s probably a side-effect of forgetting that building an architecture of any kind is at its core computer science and thus
3 0.76196283 1138 high scalability-2011-11-07-10 Core Architecture Pattern Variations for Achieving Scalability
Introduction: Srinath Perera has put together a strong list of architecture patterns based on three meta patterns: distribution, caching, and asynchronous processing. He contends these three are the primal patterns and the following patterns are but different combinations: LB (Load Balancers) + Shared nothing Units . Units that do not share anything with each other fronted with a load balancer that routes incoming messages to a unit based on some criteria. LB + Stateless Nodes + Scalable Storage . Several stateless nodes talking to a scalable storage, and a load balancer distributes load among the nodes. Peer to Peer Architectures (Distributed Hash Table (DHT) and Content Addressable Networks (CAN)) . Algorithm for scaling up logarithmically. Distributed Queues . Queue implementation (FIFO delivery) implemented as a network service. Publish/Subscribe Paradigm . Network publish subscribe brokers that route messages to each other. Gossip and Nature-inspired Architectures . Each
4 0.75237 1325 high scalability-2012-09-19-The 4 Building Blocks of Architecting Systems for Scale
Introduction: If you are looking for an excellent overview of general architecture principles then take a look at Will Larson's Introduction to Architecting Systems for Scale . Based on his experiences at Yahoo! and Digg, Will covers key concepts in some depth. A quick gloss on the building blocks: Load Balancing: Scalability & Redundancy . Horizontal scalability and redundancy are usually achieved via load balancing, the spreading of requests across multiple resources. Smart Clients . The client has a list of hosts and load balances across that list of hosts. Upside is simple for programmers. Downside is it's hard to update and change. Hardware Load Balancers . Targeted at larger companies, this is dedicated load balancing hardware. Upside is performance. Downside is cost and complexity. Software Load Balancers . The recommended approach, it's software that handles load balancing, health checks, etc. Caching . Make better use of resources you already have. Pr
5 0.71491194 951 high scalability-2010-12-01-8 Commonly Used Scalable System Design Patterns
Introduction: Ricky Ho in Scalable System Design Patterns has created a great list of scalability patterns along with very well done explanatory graphics. A summary of the patterns are: Load Balancer - a dispatcher determines which worker instance will handle a request based on different policies. Scatter and Gather - a dispatcher multicasts requests to all workers in a pool. Each worker will compute a local result and send it back to the dispatcher, who will consolidate them into a single response and then send back to the client. Result Cache - a dispatcher will first lookup if the request has been made before and try to find the previous result to return, in order to save the actual execution. Shared Space - all workers monitors information from the shared space and contributes partial knowledge back to the blackboard. The information is continuously enriched until a solution is reached. Pipe and Filter - all workers connected by pipes across which data flows. MapReduc
6 0.69677454 1260 high scalability-2012-06-07-Case Study on Scaling PaaS infrastructure
7 0.64637339 604 high scalability-2009-05-20-Paper: Flux: An Adaptive Partitioning Operator for Continuous Query Systems
8 0.63827306 492 high scalability-2009-01-16-Database Sharding for startups
9 0.60938215 427 high scalability-2008-10-22-Server load balancing architectures, Part 2: Application-level load balancing
10 0.60511321 1276 high scalability-2012-07-04-Top Features of a Scalable Database
11 0.60133719 958 high scalability-2010-12-16-7 Design Patterns for Almost-infinite Scalability
12 0.59803623 109 high scalability-2007-10-03-Save on a Load Balancer By Using Client Side Load Balancing
13 0.59607601 1646 high scalability-2014-05-12-4 Architecture Issues When Scaling Web Applications: Bottlenecks, Database, CPU, IO
14 0.58552188 1077 high scalability-2011-07-11-ATMCash Exploits Virtualization for Security - Immutability and Reversion
15 0.58230275 595 high scalability-2009-05-08-Publish-subscribe model does not scale?
16 0.57896411 898 high scalability-2010-09-09-6 Scalability Lessons
17 0.57564658 1329 high scalability-2012-09-26-WordPress.com Serves 70,000 req-sec and over 15 Gbit-sec of Traffic using NGINX
18 0.57457143 983 high scalability-2011-02-02-Piccolo - Building Distributed Programs that are 11x Faster than Hadoop
20 0.57380456 425 high scalability-2008-10-22-Scalability Best Practices: Lessons from eBay
topicId topicWeight
[(2, 0.228), (61, 0.016), (74, 0.286), (79, 0.166), (85, 0.052), (94, 0.137)]
simIndex simValue blogId blogTitle
1 0.8930006 1263 high scalability-2012-06-13-Why My Soap Film is Better than Your Hadoop Cluster
Introduction: The ever amazing slime mold is not the only way to solve complex compute problems without performing calculations. There is another: soap film . Unfortunately for soap film it isn’t nearly as photogenic as slime mold, all we get are boring looking pictures, but the underlying idea is still fascinating and ten times less spooky. As a quick introduction we’ll lean on Long Ouyang, who has really straightforward explanation of how soap film works in Approaching P=NP: Can Soap Bubbles Solve The Steiner Tree Problem In Polynomial . It’s computers, so playing the role of the motivating graph problem we have the Steiner tree problem , which Ouyang explains as: Find the minimum spanning tree for a bunch of vertices, given that you can add additional points. Soap helps solve this problem because: Soap, in water, acts a surfactant, which decreases the surface tension in water. This acts to minimize the surface energy of the liquid. This should minimize surf
same-blog 2 0.84657055 897 high scalability-2010-09-08-4 General Core Scalability Patterns
Introduction: Jesper Söderlund put together an excellent list of four general scalability patterns and four subpatterns in his post Scalability patterns and an interesting story : Load distribution - Spread the system load across multiple processing units Load balancing / load sharing - Spreading the load across many components with equal properties for handling the request Partitioning - Spreading the load across many components by routing an individual request to a component that owns that data specific Vertical partitioning - Spreading the load across the functional boundaries of a problem space, separate functions being handled by different processing units Horizontal partitioning - Spreading a single type of data element across many instances, according to some partitioning key, e.g. hashing the player id and doing a modulus operation, etc. Quite often referred to as sharding. Queuing and batch - Achieve efficiencies of scale by
3 0.82382953 566 high scalability-2009-04-13-High Performance Web Pages – Real World Examples: Netflix Case Study
Introduction: This read will provide you with information about how Netflix deals with high load on their movie rental website. It was written by Bill Scott in the fall of 2008. Read or download the PDF file here
4 0.80720812 390 high scalability-2008-09-23-Scaling your cookie recipes
Introduction: This article on scaling cookie baking recipes showed up in one my key word alerts. Lots of weird things show up in alerts, but I really like cookies and the parallels were just so delicious. Scaling in the cookie baking world is: the process of multiplying your recipe by many times to produce much more dough for many more cookies. It’s the difference between making enough dough in one batch to make two dozen cookies, or 2000 cookies. Hey, pretty close to the website notion. Yet as any good cook knows any scaled up recipe must be tweaked a little as things change at scale. Let's see what else we're supposed to do (quoted from the article): Be Patient - When making large batches of cookies, the most important thing that you have to remember is not to rush. Use Fresh Ingredients - This is always an important thing to keep in mind. Don’t use as much leavening - When you’re making a large batch of cookie dough, remember to scale down the amount of baking powder that you
5 0.73472053 581 high scalability-2009-04-26-Map-Reduce for Machine Learning on Multicore
Introduction: We are at the beginning of the multicore era. Computers will have increasingly many cores (processors), but there is still no good programming framework for these architectures, and thus no simple and unified way for machine learning to take advantage of the potential speed up. In this paper, we develop a broadly applicable parallel programming method, one that is easily applied to many different learning algorithms. Our work is in distinct contrast to the tradition in machine learning of designing (often ingenious) ways to speed up a single algorithm at a time. Specifically, we show that algorithms that fit the Statistical Query model can be written in a certain “summation form,” which allows them to be easily parallelized on multicore computers. We adapt Google’s map-reduce paradigm to demonstrate this parallel speed up technique on a variety of learning algorithms including locally weighted linear regression (LWLR), k-means, logistic regression (LR), naive Bayes (NB), SVM
6 0.70263731 1063 high scalability-2011-06-17-Stuff The Internet Says On Scalability For June 17, 2011
7 0.69445056 533 high scalability-2009-03-11-The Implications of Punctuated Scalabilium for Website Architecture
8 0.69397616 1223 high scalability-2012-04-06-Stuff The Internet Says On Scalability For April 6, 2012
9 0.69053006 1023 high scalability-2011-04-14-Strategy: Cache Application Start State to Reduce Spin-up Times
10 0.68306363 409 high scalability-2008-10-13-Challenges from large scale computing at Google
11 0.66669947 613 high scalability-2009-06-01-Data grid comparison: Oracle Coherence vs Gigaspaces XAP
12 0.66641879 863 high scalability-2010-07-22-How can we spark the movement of research out of the Ivory Tower and into production?
13 0.66595674 1561 high scalability-2013-12-09-Site Moves from PHP to Facebook's HipHop, Now Pages Load in .6 Seconds Instead of Five
14 0.66360807 227 high scalability-2008-01-28-Howto setup GFS-GNBD
15 0.66101587 229 high scalability-2008-01-29-Building scalable storage into application - Instead of MogileFS OpenAFS etc.
16 0.65978658 845 high scalability-2010-06-22-Exploring the software behind Facebook, the world’s largest site
17 0.6563139 188 high scalability-2007-12-19-How can I learn to scale my project?
18 0.65532601 213 high scalability-2008-01-15-Does Sun Buying MySQL Change Your Scaling Strategy?
19 0.65446091 17 high scalability-2007-07-16-Paper: Guide to Cost-effective Database Scale-Out using MySQL
20 0.65411377 76 high scalability-2007-08-29-Skype Failed the Boot Scalability Test: Is P2P fundamentally flawed?