high_scalability high_scalability-2010 high_scalability-2010-772 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: One important high availability principle is concurrency control. The idea is to allow only that much traffic through to your system which your system can handle successfully. For example: if your system is certified to handle a concurrency of 100 then the 101st request should either timeout, be asked to try later or wait until one of the previous 100 requests finish. The 101st request should not be allowed to negatively impact the experience of the other 100 users. Only the 101st request should be impacted. Read more here...
sentIndex sentText sentNum sentScore
1 The idea is to allow only that much traffic through to your system which your system can handle successfully. [sent-2, score-0.872]
2 For example: if your system is certified to handle a concurrency of 100 then the 101st request should either timeout, be asked to try later or wait until one of the previous 100 requests finish. [sent-3, score-2.591]
3 The 101st request should not be allowed to negatively impact the experience of the other 100 users. [sent-4, score-1.085]
wordName wordTfidf (topN-words)
[('request', 0.401), ('certified', 0.359), ('concurrency', 0.333), ('timeout', 0.287), ('principle', 0.266), ('negatively', 0.224), ('allowed', 0.21), ('asked', 0.206), ('handle', 0.191), ('previous', 0.191), ('wait', 0.171), ('either', 0.158), ('impact', 0.15), ('later', 0.138), ('allow', 0.135), ('system', 0.135), ('try', 0.117), ('requests', 0.112), ('availability', 0.108), ('traffic', 0.108), ('important', 0.106), ('idea', 0.101), ('experience', 0.1), ('example', 0.093), ('one', 0.079), ('much', 0.067), ('high', 0.061)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 772 high scalability-2010-02-05-High Availability Principle : Concurrency Control
Introduction: One important high availability principle is concurrency control. The idea is to allow only that much traffic through to your system which your system can handle successfully. For example: if your system is certified to handle a concurrency of 100 then the 101st request should either timeout, be asked to try later or wait until one of the previous 100 requests finish. The 101st request should not be allowed to negatively impact the experience of the other 100 users. Only the 101st request should be impacted. Read more here...
2 0.16992356 946 high scalability-2010-11-22-Strategy: Google Sends Canary Requests into the Data Mine
Introduction: Google runs queries against thousands of in-memory index nodes in parallel and then merges the results. One of the interesting problems with this approach, explains Google's Jeff Dean in this lecture at Stanford , is the Query of Death . A query can cause a program to fail because of bugs or various other issues. This means that a single query can take down an entire cluster of machines, which is not good for availability and response times, as it takes quite a while for thousands of machines to recover. Thus the Query of Death. New queries are always coming into the system and when you are always rolling out new software, it's impossible to completely get rid of the problem. Two solutions: Test against logs . Google replays a month's worth of logs to see if any of those queries kill anything. That helps, but Queries of Death may still happen. Send a canary request . A request is sent to one machine. If the request succeeds then it will probably succeed on all machines, s
3 0.15200864 1258 high scalability-2012-06-05-Thesis: Concurrent Programming for Scalable Web Architectures
Introduction: Benjamin Erb ( @b_erb ) from Ulm University recently published his diploma thesis on "Concurrent Programming for Scalable Web Architectures" . The thesis provides a comprehensive survey on different concepts and techniques of concurrency inside web architectures, including web servers, application logic and storage backends. It incorporates research publications, hands-on reports and also regards popular programming languages, frameworks and databases. Abstract: Web architectures are an important asset for various large-scale web applications, such as social networks or e-commerce sites. Being able to handle huge numbers of users concurrently is essential, thus scalability is one of the most important features of these architectures. Multi-core processors, highly distributed backend architectures and new web technologies force us to reconsider approaches for concurrent programming in order to implement web applications and fulfil scalability demands. While focusing on dif
Introduction: In Taming The Long Latency Tail we covered Luiz Barroso ’s exploration of the long tail latency (some operations are really slow) problems generated by large fanout architectures (a request is composed of potentially thousands of other requests). You may have noticed there weren’t a lot of solutions. That’s where a talk I attended, Achieving Rapid Response Times in Large Online Services ( slide deck ), by Jeff Dean , also of Google, comes in: In this talk, I’ll describe a collection of techniques and practices lowering response times in large distributed systems whose components run on shared clusters of machines, where pieces of these systems are subject to interference by other tasks, and where unpredictable latency hiccups are the norm, not the exception. The goal is to use software techniques to reduce variability given the increasing variability in underlying hardware, the need to handle dynamic workloads on a shared infrastructure, and the need to use lar
Introduction: This is a guestrepostby Ron Pressler, the founder and CEO ofParallel Universe, a Y Combinator company building advanced middleware for real-time applications. Little's Law helps us determine the maximum request rate a server can handle. When we apply it, we find that the dominating factor limiting a server's capacity is not the hardware but theOS.Should we buy more hardware if software is the problem? If not, how can we remove that software limitation in a way that does not make the code much harder to write and understand?Many modern web applications are composed of multiple (often many)HTTPservices (this is often called a micro-service architecture). This architecture has many advantages in terms of code reuse and maintainability, scalability and fault tolerance. In this post I'd like to examine one particular bottleneck in the approach, which hinders scalability as well as fault tolerance, and various ways to deal with it (I am using the term "scalability" very loosely in this post
6 0.11871307 1421 high scalability-2013-03-11-Low Level Scalability Solutions - The Conditioning Collection
7 0.11233409 523 high scalability-2009-02-25-Relating business, systems & technology during turbulent time -By John Zachman
8 0.10916533 728 high scalability-2009-10-26-Facebook's Memcached Multiget Hole: More machines != More Capacity
9 0.097717121 77 high scalability-2007-08-30-Log Everything All the Time
10 0.097157076 662 high scalability-2009-07-27-Handle 700 Percent More Requests Using Squid and APC Cache
11 0.092589177 1299 high scalability-2012-08-06-Paper: High-Performance Concurrency Control Mechanisms for Main-Memory Databases
12 0.09002059 1207 high scalability-2012-03-12-Google: Taming the Long Latency Tail - When More Machines Equals Worse Results
13 0.090013504 783 high scalability-2010-02-24-Hot Scalability Links for February 24, 2010
14 0.088098399 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
15 0.087515213 1415 high scalability-2013-03-04-7 Life Saving Scalability Defenses Against Load Monster Attacks
16 0.086589485 259 high scalability-2008-02-25-Any Suggestions for the Architecture Template?
17 0.086589485 260 high scalability-2008-02-25-Architecture Template Advice Needed
18 0.085051022 1646 high scalability-2014-05-12-4 Architecture Issues When Scaling Web Applications: Bottlenecks, Database, CPU, IO
19 0.083015591 673 high scalability-2009-08-07-Strategy: Break Up the Memcache Dog Pile
20 0.078096703 960 high scalability-2010-12-20-Netflix: Use Less Chatty Protocols in the Cloud - Plus 26 Fixes
topicId topicWeight
[(0, 0.095), (1, 0.048), (2, -0.035), (3, -0.039), (4, -0.019), (5, -0.022), (6, 0.051), (7, 0.033), (8, -0.074), (9, -0.036), (10, 0.014), (11, 0.054), (12, 0.001), (13, -0.034), (14, -0.001), (15, -0.032), (16, 0.03), (17, 0.012), (18, 0.008), (19, -0.014), (20, 0.028), (21, 0.011), (22, 0.031), (23, -0.046), (24, 0.027), (25, -0.054), (26, -0.008), (27, 0.05), (28, -0.006), (29, -0.022), (30, 0.053), (31, 0.002), (32, 0.032), (33, 0.027), (34, 0.035), (35, 0.04), (36, 0.013), (37, -0.032), (38, -0.0), (39, -0.012), (40, 0.02), (41, -0.021), (42, 0.028), (43, -0.032), (44, -0.034), (45, -0.008), (46, -0.014), (47, -0.0), (48, 0.023), (49, -0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.95801646 772 high scalability-2010-02-05-High Availability Principle : Concurrency Control
Introduction: One important high availability principle is concurrency control. The idea is to allow only that much traffic through to your system which your system can handle successfully. For example: if your system is certified to handle a concurrency of 100 then the 101st request should either timeout, be asked to try later or wait until one of the previous 100 requests finish. The 101st request should not be allowed to negatively impact the experience of the other 100 users. Only the 101st request should be impacted. Read more here...
2 0.75808465 1421 high scalability-2013-03-11-Low Level Scalability Solutions - The Conditioning Collection
Introduction: We talked about 42 Monster Problems That Attack As Loads Increase . And in The Aggregation Collection we talked about the value of prioritizing work and making smart queues as a way of absorbing and not reflecting traffic spikes. Now we move on to our next batch of strategies where the theme is conditioning , which is the idea of shaping and controlling flows of work within your application... Use Resources Proportional To a Fixed Limit This is probably the most important rule for achieving scalability within an application. What it means: Find the resource that has a fixed limit that you know you can support. For example, a guarantee to handle a certain number of objects in memory. So if we always use resources proportional to the number of objects it is likely we can prevent resource exhaustion. Devise ways of tying what you need to do to the individual resources. Some examples: Keep a list of purchase orders with line items over $20 (or whatever). Do not keep
3 0.75653082 1415 high scalability-2013-03-04-7 Life Saving Scalability Defenses Against Load Monster Attacks
Introduction: We talked about 42 Monster Problems That Attack As Loads Increase . Here are a few ways you can defend yourself, secrets revealed by scaling masters across the ages. Note that these are low level programming level moves, not large architecture type strategies. Use Resources Proportional To a Fixed Limit This is probably the most important rule for achieving scalability within an application. What it means: Find the resource that has a fixed limit that you know you can support. For example, a guarantee to handle a certain number of objects in memory. So if we always use resources proportional to the number of objects it is likely we can prevent resource exhaustion. Devise ways of tying what you need to do to the individual resources. Some examples: Keep a list of purchase orders with line items over $20 (or whatever). Do not keep a list of the line items because the number of items can be much larger than the number of purchase orders. You have kept the resource usage
Introduction: In Taming The Long Latency Tail we covered Luiz Barroso ’s exploration of the long tail latency (some operations are really slow) problems generated by large fanout architectures (a request is composed of potentially thousands of other requests). You may have noticed there weren’t a lot of solutions. That’s where a talk I attended, Achieving Rapid Response Times in Large Online Services ( slide deck ), by Jeff Dean , also of Google, comes in: In this talk, I’ll describe a collection of techniques and practices lowering response times in large distributed systems whose components run on shared clusters of machines, where pieces of these systems are subject to interference by other tasks, and where unpredictable latency hiccups are the norm, not the exception. The goal is to use software techniques to reduce variability given the increasing variability in underlying hardware, the need to handle dynamic workloads on a shared infrastructure, and the need to use lar
5 0.71464324 1413 high scalability-2013-02-27-42 Monster Problems that Attack as Loads Increase
Introduction: For solutions take a look at: 7 Life Saving Scalability Defenses Against Load Monster Attacks . This is a look at all the bad things that can happen to your carefully crafted program as loads increase: all hell breaks lose. Sure, you can scale out or scale up, but you can also choose to program better. Make your system handle larger loads. This saves money because fewer boxes are needed and it will make the entire application more reliable and have better response times. And it can be quite satisfying as a programmer. Large Number Of Objects We usually get into scaling problems when the number of objects gets larger. Clearly resource usage of all types is stressed as the number of objects grow. Continuous Failures Makes An Infinite Event Stream During large network failure scenarios there is never time for the system recover. We are in a continual state of stress. Lots of High Priority Work For example, rerouting is a high priority activity. If there is a large amount
7 0.71036071 1282 high scalability-2012-07-12-4 Strategies for Punching Down Traffic Spikes
8 0.70998824 205 high scalability-2008-01-10-Letting Clients Know What's Changed: Push Me or Pull Me?
9 0.69622934 406 high scalability-2008-10-08-Strategy: Flickr - Do the Essential Work Up-front and Queue the Rest
10 0.6899628 1418 high scalability-2013-03-06-Low Level Scalability Solutions - The Aggregation Collection
11 0.68857807 981 high scalability-2011-02-01-Google Strategy: Tree Distribution of Requests and Responses
12 0.68586248 1429 high scalability-2013-03-25-AppBackplane - A Framework for Supporting Multiple Application Architectures
14 0.66138262 1425 high scalability-2013-03-18-Beyond Threads and Callbacks - Application Architecture Pros and Cons
15 0.65015364 728 high scalability-2009-10-26-Facebook's Memcached Multiget Hole: More machines != More Capacity
16 0.6453349 942 high scalability-2010-11-15-Strategy: Biggest Performance Impact is to Reduce the Number of HTTP Requests
17 0.63559031 699 high scalability-2009-09-10-How to handle so many socket connection
18 0.6307686 1001 high scalability-2011-03-09-Google and Netflix Strategy: Use Partial Responses to Reduce Request Sizes
19 0.62757623 951 high scalability-2010-12-01-8 Commonly Used Scalable System Design Patterns
20 0.61627609 21 high scalability-2007-07-23-GoogleTalk Architecture
topicId topicWeight
[(2, 0.476), (38, 0.188), (61, 0.163)]
simIndex simValue blogId blogTitle
same-blog 1 0.95305228 772 high scalability-2010-02-05-High Availability Principle : Concurrency Control
Introduction: One important high availability principle is concurrency control. The idea is to allow only that much traffic through to your system which your system can handle successfully. For example: if your system is certified to handle a concurrency of 100 then the 101st request should either timeout, be asked to try later or wait until one of the previous 100 requests finish. The 101st request should not be allowed to negatively impact the experience of the other 100 users. Only the 101st request should be impacted. Read more here...
2 0.93019927 80 high scalability-2007-09-06-Product: Perdition Mail Retrieval Proxy
Introduction: Perdition is a fully featured POP3 and IMAP4 proxy server. It is able to handle both SSL and non-SSL connections and redirect users to a real-server based on a database lookup. Perdition supports modular based database access. ODBC, MySQL, PostgreSQL, GDBM, POSIX Regular Expression and NIS modules ship with the distribution. The API for modules is open allowing arbitrary modules to be written to allow access to any data store. Perdition has many uses. Including, creating large mail systems where an end-user's mailbox may be stored on one of several hosts, integrating different mail systems together, migrating between different email infrastructures, and bridging plain-text, SSL and TLS services. It can also be used as part of a firewall. The use of perditon to scale mail services beyond a single box is discussed in high capacity email.
3 0.89603269 594 high scalability-2009-05-08-Eight Best Practices for Building Scalable Systems
Introduction: Wille Faler has created an excellent list of best practices for building scalable and high performance systems. Here's a short summary of his points: Offload the database - Avoid hitting the database, and avoid opening transactions or connections unless you absolutely need to use them. What a difference a cache makes - For read heavy applications caching is the easiest way offload the database. Cache as coarse-grained objects as possible - Coarse-grained objects save CPU and time by requiring fewer reads to assemble objects. Don’t store transient state permanently - Is it really necessary to store your transient data in the database? Location, Location - put things close to where they are supposed to be delivered. Constrain concurrent access to limited resource - it's quicker to let a single thread do work and finish rather than flooding finite resources with 200 client threads. Staged, asynchronous processing - separate a process using asynchronicity int
4 0.89260614 436 high scalability-2008-11-02-Strategy: How to Manage Sessions Using Memcached
Introduction: Dormando shows an enlightened middle way for storing sessions in cache and the database. Sessions are a perfect cache candidate because they are transient, smallish, and since they are usually accessed on every page access removing all that load from the database is a good thing. But as Dormando points out session caches have problems. If you remove expiration times from the cache and you run out of memory then no more logins. If a cache server fails or needs to be upgrade then you just logged out a bunch of potentially angry users. The middle ground Dormando proposes is using both the cache and the database: Reads : read from the cache first, then the database. Typical cache logic. Writes : write to memcached every time, write to the database every N seconds (assuming the data has changed). There's a small chance of data loss, but you've still greatly reduced the database load while providing reliability. Nice solution.
5 0.89045262 878 high scalability-2010-08-12-Strategy: Terminate SSL Connections in Hardware and Reduce Server Count by 40%
Introduction: This is an interesting tidbit from near the end of the Packet Pushers podcast Show 15 – Saving the Web With Dinky Putt Putt Firewalls . The conversation was about how SSL connections need to terminate before they can be processed by a WAF ( Web Application Firewall ), which inspects HTTP for security problems like SQL injection and cross-site scripting exploits. Much was made that if programmers did their job better these appliances wouldn't be necessary, but I digress. To terminate SSL most shops run SSL connections into Intel based Linux boxes running Apache. This setup is convenient for developers, but it's not optimized for SSL, so it's slow and costly. Much of the capacity of these servers are unnecessarily consumed processing SSL. Load balancers on the other hand have crypto cards that terminate SSL very efficiently in hardware. Efficiently enough that if you are willing to get rid of the general purpose Linux boxes and use your big iron load balancers, your server count c
6 0.88649338 1199 high scalability-2012-02-27-Zen and the Art of Scaling - A Koan and Epigram Approach
7 0.88632905 56 high scalability-2007-08-03-Running Hadoop MapReduce on Amazon EC2 and Amazon S3
8 0.88632905 565 high scalability-2009-04-13-Benchmark for keeping data in browser in AJAX projects
9 0.88624483 205 high scalability-2008-01-10-Letting Clients Know What's Changed: Push Me or Pull Me?
10 0.88560438 911 high scalability-2010-09-30-More Troubles with Caching
11 0.8853237 223 high scalability-2008-01-25-Google: Introduction to Distributed System Design
12 0.88344455 723 high scalability-2009-10-16-Paper: Scaling Online Social Networks without Pains
13 0.88335526 967 high scalability-2011-01-03-Stuff The Internet Says On Scalability For January 3, 2010
14 0.88235486 836 high scalability-2010-06-04-Strategy: Cache Larger Chunks - Cache Hit Rate is a Bad Indicator
15 0.88101327 1155 high scalability-2011-12-12-Netflix: Developing, Deploying, and Supporting Software According to the Way of the Cloud
16 0.88066268 1126 high scalability-2011-09-27-Use Instance Caches to Save Money: Latency == $$$
17 0.87940758 844 high scalability-2010-06-18-Paper: The Declarative Imperative: Experiences and Conjectures in Distributed Logic
18 0.87870282 1591 high scalability-2014-02-05-Little’s Law, Scalability and Fault Tolerance: The OS is your bottleneck. What you can do?
19 0.87489808 109 high scalability-2007-10-03-Save on a Load Balancer By Using Client Side Load Balancing
20 0.87448144 455 high scalability-2008-12-01-MySQL Database Scale-out and Replication for High Growth Businesses