high_scalability high_scalability-2008 high_scalability-2008-407 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Update 3: The book was released! Find it on Amazon at The Art of Capacity Planning . Update 2: Maybe the iPhone can use a little capacity planning? What's Behind the iPhone 3G Glitches : One source says Apple programmed the Infineon chip to demand a more powerful 3G signal than the iPhone really requires. So if too many people try to make a call or go on the Internet in a given area, some of the devices will decide there's insufficient power and switch to the slower network—even if there is enough 3G bandwidth available. Update: To get a taste of what will be served, mySQL DBA has a nice post titled Capacity Planning, Architecture, Scaling, Response time, Throughput . You learn how to figure out when your application will break by building a 3rd order polynomial. Cool stuff! John Allspaw who is the Operations Engineering Manager at Flickr is about to publish a book with O'Reilly. There are not much details so far but it seems interesting and relev
sentIndex sentText sentNum sentScore
1 Update 2: Maybe the iPhone can use a little capacity planning? [sent-3, score-0.201]
2 What's Behind the iPhone 3G Glitches : One source says Apple programmed the Infineon chip to demand a more powerful 3G signal than the iPhone really requires. [sent-4, score-0.428]
3 So if too many people try to make a call or go on the Internet in a given area, some of the devices will decide there's insufficient power and switch to the slower network—even if there is enough 3G bandwidth available. [sent-5, score-0.584]
4 Update: To get a taste of what will be served, mySQL DBA has a nice post titled Capacity Planning, Architecture, Scaling, Response time, Throughput . [sent-6, score-0.144]
5 You learn how to figure out when your application will break by building a 3rd order polynomial. [sent-7, score-0.164]
6 John Allspaw who is the Operations Engineering Manager at Flickr is about to publish a book with O'Reilly. [sent-9, score-0.306]
7 There are not much details so far but it seems interesting and relevant to High Scalability. [sent-10, score-0.098]
8 Allspaw combines personal anecdotes from many phases of Flickr's growth with insights from his colleagues in many other industries to give you solid guidelines for measuring your growth, predicting trends, and making cost-effective preparations. [sent-11, score-1.484]
wordName wordTfidf (topN-words)
[('iphone', 0.3), ('planning', 0.243), ('flickr', 0.206), ('capacity', 0.201), ('art', 0.198), ('book', 0.183), ('glitches', 0.17), ('insufficient', 0.17), ('atthe', 0.16), ('guidelines', 0.153), ('explosive', 0.15), ('taste', 0.144), ('phases', 0.14), ('predicting', 0.14), ('allspaw', 0.138), ('colleagues', 0.136), ('sudden', 0.132), ('growth', 0.13), ('programmed', 0.126), ('prediction', 0.125), ('exponential', 0.123), ('publish', 0.123), ('measurement', 0.122), ('signal', 0.122), ('measuring', 0.122), ('industries', 0.114), ('insights', 0.108), ('chip', 0.106), ('personal', 0.103), ('combines', 0.101), ('evaluating', 0.101), ('relevant', 0.098), ('john', 0.093), ('decide', 0.091), ('released', 0.091), ('break', 0.089), ('slower', 0.088), ('solid', 0.087), ('trends', 0.087), ('apple', 0.085), ('area', 0.083), ('switch', 0.08), ('devices', 0.08), ('figure', 0.075), ('many', 0.075), ('measure', 0.074), ('demand', 0.074), ('fit', 0.072), ('scaling', 0.072), ('maybe', 0.07)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 407 high scalability-2008-10-10-The Art of Capacity Planning: Scaling Web Resources
Introduction: Update 3: The book was released! Find it on Amazon at The Art of Capacity Planning . Update 2: Maybe the iPhone can use a little capacity planning? What's Behind the iPhone 3G Glitches : One source says Apple programmed the Infineon chip to demand a more powerful 3G signal than the iPhone really requires. So if too many people try to make a call or go on the Internet in a given area, some of the devices will decide there's insufficient power and switch to the slower network—even if there is enough 3G bandwidth available. Update: To get a taste of what will be served, mySQL DBA has a nice post titled Capacity Planning, Architecture, Scaling, Response time, Throughput . You learn how to figure out when your application will break by building a 3rd order polynomial. Cool stuff! John Allspaw who is the Operations Engineering Manager at Flickr is about to publish a book with O'Reilly. There are not much details so far but it seems interesting and relev
Introduction: Update 2: Velocity 09: John Allspaw, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr . Insightful talk. Some highlights: Change is good if you can build tools and culture to lower the risk of change. Operations and developers need to become of one mind and respect each other. An automated infrastructure is the one tool you need most. Common source control. One step build. One step deploy. Don't be a pussy, deploy. Always ship trunk. Feature flags - don't branch code, make features runtime configurable in code. Dark launch - release data paths early without UI component. Shared metrics. Adaptive feedback to prioritize important features. IRC for communication for human context. Best solutions occur when dev and op work together and trust each other. Trust is earned by helping each other solve their problems. Look at what new features imply for operations, what can go wrong, and how to recover. Provide knobs and levers to help operations. Devs should have access to production
3 0.16643566 552 high scalability-2009-04-01-Art of scalability (1) - Scalability principles
Introduction: Art of scalability - series in my blog about Scalability principles, and guild lines. To read the whole series: Art of scalability (1) - Scalability principles Art of scalability (2) - Scalability guidelines part 1 Art of scalability (3) - Scalability guidelines part 2 Art of scalability (4) - Scalability guidelines part 3
Introduction: Here's a common situation and question from the mechanical-sympathy Google group by Avinash Agrawal on the black art of capacity planning: How to get started with sizing and capacity planning, assuming we don't know the software behavior and its completely new product to deal with? Gil Tene , Vice President of Technology and CTO & Co-Founder, wrote a very understandable and useful answer that is worth highlighting: Start with requirements. I see way too many "capacity planning" exercises that go off spending weeks measuring some irrelevant metrics about a system (like how many widgets per hour can this thing do) without knowing what they actually need it to do. There are two key sets of metrics to state here: the "how much" set and the "how bad" set: In the "How Much" part, you need to establish, based on expected business needs, Numbers for things (like connections, users, streams, transactions or messages per second) that you expect to interact with at the peak t
5 0.13307442 381 high scalability-2008-09-08-Guerrilla Capacity Planning and the Law of Universal Scalability
Introduction: In the era of Web 2.0 traditional approaches to capacity planning are often difficult to implement. Guerrilla Capacity Planning facilitates rapid forecasting of capacity requirements based on the opportunistic use of whatever performance data and tools are available. One unique Guerrilla tool is Virtual Load Testing, based on Dr. Gunther's "Universal Law of Computational Scaling", which provides a highly cost-effective method for assessing application scalability. Neil Gunther, M.Sc., Ph.D. is an internationally recognized computer system performance consultant who founded Performance Dynamics Company in 1994. Some reasons why you should understand this law: 1. A lot of people use the term "scalability" without clearly defining it, let alone defining it quantitatively. Computer system scalability must be quantified. If you can't quantify it, you can't guarantee it. The universal law of computational scaling provides that quantification. 2. One the greatest impediments to applying
6 0.12065759 41 high scalability-2007-07-30-Product: Flickr
7 0.11855865 569 high scalability-2009-04-14-Scalability resources
8 0.11182433 1654 high scalability-2014-06-05-Cloud Architecture Revolution
9 0.10995157 768 high scalability-2010-02-01-What Will Kill the Cloud?
10 0.10907973 1533 high scalability-2013-10-16-Interview With Google's Ilya Grigorik On His New Book: High Performance Browser Networking
11 0.10886855 1053 high scalability-2011-06-06-Apple iCloud: Syncing and Distributed Storage Over Streaming and Centralized Storage
12 0.10349827 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
13 0.10319015 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
14 0.10167398 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
16 0.10105415 539 high scalability-2009-03-16-Books: Web 2.0 Architectures and Cloud Application Architectures
17 0.10024925 614 high scalability-2009-06-01-Guess How Many Users it Takes to Kill Your Site?
18 0.097371817 152 high scalability-2007-11-13-Flickr Architecture
19 0.096999861 348 high scalability-2008-07-09-Federation at Flickr: Doing Billions of Queries Per Day
20 0.095596045 343 high scalability-2008-06-09-Apple's iPhone to Use a Centralized Push Based Notification Architecture
topicId topicWeight
[(0, 0.152), (1, 0.033), (2, 0.029), (3, 0.031), (4, -0.018), (5, -0.074), (6, -0.05), (7, 0.003), (8, -0.004), (9, 0.02), (10, -0.051), (11, 0.004), (12, -0.029), (13, 0.052), (14, 0.024), (15, -0.016), (16, 0.033), (17, -0.007), (18, -0.011), (19, 0.03), (20, -0.012), (21, 0.013), (22, -0.072), (23, 0.045), (24, -0.037), (25, -0.029), (26, -0.02), (27, -0.003), (28, 0.04), (29, 0.062), (30, 0.026), (31, -0.076), (32, 0.095), (33, -0.028), (34, -0.004), (35, -0.005), (36, -0.044), (37, -0.021), (38, -0.052), (39, -0.013), (40, -0.061), (41, 0.004), (42, 0.015), (43, -0.082), (44, -0.034), (45, 0.065), (46, -0.055), (47, 0.015), (48, -0.097), (49, -0.064)]
simIndex simValue blogId blogTitle
same-blog 1 0.96115834 407 high scalability-2008-10-10-The Art of Capacity Planning: Scaling Web Resources
Introduction: Update 3: The book was released! Find it on Amazon at The Art of Capacity Planning . Update 2: Maybe the iPhone can use a little capacity planning? What's Behind the iPhone 3G Glitches : One source says Apple programmed the Infineon chip to demand a more powerful 3G signal than the iPhone really requires. So if too many people try to make a call or go on the Internet in a given area, some of the devices will decide there's insufficient power and switch to the slower network—even if there is enough 3G bandwidth available. Update: To get a taste of what will be served, mySQL DBA has a nice post titled Capacity Planning, Architecture, Scaling, Response time, Throughput . You learn how to figure out when your application will break by building a 3rd order polynomial. Cool stuff! John Allspaw who is the Operations Engineering Manager at Flickr is about to publish a book with O'Reilly. There are not much details so far but it seems interesting and relev
Introduction: Update 2: Velocity 09: John Allspaw, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr . Insightful talk. Some highlights: Change is good if you can build tools and culture to lower the risk of change. Operations and developers need to become of one mind and respect each other. An automated infrastructure is the one tool you need most. Common source control. One step build. One step deploy. Don't be a pussy, deploy. Always ship trunk. Feature flags - don't branch code, make features runtime configurable in code. Dark launch - release data paths early without UI component. Shared metrics. Adaptive feedback to prioritize important features. IRC for communication for human context. Best solutions occur when dev and op work together and trust each other. Trust is earned by helping each other solve their problems. Look at what new features imply for operations, what can go wrong, and how to recover. Provide knobs and levers to help operations. Devs should have access to production
3 0.64397824 348 high scalability-2008-07-09-Federation at Flickr: Doing Billions of Queries Per Day
Introduction: Flickr's lone database guy Dathan Pattishall made his excellent presentation available on how on how Flickr scales its backend to handle tremendous loads. Some of this information is available in Flickr Architecture , but the paper is so good it's worth another read. If you want to see sharding done right, at scale, take a look.
4 0.63957447 89 high scalability-2007-09-10-Is there a difference between partitioning and federation and sharding?
Introduction: Unlike Theo Schlossnagle, author of Scalable Internet Architectures , I am not a stickler for semantics because I have an unswerving faith in the ultimate unknowability of the world as experienced by others. That's why it is Theo who bravely tackles the differences in his informative blog post Partitioning vs. Federation vs. Sharding . Royans Tharakan also talks about it on his blog . Is there a difference and does it really matter to all our intrepid scalable website builders? Generally whatever Theo says is probably close to the truth. Yet, in my mind I think of partitioning as a basic level category and federation and sharding as more specific (subordinate) instances of partitioning. And partitioning is a more specific instance of the more more general (superordinate) category divide-and-conquer. Which isn't a useful way to think about the topic at all. So, let's say federation is like Star Trek. The Vulcans, Klingons, and Humans live in very separate p
5 0.6335876 1534 high scalability-2013-10-18-Stuff The Internet Says On Scalability For October 18th, 2013
Introduction: Hey, it's HighScalability time: Test your sense of scale. Is this image of something microscopic or macroscopic? Find out . $3.5 million : Per Episode Cost of Breaking Bad Quotable Quotes: @GammaCounter : "There are 400 billion trees in the Amazon River basin, close to the number of stars in the Milky Way galaxy." @rbranson : Virtualization has near-zero overhead, unless the VM spends most of it's time copying between RAM and network… like memcached or haproxy. @HackerNewsOnion : Programming is 1% inspiration, 99% trying to get your environment working. @aneel : "roundtrips, not bandwidth, is now often the bottleneck for most applications" @jamesurquhart : Not to mention the fact that auto-scaling should happen above IaaS layer. Think multi-cloud. Sheref Mansy : A machine keeps sort of chugging away, without worrying about its environment. But a living system has to. V.D. Veksler : it just came to my attention that Javascri
6 0.62366939 381 high scalability-2008-09-08-Guerrilla Capacity Planning and the Law of Universal Scalability
7 0.61873776 51 high scalability-2007-07-31-Book: Scalable Internet Architectures
8 0.61611032 1172 high scalability-2012-01-10-A Perfect Fifth of Notes on Scalability
9 0.61570662 193 high scalability-2007-12-26-Finding an excellent LAMP developer
10 0.60250646 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
11 0.60008943 1150 high scalability-2011-12-02-Stuff The Internet Says On Scalability For December 2, 2011
12 0.59797913 94 high scalability-2007-09-17-Blog: Adding Simplicity by Dan Pritchett
14 0.58977169 1654 high scalability-2014-06-05-Cloud Architecture Revolution
15 0.58934611 569 high scalability-2009-04-14-Scalability resources
16 0.58767861 185 high scalability-2007-12-13-Is premature scalation a real disease?
17 0.57606393 877 high scalability-2010-08-12-Designing Web Applications for Scalability
18 0.57216364 1613 high scalability-2014-03-17-Intuitively Showing How To Scale a Web Application Using a Coffee Shop as an Example
19 0.56757677 988 high scalability-2011-02-11-Stuff The Internet Says On Scalability For February 11, 2011
20 0.56556565 984 high scalability-2011-02-04-Stuff The Internet Says On Scalability For February 4, 2011
topicId topicWeight
[(1, 0.271), (2, 0.233), (30, 0.068), (32, 0.017), (40, 0.063), (58, 0.106), (61, 0.073), (79, 0.025), (85, 0.042)]
simIndex simValue blogId blogTitle
same-blog 1 0.97139615 407 high scalability-2008-10-10-The Art of Capacity Planning: Scaling Web Resources
Introduction: Update 3: The book was released! Find it on Amazon at The Art of Capacity Planning . Update 2: Maybe the iPhone can use a little capacity planning? What's Behind the iPhone 3G Glitches : One source says Apple programmed the Infineon chip to demand a more powerful 3G signal than the iPhone really requires. So if too many people try to make a call or go on the Internet in a given area, some of the devices will decide there's insufficient power and switch to the slower network—even if there is enough 3G bandwidth available. Update: To get a taste of what will be served, mySQL DBA has a nice post titled Capacity Planning, Architecture, Scaling, Response time, Throughput . You learn how to figure out when your application will break by building a 3rd order polynomial. Cool stuff! John Allspaw who is the Operations Engineering Manager at Flickr is about to publish a book with O'Reilly. There are not much details so far but it seems interesting and relev
2 0.92756826 983 high scalability-2011-02-02-Piccolo - Building Distributed Programs that are 11x Faster than Hadoop
Introduction: Piccolo (not this or this ) is a system for distributed computing, Piccolo is a n ew data-centric programming model for writing parallel in-memory applications in data centers . Unlike existing data-flow models, Piccolo allows computation running on different machines to share distributed, mutable state via a key-value table interface. T raditional data-centric models (such as Hadoop) which present the user a single object at a time to operate on, Piccolo exposes a global table interface which is available to all parts of the computation simultaneously. This allows users to specify programs in an intuitive manner very similar to that of writing programs for a single machine. Using an in-memory key-value store is a very different approach from the canonical map-reduce, which is based on using distributed file systems. The results are impressive: Experiments have shown that Piccolo is fast and pro-vides excellent scaling for many applications. The performance of PageRank and
3 0.92681235 1517 high scalability-2013-09-16-The Hidden DNS Tax - Cascading Timeouts and Errors
Introduction: This is a guest post by Nick Burling , VP of Product Management of Bluestripe . Readers of High Scalability know are well versed in performance optimization techniques. Reverse proxies, Varnish, Redis — you hear about them daily. But what you may not realize is that one of the oldest technologies in your stack can be one of your biggest bottlenecks: DNS. People don't spend a lot of time thinking about DNS. It's not sexy. It's an infrastructure service, and it's just supposed to work. At BlueStripe, we work with many teams running applications that support millions of web requests a day. We keep seeing DNS delays and errors that the platform operations team never knows about. It's so common we've start calling it the Hidden DNS Tax . What is the Hidden DNS Tax? The Hidden DNS Tax is a hard-to-see performance hit your users take from DNS timeouts and errors in your back-end architecture. We've seen it bring down the main web application for a Fortune 10 company.
4 0.92641062 1082 high scalability-2011-07-18-New Relic Architecture - Collecting 20+ Billion Metrics a Day
Introduction: This is a guest post by Brian Doll , Application Performance Engineer at New Relic. New Relic’s multitenant, SaaS web application monitoring service collects and persists over 100,000 metrics every second on a sustained basis, while still delivering an average page load time of 1.5 seconds. We believe that good architecture and good tools can help you handle an extremely large amount of data while still providing extremely fast service. Here we'll show you how we do it. New Relic is Application Performance Management (APM) as a Service In-app agent instrumentation (bytecode instrumentation, etc.) Support for 5 programming languages (Ruby, Java, PHP, .NET, Python) 175,000+ app processes monitored globally 10,000+ customers The Stats 20+ Billion application metrics collected every day 1.7+ Billion web page metrics collected every week Each "timeslice" metric is about 250 bytes 100k timeslice records inserted every second 7 Billion new rows of d
5 0.92583317 788 high scalability-2010-03-04-How MySpace Tested Their Live Site with 1 Million Concurrent Users
Introduction: This is a guest post by Dan Bartow, VP of SOASTA , talking about how they pelted MySpace with 1 million concurrent users using 800 EC2 instances. I thought this was an interesting story because: that's a lot of users, it takes big cajones to test your live site like that, and not everything worked out quite as expected. I'd like to thank Dan for taking the time to write and share this article. In December of 2009 MySpace launched a new wave of streaming music video offerings in New Zealand, building on the previous success of MySpace music. These new features included the ability to watch music videos, search for artist’s videos, create lists of favorites, and more. The anticipated load increase from a feature like this on a popular site like MySpace is huge, and they wanted to test these features before making them live. If you manage the infrastructure that sits behind a high traffic application you don’t want any surprises. You want to understand your breakin
6 0.92384535 924 high scalability-2010-10-21-What is Network-based Application Virtualization and Why Do You Need It?
7 0.92167234 1258 high scalability-2012-06-05-Thesis: Concurrent Programming for Scalable Web Architectures
8 0.92033386 310 high scalability-2008-04-29-High performance file server
9 0.92027181 373 high scalability-2008-08-29-Product: ScaleOut StateServer is Memcached on Steroids
10 0.91819727 292 high scalability-2008-03-30-Scaling Out MySQL
11 0.91779739 727 high scalability-2009-10-25-Is Your Data Really Secured?
12 0.91721696 887 high scalability-2010-08-24-Sponsored Post: deviantART, Okta, EzRez, Cloud Sigma, ManageEngine, Site24x7
13 0.91671193 382 high scalability-2008-09-09-Content Delivery Networks (CDN) – a comprehensive list of providers
14 0.91500688 633 high scalability-2009-06-19-GemFire 6.0: New innovations in data management
15 0.91408861 913 high scalability-2010-10-01-Hot Scalability Links For Oct 1, 2010
16 0.91408402 876 high scalability-2010-08-10-Sponsored Post: Okta, EzRez, VoltDB, Digg, Cloud Sigma, Applications Manager, Site24x7
17 0.91390496 866 high scalability-2010-07-27-Sponsored Post: Okta, EzRez, VoltDB, Digg, Cloud Sigma, Applications Manager, Site24x7
18 0.91174704 1309 high scalability-2012-08-22-Cloud Deployment: It’s All About Cloud Automation
19 0.91159183 701 high scalability-2009-09-10-When optimizing - don't forget the Java Virtual Machine (JVM)
20 0.91155887 1633 high scalability-2014-04-16-Six Lessons Learned the Hard Way About Scaling a Million User System