high_scalability high_scalability-2008 high_scalability-2008-398 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system: * The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency. * Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool. * Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability. * Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter. * Forgetting to che
sentIndex sentText sentNum sentScore
1 Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system: * The Golden Hammer. [sent-1, score-0.569]
2 Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. [sent-2, score-0.183]
3 Using a database to store key-value pairs is one example. [sent-3, score-0.112]
4 Another example is using threads to program for concurrency. [sent-4, score-0.151]
5 Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. [sent-6, score-0.25]
6 For example, connection management to the database through a thread pool. [sent-7, score-0.162]
7 In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. [sent-11, score-0.868]
8 Failure to enforce diligent control is a contributing scalability inhibiter. [sent-12, score-0.459]
9 To properly scale a system it is imperative to manage the time alloted for requests to be handled. [sent-14, score-0.532]
10 One popular solution to the operation issue is a Hero who can and often will manage the bulk of the operational needs. [sent-16, score-0.56]
11 For a large system of many components this approach does not scale, yet it is one of the most frequently-deployed solutions. [sent-17, score-0.076]
12 A system too dependent on human intervention, frequently the result of having a Hero, is dangerously exposed to issues of reproducibility and hit-by-a-bus syndrome. [sent-19, score-0.631]
13 Monitoring, like testing, is often one of the first items sacrificed when time is tight. [sent-21, score-0.329]
wordName wordTfidf (topN-words)
[('hero', 0.435), ('manage', 0.218), ('worst', 0.206), ('diligent', 0.187), ('formulating', 0.187), ('reproducibility', 0.176), ('failure', 0.174), ('sacrificed', 0.161), ('forgetting', 0.152), ('intervention', 0.152), ('forcing', 0.148), ('contributing', 0.137), ('imperative', 0.137), ('enforce', 0.135), ('golden', 0.129), ('ball', 0.126), ('facilitate', 0.123), ('brian', 0.123), ('tight', 0.121), ('agility', 0.119), ('exposed', 0.112), ('pairs', 0.112), ('dependency', 0.111), ('dependencies', 0.11), ('intended', 0.106), ('definition', 0.105), ('dependent', 0.105), ('highlights', 0.104), ('travel', 0.104), ('properly', 0.101), ('bulk', 0.095), ('often', 0.092), ('relationships', 0.092), ('frequently', 0.088), ('management', 0.086), ('practices', 0.08), ('example', 0.079), ('operational', 0.078), ('sometimes', 0.077), ('practice', 0.077), ('issue', 0.077), ('items', 0.076), ('system', 0.076), ('connection', 0.076), ('experienced', 0.076), ('architect', 0.075), ('human', 0.074), ('threads', 0.072), ('understanding', 0.072), ('fail', 0.072)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 398 high scalability-2008-09-30-Scalability Worst Practices
Introduction: Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system: * The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency. * Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool. * Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability. * Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter. * Forgetting to che
2 0.15476631 1026 high scalability-2011-04-18-6 Ways Not to Scale that Will Make You Hip, Popular and Loved By VCs
Introduction: This is a hilarious presentation by Josh Berkus , called Scale Fail , given at O'Reilly MySQL CE 2011. Josh is entertaining, well spoken, and cleverly hides insight inside chaos. And he makes some dang good points along the way. Josh has a problem, you see Josh has learned how to make sites that are both scalable and reliable. So he's puzzled why companies "whose downtime interfaces (Twitter) are more well known than their uptime interfaces" get all the attention, respect, and money for being failures. Just doing your job doesn't make you a hero. You need these self-inflicted wounds in-order to have the war stories to share at conferences. They get the attention. Just doing your job is boring. This is so unfair in that way life can be. So if you want to turn the tables and take the low road to fame and fortune, here's Josh's program for learning how not to scale: Be trendy . Use the tool that has the most buzz: NoSQL, Cloud, MapReduce, Rails, RabbitMQ. It helps you no
3 0.11090158 442 high scalability-2008-11-13-Plenty of Fish Says Scaling for Free Doesn't Pay
Introduction: Plenty of Fish CEO Markus Frind, famous nerd hero for making over $10 million a year from Google ads on a free dating site he made and ran all by himself, now sees a problem with the free model : The problem with free is that every time you double the size of your database the cost of maintaining the site grows 6 fold. I really underestimated how much resources it would take, I have one database table now that exceeds 3 billion records. The bigger you get as a free site the less money you make per visit and the more it costs to service a visit...There is really no money in being free and we have to start experimenting with other models now or we won’t be able to compete in 3 or 4 years. As one commenter succinctly put it: the “golden time” of AdSense is over . Time to look at costs. The POF architecture is to run scarily huge tables on single machines. They also buy and maintain their own SAN. So it seems scaling up is what is increasing costs and decreasing profits. I wo
4 0.10660512 689 high scalability-2009-08-28-Strategy: Solve Only 80 Percent of the Problem
Introduction: Solve only 80% of a problem. That's usually good enough and you'll not only get done faster, you'll actually have a chance of getting done at all. This strategy is given by Amix in HOW TWITTER (AND FACEBOOK) SOLVE PROBLEMS PARTIALLY . The idea is solving 100% of a complex problem can be so hard and so expensive that you'll end up wasting all your bullets on a problem that could have been satisfactoraly solved in a much simpler way. The example given is for Twitter's real-time search. Real-time search almost by definition is focussed on recent events. So in the design should you be able to search historically back from the beginning of time or should you just be able to search for recent time periods? A complete historical search is the 100% solution. The recent data only search is the 80% solution. Which should you choose? The 100% solution is dramatically more difficult to solve. It requires searching disk in real-time which is a killer. So it makes more sense to work on the
5 0.10026871 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
Introduction: For everything given something seems to be taken. Caching is a great scalability solution, but caching also comes with problems . Sharding is a great scalability solution, but as Foursquare recently revealed in a post-mortem about their 17 hours of downtime, sharding also has problems. MongoDB, the database Foursquare uses, also contributed their post-mortem of what went wrong too. Now that everyone has shared and resharded, what can we learn to help us skip these mistakes and quickly move on to a different set of mistakes? First, like for Facebook , huge props to Foursquare and MongoDB for being upfront and honest about their problems. This helps everyone get better and is a sign we work in a pretty cool industry. Second, overall, the fault didn't flow from evil hearts or gross negligence. As usual the cause was more mundane: a key system, that could be a little more robust, combined with a very popular application built by a small group of people, under immense pressure
7 0.087753795 1425 high scalability-2013-03-18-Beyond Threads and Callbacks - Application Architecture Pros and Cons
8 0.082069598 1591 high scalability-2014-02-05-Little’s Law, Scalability and Fault Tolerance: The OS is your bottleneck. What you can do?
9 0.079850033 474 high scalability-2008-12-21-The I.H.S.D.F. Theorem: A Proposed Theorem for the Trade-offs in Horizontally Scalable Systems
10 0.078430742 96 high scalability-2007-09-18-Amazon Architecture
11 0.07358066 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
12 0.073287308 1654 high scalability-2014-06-05-Cloud Architecture Revolution
13 0.072099864 659 high scalability-2009-07-20-A Scalability Lament
14 0.069993198 1413 high scalability-2013-02-27-42 Monster Problems that Attack as Loads Increase
15 0.069379359 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
16 0.068145946 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
17 0.067225046 910 high scalability-2010-09-30-Facebook and Site Failures Caused by Complex, Weakly Interacting, Layered Systems
18 0.0669339 1421 high scalability-2013-03-11-Low Level Scalability Solutions - The Conditioning Collection
19 0.066059291 1429 high scalability-2013-03-25-AppBackplane - A Framework for Supporting Multiple Application Architectures
20 0.065437734 317 high scalability-2008-05-10-Hitting 300 SimbleDB Requests Per Second on a Small EC2 Instance
topicId topicWeight
[(0, 0.126), (1, 0.044), (2, 0.006), (3, 0.004), (4, 0.01), (5, 0.005), (6, 0.029), (7, -0.012), (8, -0.057), (9, -0.048), (10, -0.024), (11, 0.046), (12, -0.021), (13, -0.016), (14, 0.018), (15, -0.047), (16, 0.038), (17, -0.009), (18, -0.007), (19, 0.035), (20, 0.016), (21, -0.012), (22, 0.012), (23, -0.007), (24, -0.025), (25, -0.003), (26, -0.009), (27, 0.047), (28, 0.033), (29, 0.036), (30, -0.006), (31, 0.035), (32, 0.04), (33, -0.011), (34, -0.013), (35, 0.013), (36, -0.012), (37, 0.028), (38, 0.021), (39, 0.036), (40, 0.007), (41, -0.021), (42, -0.029), (43, 0.02), (44, -0.013), (45, 0.001), (46, -0.01), (47, -0.001), (48, 0.001), (49, 0.025)]
simIndex simValue blogId blogTitle
same-blog 1 0.96524543 398 high scalability-2008-09-30-Scalability Worst Practices
Introduction: Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system: * The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency. * Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool. * Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability. * Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter. * Forgetting to che
2 0.76600903 1418 high scalability-2013-03-06-Low Level Scalability Solutions - The Aggregation Collection
Introduction: What good are problems without solutions? In 42 Monster Problems That Attack As Loads Increase we talked about problems. In this first post (OK, there was an earlier post, but I'm doing some reorganizing), we'll cover what I call aggregation strategies. Keep in mind these are low level architecture type suggestions of how to structure the components of your code and how they interact. We're not talking about massive scale-out clusters here, but of what your applications might like like internally, way below the service level interface level. There's a lot more to the world than evented architectures. Aggregation simply means we aren't using stupid queues. Our queues will be smart. We are deeply aware of queues as containers of work that eventually dictate how the entire system performs. As work containers we know intimately what requests and data sit in our queues and we can use that intelligence to our great advantage. Prioritize Work The key idea to it all is an almost mi
3 0.75840068 1454 high scalability-2013-05-08-Typesafe Interview: Scala + Akka is an IaaS for Your Process Architecture
Introduction: This is an email interview with Viktor Klang , Director of Engineering at Typesafe , on the Scala Futures model & Akka, both topics on which is he is immensely passionate and knowledgeable. How do you structure your application? That’s the question I explored in the article Beyond Threads And Callbacks . An option I did not talk about, mostly because of my own ignorance, is a powerful stack you may not be all that familiar with: Scala and Akka. To remedy my oversight is our acting tour guide, Typesafe’s Viktor Klang, long time Scala hacker and Java enterprise systems architect. Viktor was very patient in answering my questions and was enthusiastic about sharing his knowledge. He’s a guy who definitely knows what he is talking about. I’ve implemented several Actor systems along with the messaging infrastructure, threading, async IO, service orchestration, failover, etc, so I’m innately skeptical about frameworks that remove control from the programmer at
4 0.73778605 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
Introduction: This is a guest posting by Marty Abbott and Michael Fisher, authors of The Art of Scalability . I'm still reading their book and will have an interview with them a little later. If 2010 is the year that you’ve decided to kickoff your startup or if you’ve already got something off the ground and are expecting double or triple digit growth, this list is for you. We all want the attention of user s to achieve viral growth but as many can attest , too much attention can bring a startup to its knees. If you’ve used Twitter for any amount of time you’re sure to have seen the “Fail Whale”, which is so often seen that it has its own fan club . Take a look at the graph below from Compete.com showing Twitter’s unique visitors. One can argue that limitations in the product offering have as much to do with the flattening of growth over the past six months as does the availability , but it’s hard to beli
Introduction: With a new Planet of the Apes coming out, this may be a touchy subject with our new overlords, but Netflix is using a whole lot more trouble injecting monkeys to test and iteratively harden their systems. We learned previously how Netflix used Chaos Monkey , a tool to test failover handling by continuously failing EC2 nodes. That was just a start. More monkeys have been added to the barrel. Node failure is just one problem in a system. Imagine a problem and you can imagine creating a monkey to test if your system is handling that problem properly. Yury Izrailevsky talks about just this approach in this very interesting post: The Netflix Simian Army . I know what you are thinking, if monkeys are so great then why has Netflix been down lately. Dmuino addressed this potential embarrassment, putting all fears of cloud inferiority to rest: Unfortunately we're not running 100% on the cloud today. We're working on it, and we could use more help. The latest outage was caused by a com
7 0.72797984 1425 high scalability-2013-03-18-Beyond Threads and Callbacks - Application Architecture Pros and Cons
8 0.72677469 1553 high scalability-2013-11-25-How To Make an Infinitely Scalable Relational Database Management System (RDBMS)
9 0.72336853 1209 high scalability-2012-03-14-The Azure Outage: Time Is a SPOF, Leap Day Doubly So
10 0.71874475 1204 high scalability-2012-03-06-Ask For Forgiveness Programming - Or How We'll Program 1000 Cores
11 0.71808088 679 high scalability-2009-08-11-13 Scalability Best Practices
12 0.71775621 910 high scalability-2010-09-30-Facebook and Site Failures Caused by Complex, Weakly Interacting, Layered Systems
13 0.7165485 96 high scalability-2007-09-18-Amazon Architecture
15 0.71059752 1415 high scalability-2013-03-04-7 Life Saving Scalability Defenses Against Load Monster Attacks
16 0.71041113 1413 high scalability-2013-02-27-42 Monster Problems that Attack as Loads Increase
17 0.70906466 1276 high scalability-2012-07-04-Top Features of a Scalable Database
18 0.70761281 1421 high scalability-2013-03-11-Low Level Scalability Solutions - The Conditioning Collection
19 0.70673585 1628 high scalability-2014-04-08-Microservices - Not a free lunch!
20 0.70082194 189 high scalability-2007-12-21-Strategy: Limit Result Sets
topicId topicWeight
[(1, 0.097), (2, 0.269), (10, 0.129), (47, 0.035), (61, 0.058), (79, 0.084), (89, 0.225)]
simIndex simValue blogId blogTitle
1 0.94147074 1503 high scalability-2013-08-19-What can the Amazing Race to the South Pole Teach us About Startups?
Introduction: At the heart of every software adventure exists a journey in service of a quest. Melodramatic much? Sorry, but while wandering dazzled through Race to the End of the Earth , a fantastic exhibit at the Royal BC Museum on the 1911-1912 race to the South Pole between Norwegian explorer Roald Amundsen and British naval officer Robert Scott , I couldn’t help but think of the two radically different approaches each team took to the race and it shocked me to see that some of the same principles that lead to success or failure in software development also seem to lead to success or failure in exploration. I wish I could reproduce the experience of walking through the exhibit . Plaque after plaque I remember wondering out loud at Scott’s choices and then nod in agreement with Amundsen’s approach. The core conflict was straight out of any ancient Agile (Amundsen) vs Waterfall (Scott) thread you can find on Usenet. And Waterfall lost. As background here are some sources you may want
same-blog 2 0.8965385 398 high scalability-2008-09-30-Scalability Worst Practices
Introduction: Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system: * The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency. * Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool. * Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability. * Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter. * Forgetting to che
3 0.87993443 32 high scalability-2007-07-26-Product: eAccelerator a PHP Accelerator
Introduction: eAccelerator is a free open-source PHP accelerator, optimizer, and dynamic content cache. It increases the performance of PHP scripts by caching them in their compiled state, so that the overhead of compiling is almost completely eliminated. It also optimizes scripts to speed up their execution. eAccelerator typically reduces server load and increases the speed of your PHP code by 1-10 times.
4 0.82124573 1210 high scalability-2012-03-16-Stuff The Internet Says On Scalability For March 16, 2012
Introduction: HighScalability is What We Do: 454,400 : Number of Amazon servers; 45PB : Facebook Data Warehouse, grows exponentially; 5 Atoms : Ultimate limit of thermodynamics; YouTube : 4 billion views/day, 60 hours of video uploaded every minute, revenue doubled in 2010 Quotable quotes: @adrianco : Walmart labs run large single region Cassandra clusters with Intel SSDs and have been in production for two years. Working well for them. @mybellemac : Scalability is a mother. #pinterest @fakesigi : Thanks for the correction. I saw cloud computing, scalability and my brain turned off. @BVA100 : I disagree with "If it ain't broke, don't fix it". We ought to be forward thinkers, concerned with leading indicators and scalability. Dilbert on the meaning of it all . Cassandra and Solid State Drives . DataStax's Rick Branson with a sweet explanation of how Cassandra was built for a world of spinning disks, which means it only writes sequentially, w
5 0.81637144 498 high scalability-2009-01-20-Product: Amazon's SimpleDB
Introduction: Update 35 : How and Why Glue is Using Amazon SimpleDB instead of a Relational Database . Discusses a key design decision that required duplicating data in order to mimic RDBMS joins: Given the trade off between potential inconsistencies and scalability, social services have to choose the latter. Update 34 : Apparently Amazon pulled this article. I'm not sure what that means. Maybe time went backwards or something? Amazon dramatically drops SimpleDB pricing to $0.25 per GB per month from $1.50 per GB . This puts SimpleDB on par with Google App Engine . They also announced a few new features: a SQL-like SELECT API as well as a Batch Put operation to streamline uploading of multiple items or attributes . One of the complaints against SimpleDB is that programmers end up writing too much code to do simple things. These features and a much cheaper price should help considerably. And you can store lots of data now. GAE is still capped. Update 33 : Amazon announces
6 0.81374514 1187 high scalability-2012-02-03-Stuff The Internet Says On Scalability For February 3, 2012
7 0.81336063 1425 high scalability-2013-03-18-Beyond Threads and Callbacks - Application Architecture Pros and Cons
8 0.80604589 240 high scalability-2008-02-05-Handling of Session for a site running from more than 1 data center
9 0.80476362 157 high scalability-2007-11-16-Product: lbpool - Load Balancing JDBC Pool
10 0.80312115 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
11 0.80202281 1429 high scalability-2013-03-25-AppBackplane - A Framework for Supporting Multiple Application Architectures
12 0.800071 1251 high scalability-2012-05-24-Build your own twitter like real time analytics - a step by step guide
13 0.79959184 1291 high scalability-2012-07-25-Vertical Scaling Ascendant - How are SSDs Changing Architectures?
14 0.79927856 1369 high scalability-2012-12-10-Switch your databases to Flash storage. Now. Or you're doing it wrong.
16 0.79610592 1197 high scalability-2012-02-21-Pixable Architecture - Crawling, Analyzing, and Ranking 20 Million Photos a Day
17 0.79585338 160 high scalability-2007-11-19-Tailrank Architecture - Learn How to Track Memes Across the Entire Blogosphere
18 0.79584676 1207 high scalability-2012-03-12-Google: Taming the Long Latency Tail - When More Machines Equals Worse Results
19 0.7956022 619 high scalability-2009-06-05-HotPads Shows the True Cost of Hosting on Amazon
20 0.79545385 925 high scalability-2010-10-22-Paper: Netflix’s Transition to High-Availability Storage Systems