high_scalability high_scalability-2010 high_scalability-2010-757 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is a guest posting by Marty Abbott and Michael Fisher, authors of The Art of Scalability . I'm still reading their book and will have an interview with them a little later. If 2010 is the year that you’ve decided to kickoff your startup or if you’ve already got something off the ground and are expecting double or triple digit growth, this list is for you. We all want the attention of user s to achieve viral growth but as many can attest , too much attention can bring a startup to its knees. If you’ve used Twitter for any amount of time you’re sure to have seen the “Fail Whale”, which is so often seen that it has its own fan club . Take a look at the graph below from Compete.com showing Twitter’s unique visitors. One can argue that limitations in the product offering have as much to do with the flattening of growth over the past six months as does the availability , but it’s hard to beli
sentIndex sentText sentNum sentScore
1 If 2010 is the year that you’ve decided to kickoff your startup or if you’ve already got something off the ground and are expecting double or triple digit growth, this list is for you. [sent-3, score-0.292]
2 One can argue that limitations in the product offering have as much to do with the flattening of growth over the past six months as does the availability , but it’s hard to believe the inability of users to actually use the service has not hindered growth. [sent-8, score-0.361]
3 What should you do if you want your startup to scale with double and triple digit growth? [sent-9, score-0.398]
4 It also means that you can more easily and quickly respond to customer demands by buying one more box rather than having to spec and buy a bigger box. [sent-16, score-0.277]
5 It’s critically important where in the database tier are concerned as that’s where most architectures tend to converge and where most problems with scale tend to occur. [sent-17, score-0.314]
6 Don’t rely heavily on stored procedures, especially those with business logic embedded within them, as it will keep you from quickly and easily moving to competing database products for reasons of cost or availability. [sent-19, score-0.495]
7 3) Soar Through the Clouds If you need high availability 24x7, relying solely upon a cloud for your services will likely let you down for the near future. [sent-23, score-0.282]
8 We are huge believers in having the beauty of your product nested within the blueprint of your architecture and your design – not in your hardware . [sent-29, score-0.32]
9 We love to buy cheap hardware that’s easily disposed of or replaced rather than painstakingly caring for and maintaining expensive hardware with designer labels. [sent-30, score-0.332]
10 Design your product for goldfish (commodity hardware ) and you will nearly always lower your total cost of ownership and increase your ability to leverage clouds and rapid horizontal scale. [sent-31, score-0.516]
11 By relying on a third party, you are forcing yourself to upgrade when they say to do so, migrate product lines when they say and potentially lose a product if they choose to move away from that product line or go out of business. [sent-41, score-0.433]
12 It isn’t that difficult in our experience to build horizontal scale into your architecture, even within the database tier. [sent-43, score-0.324]
13 While there are competing theories on this particular matter , most of us have plenty of opportunity to learn from past mistakes in both our engineering and operations environments. [sent-46, score-0.318]
14 8) Communicate Asynchronously As Much As Possible Synchronicity may have been a great Police album, but it kills within high transaction product platforms. [sent-51, score-0.313]
15 Moreover, slow server responses are likely to have downstream effects upon other services calling the slow servers. [sent-53, score-0.486]
16 It is comparatively cheap to (D)esign (architect) a system to scale, increasingly more expensive to (I)mplement that architecture within the actual software and fairly expensive to actually buy and (D)eploy all the systems within a production environment. [sent-67, score-0.495]
17 Our D-I-D approach to scale helps you to quickly react to scale needs by always designing well ahead of need, coding or implementing slightly ahead of need and deploying the capital assets “just in time” for your growth. [sent-68, score-0.315]
18 ” Build not only fault tolerance, but fault isolation into your architectures. [sent-70, score-0.366]
19 Network architectures have long had the notion of fault isolation through collision domains . [sent-71, score-0.323]
20 We refer to these fault isolation zones as “swim lanes. [sent-73, score-0.421]
wordName wordTfidf (topN-words)
[('goldfish', 0.228), ('isolation', 0.14), ('upon', 0.13), ('swim', 0.13), ('within', 0.121), ('product', 0.117), ('digit', 0.115), ('fault', 0.113), ('scale', 0.106), ('slow', 0.106), ('quickly', 0.103), ('triple', 0.099), ('easily', 0.099), ('rely', 0.098), ('phases', 0.098), ('horizontal', 0.097), ('refer', 0.093), ('growth', 0.091), ('moreover', 0.091), ('surprise', 0.09), ('expensive', 0.089), ('aggressively', 0.088), ('opportunity', 0.088), ('simplify', 0.084), ('portion', 0.083), ('design', 0.082), ('relying', 0.082), ('levels', 0.082), ('learn', 0.08), ('fail', 0.08), ('startup', 0.078), ('believe', 0.077), ('past', 0.076), ('great', 0.075), ('zones', 0.075), ('buy', 0.075), ('cost', 0.074), ('mistakes', 0.074), ('responses', 0.074), ('architectures', 0.07), ('likely', 0.07), ('art', 0.069), ('tend', 0.069), ('workable', 0.069), ('marty', 0.069), ('overreliance', 0.069), ('fisher', 0.069), ('painstakingly', 0.069), ('thatwill', 0.069), ('underperforming', 0.069)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999976 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
Introduction: This is a guest posting by Marty Abbott and Michael Fisher, authors of The Art of Scalability . I'm still reading their book and will have an interview with them a little later. If 2010 is the year that you’ve decided to kickoff your startup or if you’ve already got something off the ground and are expecting double or triple digit growth, this list is for you. We all want the attention of user s to achieve viral growth but as many can attest , too much attention can bring a startup to its knees. If you’ve used Twitter for any amount of time you’re sure to have seen the “Fail Whale”, which is so often seen that it has its own fan club . Take a look at the graph below from Compete.com showing Twitter’s unique visitors. One can argue that limitations in the product offering have as much to do with the flattening of growth over the past six months as does the availability , but it’s hard to beli
2 0.17817555 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
Introduction: Who's Hiring? FiftyThree , the company behind the award-winning iPad app Paper , is looking for a {Backend || DevOps} Engineer to help us build our next great product: a service to "bring ideas together". http://www.fiftythree.com/jobs New Relic is looking for a Java Scalability Engineer in Portland, OR. Ready to scale a web service with more incoming bits/second than Twitter? http://newrelic.com/about/jobs Palantir Technologies is hiring a Software Engineer to develop our technologies for exploring huge information sets and presenting complex information to end-users. Join the team at ROBLOX as a Senior Database Administrato r and help us advance our rapidly growing gaming platform with over 30K web hits/sec, 75K+ database requests/sec, and over 1 petabyte of monthly CDN traffic. Sound cool? Apply here . Fun and Informative Events Surge - A Scalability & Performance Conference, presented by OmniT is happening on Sept. 27th - 28th. Special,
4 0.15260604 96 high scalability-2007-09-18-Amazon Architecture
Introduction: This is a wonderfully informative Amazon update based on Joachim Rohde's discovery of an interview with Amazon's CTO. You'll learn about how Amazon organizes their teams around services, the CAP theorem of building scalable systems, how they deploy software, and a lot more. Many new additions from the ACM Queue article have also been included. Amazon grew from a tiny online bookstore to one of the largest stores on earth. They did it while pioneering new and interesting ways to rate, review, and recommend products. Greg Linden shared is version of Amazon's birth pangs in a series of blog articles Site: http://amazon.com Information Sources Early Amazon by Greg Linden How Linux saved Amazon millions Interview Werner Vogels - Amazon's CTO Asynchronous Architectures - a nice summary of Werner Vogels' talk by Chris Loosley Learning from the Amazon technology platform - A Conversation with Werner Vogels Werner Vogels' Weblog - building scalable and robus
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
7 0.14992714 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
9 0.14590502 679 high scalability-2009-08-11-13 Scalability Best Practices
10 0.1452089 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
15 0.13859661 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
topicId topicWeight
[(0, 0.297), (1, 0.069), (2, 0.011), (3, 0.029), (4, 0.013), (5, -0.044), (6, -0.036), (7, -0.027), (8, -0.046), (9, -0.039), (10, -0.012), (11, 0.021), (12, -0.072), (13, 0.03), (14, 0.026), (15, 0.006), (16, 0.058), (17, -0.008), (18, 0.009), (19, 0.046), (20, -0.006), (21, 0.009), (22, -0.002), (23, -0.027), (24, -0.051), (25, -0.022), (26, -0.018), (27, -0.003), (28, 0.042), (29, 0.058), (30, 0.04), (31, 0.021), (32, 0.026), (33, 0.004), (34, -0.044), (35, 0.031), (36, -0.056), (37, 0.031), (38, -0.008), (39, 0.014), (40, -0.006), (41, -0.017), (42, -0.024), (43, -0.058), (44, -0.005), (45, -0.003), (46, -0.001), (47, 0.004), (48, -0.044), (49, -0.061)]
simIndex simValue blogId blogTitle
same-blog 1 0.98111963 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
Introduction: This is a guest posting by Marty Abbott and Michael Fisher, authors of The Art of Scalability . I'm still reading their book and will have an interview with them a little later. If 2010 is the year that you’ve decided to kickoff your startup or if you’ve already got something off the ground and are expecting double or triple digit growth, this list is for you. We all want the attention of user s to achieve viral growth but as many can attest , too much attention can bring a startup to its knees. If you’ve used Twitter for any amount of time you’re sure to have seen the “Fail Whale”, which is so often seen that it has its own fan club . Take a look at the graph below from Compete.com showing Twitter’s unique visitors. One can argue that limitations in the product offering have as much to do with the flattening of growth over the past six months as does the availability , but it’s hard to beli
2 0.8292616 185 high scalability-2007-12-13-Is premature scalation a real disease?
Introduction: Update 3: InfoQ's Big Architecture Up Front - A Case of Premature Scalaculation? twines several different threads on the topic together into a fine noose. Update 2: Kevin says the biggest problems he sees with startups is they need to scale their backend (no, the other one). Update: My bad. It's hard to sell scalability so just forget it. The premise of Startups and The Problem Of Premature Scalaculation and Don’t scale: 99.999% uptime is for Wal-Mart is that you shouldn't spend precious limited resources worrying about scaling before you've first implemented the functionality that will make you successful enough to have scaling problems in the first place. It's kind of an embodied life force model of system creation. Energy is scarce so any parasites siphoning off energy must be hunted down and destroyed so the body has its best chance of survival. Is this really how it works? If I ever believed this I certainly don't believe it anymore. The world has c
Introduction: Update 2: Velocity 09: John Allspaw, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr . Insightful talk. Some highlights: Change is good if you can build tools and culture to lower the risk of change. Operations and developers need to become of one mind and respect each other. An automated infrastructure is the one tool you need most. Common source control. One step build. One step deploy. Don't be a pussy, deploy. Always ship trunk. Feature flags - don't branch code, make features runtime configurable in code. Dark launch - release data paths early without UI component. Shared metrics. Adaptive feedback to prioritize important features. IRC for communication for human context. Best solutions occur when dev and op work together and trust each other. Trust is earned by helping each other solve their problems. Look at what new features imply for operations, what can go wrong, and how to recover. Provide knobs and levers to help operations. Devs should have access to production
4 0.82639283 1172 high scalability-2012-01-10-A Perfect Fifth of Notes on Scalability
Introduction: Jeremiah Peschka with a great a set of Notes on Scalability , just in case you do reach your wildest expectations of success: Build it to Break . Plan for the fact that everything you make is going to break. Design in layers that are independent and redundant. Everything is a Feature . Your application is a set of features created by a series of conscious choices made by considering trade-offs. Scale Out, Not Up . Purchasing more hardware is easier than coding and managing horizontal resources. Buy More Storage . Large numbers of smaller, faster drives have more IOPS than fewer, larger drives. You’re Going to Do It Wrong . Be prepared to iterate on your ideas. You will make mistakes. Be prepared to re-write code and to quickly move on to the next idea. Please read the original article for a much more expansive treatment.
5 0.81983519 1431 high scalability-2013-03-29-Stuff The Internet Says On Scalability For March 29, 2013
Introduction: Hey, it's HighScalability time: ( Ukrainian daredevil scaling buildings) 44.6 billion - Tumblr posts; 300 Gb/s - DDoS DNS amplification attacks; 100 million - Eventbrite tickets processed. Quotable Quotes: @tveskov : Alan Kay: “The past 30 years have been completely mundane. It’s all been scaling (of old technology) and Angry Birds” @phrawzty : OH "Complexity is accelerating. We must augment our ability to manage it." #monitorama @stallent : new wave of apps that are bringing the server doing actual legit valuable work back into vogue know scaling is more than hard @calvdee : Fending off a 300Gb/s DDoS attacks would constitute a feat of #highscalability @solarce : "A spider farted in Finland and screwed up my IOPS!" -- @lusis #monitorama @cra : Listening to Shawn Pearce talk about scaling #git at Google with JGit... Android AOSP repos: 19.4GB, 2.5Mreq/day, 5.0TB/day #e
7 0.81090844 1420 high scalability-2013-03-08-Stuff The Internet Says On Scalability For March 8, 2013
8 0.80487335 96 high scalability-2007-09-18-Amazon Architecture
9 0.80338645 1199 high scalability-2012-02-27-Zen and the Art of Scaling - A Koan and Epigram Approach
10 0.80201995 1411 high scalability-2013-02-22-Stuff The Internet Says On Scalability For February 22, 2013
11 0.80076498 1206 high scalability-2012-03-09-Stuff The Internet Says On Scalability For March 9, 2012
12 0.79953647 1368 high scalability-2012-12-07-Stuff The Internet Says On Scalability For December 7, 2012
13 0.79931331 1106 high scalability-2011-08-26-Stuff The Internet Says On Scalability For August 26, 2011
14 0.79473829 1613 high scalability-2014-03-17-Intuitively Showing How To Scale a Web Application Using a Coffee Shop as an Example
15 0.79425555 1534 high scalability-2013-10-18-Stuff The Internet Says On Scalability For October 18th, 2013
16 0.79211444 139 high scalability-2007-10-30-Paper: Dynamo: Amazon’s Highly Available Key-value Store
17 0.79028279 1378 high scalability-2012-12-28-Stuff The Internet Says On Scalability For December 28, 2012
19 0.78696305 1182 high scalability-2012-01-27-Stuff The Internet Says On Scalability For January 27, 2012
20 0.78542018 1596 high scalability-2014-02-14-Stuff The Internet Says On Scalability For February 14th, 2014
topicId topicWeight
[(1, 0.141), (2, 0.181), (10, 0.065), (27, 0.017), (30, 0.031), (40, 0.237), (47, 0.012), (61, 0.096), (77, 0.028), (79, 0.109), (85, 0.016), (94, 0.013)]
simIndex simValue blogId blogTitle
1 0.97944796 860 high scalability-2010-07-17-Hot Scalability Links for July 17, 2010
Introduction: And by hot I also mean temperature. Summer has arrived. It's sizzling here in Silicon Valley. Thank you air conditioning! Scale the web by appointing a Crawler Czar? Tom Foremski has the idea that Google should open up their index so sites wouldn't have to endure the constant pounding by ravenous crawler bots. Don MacAskill of SmugMug estimates 50% of our web server CPU resources are spent serving crawlers. What a waste. How this would all work with real-time feeds, paid feeds (Twitter, movies, ...), etc. is unknown, but does it make sense for all that money to be spent on extracting the same data over and over again? Tweets of Gold: jamesurquhart : Key to applications is architecture. Key for infrastructure supporting archs is configurability. Configurability==features . tjake : People who choose their datastore based oh hearsay and not their own evaluation are doomed . b6n : No global lock ever goes unpunished. MichaelSurt
Introduction: In Algorithm Design for Performance Aware VM Consolidation we learn some shocking facts (gambling in Casablanca?): Average server utilization in many data centers is low, estimated between 5% and 15%. This is wasteful because an idle server often consumes more than 50% of peak power. Surely that's just for old style datacenters? Nope. In Google data centers, workloads that are consolidated use only 50% of the processor cores. Every other processor core is left unused simply to ensure that performance does not degrade. It's a VM wasteland. The goal is to reduce waste by packing VMs onto machines without hurting performance or wasting resources. The idea is to select VMs that interfere the least with each other and places them together on the same server. It's a NP-Complete problem, but this paper describes a practical method that performs provably close to the optimal. Interestingly they can optimize for performance or power efficiency, so you can use different algorithm
3 0.95869213 1466 high scalability-2013-05-29-Amazon: Creating a Customer Utopia One Culture Hack at a Time
Introduction: If you don’t cannibalize yourself, someone else will. -- Steve Jobs America as the New World has a long history of inspiring Utopian communities . Early experiments were famously religious. But there have been many others as new waves of thought have inspired people to organize and try something different. In the 1840s Transcendentalists, believing the true path lay in the perfection of the individual, created intentional communities like Brook Farm . We've also seen socialist , anarchist , hippy , and virtually every other kind of Utopia in-between. Psychologist B.F. Skinner wrote an infamous book, Walden Two , with a more "scientific" take on creating a Utopian community and Ayn Rand in Atlas Shrugged had her free market version. I believe in startup organizations we see the modern version of Utopian energy in action. We now call it by names like "culture hacking", but the goal is the same: to create a new form of human community to achieve a profound go
4 0.9509964 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Earlier we talked about how a single botnet could harness more compute power than our largest super computers. Well, that's just the start of it. The amount of computer power available to the Ambient Cloud will be truly astounding. 2 Billion Personal Computers The number of personal computers is still growing. By 2014 one estimate is there will be 2 billion PCs . That's a giant reservoir of power to exploit, especially considering these new boxes are stuffed with multiple powerful processors and gigabytes of memory. 7 Billion Smartphones By now it's common wisdom smartphones are the computing platform of the future. It's plausible to assume the total number of mobile phones in use will roughly equal the number of people on earth. That's 7 billion smartphones. Smartphones aren't just tiny little wannabe computers anymore either. They are rea
Introduction: While working with Memcache the other night, it dawned on me that it’s usage as a distributed caching mechanism was really just one of many ways to use it. That there are in fact many alternative usages that one could find for Memcache if they could just realize what Memcache really is at its core – a simple distributed hash-table – is an important point worthy of further discussion. To be clear, when I say “simple”, by no means am I implying that Memcache’s implementation is simple, just that the ideas behind it are such. Think about that for a minute. What else could we use a simple distributed hash-table for, besides caching? How about using it as an alternative to the traditional shard lookup method we used in our Master Index Lookup scalability strategy, discussed previously here.
6 0.93880051 848 high scalability-2010-06-25-Hot Scalability Links for June 25, 2010
7 0.93693966 330 high scalability-2008-05-27-Should Twitter be an All-You-Can-Eat Buffet or a Vending Machine?
8 0.92463452 1414 high scalability-2013-03-01-Stuff The Internet Says On Scalability For February 29, 2013
9 0.91417646 27 high scalability-2007-07-25-Product: 3 PAR REMOTE COPY
10 0.90904599 768 high scalability-2010-02-01-What Will Kill the Cloud?
11 0.90859377 300 high scalability-2008-04-07-Scalr - Open Source Auto-scaling Hosting on Amazon EC2
same-blog 12 0.9071753 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
14 0.90005219 879 high scalability-2010-08-12-Think of Latency as a Pseudo-permanent Network Partition
15 0.89855844 1471 high scalability-2013-06-06-Paper: Memory Barriers: a Hardware View for Software Hackers
16 0.89529514 338 high scalability-2008-06-02-Total Cost of Ownership for different web development frameworks
17 0.89478445 402 high scalability-2008-10-05-Paper: Scalability Design Patterns
18 0.87459278 1492 high scalability-2013-07-17-How do you create a 100th Monkey software development culture?
19 0.82175833 97 high scalability-2007-09-18-Session management in highly scalable web sites
20 0.80747598 1515 high scalability-2013-09-11-Ten Lessons from GitHub’s First Year in 2008