high_scalability high_scalability-2010 high_scalability-2010-765 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. There's a pattern, already begun, that has accelerated by the need for applications to scale and increase complexity, the end result of which will be that applications give up their independence and enter a kind of feudal relationship with their platform provider. To understand how this process works, like a glacier slowly and inevitably carving out a deep river valley, here's the type of question I get quite a lot: I've learned PHP and MySQL and I've built a web app that I HOPE will receive traffic comparable to eBay's with a similar database structure. I see all these different opinions and different techniques and languages being recommended and it's so confusing. All I want is perhaps one book or one website that focuses on PHP and MySQL and building a large database web app like eBay's. Does something like this exist? I'm always at a loss fo
sentIndex sentText sentNum sentScore
1 There's a pattern, already begun, that has accelerated by the need for applications to scale and increase complexity, the end result of which will be that applications give up their independence and enter a kind of feudal relationship with their platform provider. [sent-2, score-0.724]
2 Applications have started to become mashups where the hard parts are written in terms of a platform that does all those hard parts. [sent-33, score-0.334]
3 Once adopted, a platform becomes hard to leave which adds a political dimension to the evolution of future application architectures. [sent-35, score-0.339]
4 This result may not be readily apparent because we are still so close to the era of small disconnected systems: application writers are becoming 'vassals' to their feudal platform 'lords', a kind of neo-feudalism that bares a striking relationship to the previous agriculture based feudalism. [sent-36, score-0.655]
5 This somewhat imprecise feudal analogy goes like this: the platform provider (the King) provides services, land, and protection to their platform users (vassals). [sent-37, score-0.975]
6 Datacenters are the New Land In the feudal era having land meant everything. [sent-46, score-0.592]
7 Only land owners could grow crops for food, for trade, and for money. [sent-47, score-0.393]
8 When the feudal era collapsed industry and trade became the new means for acquiring wealth. [sent-49, score-0.351]
9 To stretch the analogy further, in terms of building applications we have been in the hunter gather era, we are entering the feudal era of great land owners, and we are contemplating what might constitute the next industrial revolution. [sent-53, score-0.698]
10 A DDoS attack can take down even the largest of companies and no small land holder can even begin to defend themselves. [sent-66, score-0.423]
11 They informed me my email was shut down because there was a DDoS attack on my domain. [sent-77, score-0.335]
12 For my email provider that attack was anything but inconsequential. [sent-87, score-0.413]
13 On the frontier of the US when homesteaders came under attack and could no longer protect themselves, they had to move to the nearest fort for protection. [sent-91, score-0.503]
14 Meanwhile back at the frontier my previous email provider said the attack stopped after 10 days and I that could go back. [sent-100, score-0.52]
15 In good times you can homestead in the wilderness, but when the bandits attack and you can no longer bring your goods to market, you seek the protection of the King. [sent-112, score-0.338]
16 The King has Subtle Forms of Persuasion Originally I considered the driving forces of scale and complexity as sufficient to compel developers to end up in the arms of platform provider. [sent-113, score-0.31]
17 By linking pagerank to performance and providing all the intermediary steps necessary to become performant, the result is likely to drive developers to a platform for refuge. [sent-125, score-0.477]
18 Google is certainly not the only platform option, but people will go to a platform people as an easy prepackaged solution to a tough to solve problem. [sent-127, score-0.51]
19 Instead, we're facing the prospect of Facebook as the platform, Apple as the platform, Google as the platform, Amazon as the platform, where big companies slug it out until one is king of the hill. [sent-132, score-0.324]
20 Hinchcliffe thinks there will be a protracted platform war with a fairly predictable and oft-repeated cycle of events for which a small number of large winners are likely to emerge victorious . [sent-139, score-0.377]
wordName wordTfidf (topN-words)
[('king', 0.324), ('platform', 0.255), ('land', 0.241), ('feudal', 0.221), ('attack', 0.182), ('email', 0.153), ('google', 0.139), ('era', 0.13), ('war', 0.122), ('protection', 0.108), ('frontier', 0.107), ('ddos', 0.1), ('pagerank', 0.094), ('becomes', 0.084), ('owners', 0.083), ('become', 0.079), ('provider', 0.078), ('vassals', 0.077), ('tim', 0.076), ('appliances', 0.074), ('hinchcliffe', 0.074), ('fort', 0.074), ('homesteaders', 0.074), ('crops', 0.069), ('interoperable', 0.069), ('castles', 0.069), ('protect', 0.066), ('datacenter', 0.062), ('ebay', 0.062), ('digital', 0.058), ('analogy', 0.058), ('walls', 0.057), ('rob', 0.056), ('runner', 0.056), ('end', 0.055), ('slice', 0.055), ('ambient', 0.054), ('autonomic', 0.053), ('datacenters', 0.052), ('blade', 0.052), ('result', 0.049), ('made', 0.049), ('forced', 0.049), ('computing', 0.048), ('seek', 0.048), ('applications', 0.048), ('enter', 0.048), ('failures', 0.048), ('performant', 0.048), ('meets', 0.047)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000008 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. There's a pattern, already begun, that has accelerated by the need for applications to scale and increase complexity, the end result of which will be that applications give up their independence and enter a kind of feudal relationship with their platform provider. To understand how this process works, like a glacier slowly and inevitably carving out a deep river valley, here's the type of question I get quite a lot: I've learned PHP and MySQL and I've built a web app that I HOPE will receive traffic comparable to eBay's with a similar database structure. I see all these different opinions and different techniques and languages being recommended and it's so confusing. All I want is perhaps one book or one website that focuses on PHP and MySQL and building a large database web app like eBay's. Does something like this exist? I'm always at a loss fo
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
4 0.21013623 450 high scalability-2008-11-24-Scalability Perspectives #3: Marc Andreessen – Internet Platforms
Introduction: Scalability Perspectives is a series of posts that highlights the ideas that will shape the next decade of IT architecture. Each post is dedicated to a thought leader of the information age and his vision of the future. Be warned though – the journey into the minds and perspectives of these people requires an open mind. Marc Andreessen Marc Andreessen is known as an internet pioneer, entrepreneur, investor, startup coach, blogger, and a multi-millionaire software engineer best known as co-author of Mosaic, the first widely-used web browser, and founder of Netscape Communications Corporation. He was the chair of Opsware, a software company he founded originally as Loudcloud, when it was acquired by Hewlett-Packard. He is also a co-founder of Ning, a company which provides a platform for social-networking websites. He has recently joined the Board of Directors of Facebook and eBay. Marc is an investor in several startups including Digg, Metaplace, Plazes, Qik, and Twitter. His pas
5 0.20521219 790 high scalability-2010-03-09-Applications as Virtual States
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. As I was writing an article on the architecture of the Storm Botnet , I couldn't help but notice the deep similarity of how Storm works and changes we're seeing in the evolution of political systems. In particular, the rise of the virtual-state . As crazy as this may sound, I think this is also the direction applications will need follow to survive in a complex world of billions of compute devices. You may have already heard of virtual corporations . Virtual corporations are companies with limited office space, a distributed workforce, and production facilities located wherever it is profitable to locate them. The idea is to stay lean and compete using the rapid development and introduction of new products into high value-added markets . If you spot a market opportunity with a small time window, building your own factories and hiring and engine
6 0.20349437 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
7 0.15561077 768 high scalability-2010-02-01-What Will Kill the Cloud?
8 0.15351664 129 high scalability-2007-10-23-Hire Facebook, Ning, and Salesforce to Scale for You
9 0.15124042 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
10 0.1393306 301 high scalability-2008-04-08-Google AppEngine - A First Look
11 0.13776122 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
12 0.12957376 758 high scalability-2010-01-11-Have We Reached the End of Scaling?
13 0.12738545 517 high scalability-2009-02-21-Google AppEngine - A Second Look
14 0.12640972 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
15 0.12455397 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
16 0.12338648 687 high scalability-2009-08-24-How Google Serves Data from Multiple Datacenters
17 0.11911555 1501 high scalability-2013-08-13-In Memoriam: Lavabit Architecture - Creating a Scalable Email Service
18 0.11895707 753 high scalability-2009-12-21-Hot Holiday Scalability Links for 2009
19 0.11383051 363 high scalability-2008-08-12-Strategy: Limit The New, Not The Old
20 0.11276191 299 high scalability-2008-04-07-Rumors of Signs and Portents Concerning Freeish Google Cloud
topicId topicWeight
[(0, 0.245), (1, 0.094), (2, 0.068), (3, 0.116), (4, -0.036), (5, -0.12), (6, -0.05), (7, 0.044), (8, 0.002), (9, 0.024), (10, -0.049), (11, 0.044), (12, -0.016), (13, 0.083), (14, 0.118), (15, -0.047), (16, -0.091), (17, -0.059), (18, 0.051), (19, 0.031), (20, -0.031), (21, 0.052), (22, 0.006), (23, -0.015), (24, 0.062), (25, 0.069), (26, 0.005), (27, -0.013), (28, -0.024), (29, -0.085), (30, -0.115), (31, -0.039), (32, 0.004), (33, -0.08), (34, -0.022), (35, -0.068), (36, -0.083), (37, -0.012), (38, -0.038), (39, 0.005), (40, 0.102), (41, 0.026), (42, -0.0), (43, 0.029), (44, 0.063), (45, 0.037), (46, 0.052), (47, 0.008), (48, -0.001), (49, 0.047)]
simIndex simValue blogId blogTitle
same-blog 1 0.96307474 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. There's a pattern, already begun, that has accelerated by the need for applications to scale and increase complexity, the end result of which will be that applications give up their independence and enter a kind of feudal relationship with their platform provider. To understand how this process works, like a glacier slowly and inevitably carving out a deep river valley, here's the type of question I get quite a lot: I've learned PHP and MySQL and I've built a web app that I HOPE will receive traffic comparable to eBay's with a similar database structure. I see all these different opinions and different techniques and languages being recommended and it's so confusing. All I want is perhaps one book or one website that focuses on PHP and MySQL and building a large database web app like eBay's. Does something like this exist? I'm always at a loss fo
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
4 0.84808034 790 high scalability-2010-03-09-Applications as Virtual States
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. As I was writing an article on the architecture of the Storm Botnet , I couldn't help but notice the deep similarity of how Storm works and changes we're seeing in the evolution of political systems. In particular, the rise of the virtual-state . As crazy as this may sound, I think this is also the direction applications will need follow to survive in a complex world of billions of compute devices. You may have already heard of virtual corporations . Virtual corporations are companies with limited office space, a distributed workforce, and production facilities located wherever it is profitable to locate them. The idea is to stay lean and compete using the rapid development and introduction of new products into high value-added markets . If you spot a market opportunity with a small time window, building your own factories and hiring and engine
5 0.84038717 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. We tend to think compute of resources as residing primarily in datacenters. Given the fast pace of innovation we will likely see compute resources become pervasive. Some will reside in datacenters, but compute resources can be anywhere, not just in the datacenter, we'll actually see the bulk of compute resources live outside of datacenters in the future. Given the diversity of compute resources it's reasonable to assume they won't be homogeneous or conform to a standard API. They will specialize by service. Programmers will have to use those specialized service interfaces to build applications that are adaptive enough to take advantage of whatever leverage they can find, whenever and wherever they can find it. Once found the application will have to reorganize on the fly to use whatever new resources it has found and let go of whatever resources it doe
6 0.77029717 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
7 0.72763956 768 high scalability-2010-02-01-What Will Kill the Cloud?
8 0.72071332 1377 high scalability-2012-12-26-Ask HS: What will programming and architecture look like in 2020?
9 0.71170628 450 high scalability-2008-11-24-Scalability Perspectives #3: Marc Andreessen – Internet Platforms
10 0.70851749 1584 high scalability-2014-01-22-How would you build the next Internet? Loons, Drones, Copters, Satellites, or Something Else?
11 0.70067734 1012 high scalability-2011-03-28-Aztec Empire Strategy: Use Dual Pipes in Your Aqueduct for High Availability
12 0.68881398 201 high scalability-2008-01-04-For $5 Million You Can Buy Enough Storage to Compete with Google
13 0.68837357 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
14 0.68627685 363 high scalability-2008-08-12-Strategy: Limit The New, Not The Old
16 0.6730482 1091 high scalability-2011-08-02-How Will DIDO Wireless Networking Change Everything?
18 0.65973955 753 high scalability-2009-12-21-Hot Holiday Scalability Links for 2009
20 0.65547788 1410 high scalability-2013-02-20-Smart Companies Fail Because they Do Everything Right - Staying Alive to Scale
topicId topicWeight
[(1, 0.1), (2, 0.157), (10, 0.023), (11, 0.05), (17, 0.112), (30, 0.016), (40, 0.025), (47, 0.017), (51, 0.015), (61, 0.091), (77, 0.027), (79, 0.122), (80, 0.017), (85, 0.056), (94, 0.046), (96, 0.011)]
simIndex simValue blogId blogTitle
1 0.96558803 1225 high scalability-2012-04-09-Why My Slime Mold is Better than Your Hadoop Cluster
Introduction: Update : Organism without a brain creates external memories for navigation shows slime mold is even cooler than originally thought, storing a record of where it's been using slime: The authors conclude, the slime isn't just the mold's calling card. Instead, it's a way of marking the environment so that the organism can sense where it's been, and not expend effort on searches that won't pay off. Although the situation isn't an exact parallel, the authors make a comparison to the pheromone trails used by ants. In After Life: The Strange Science Of Decay there’s a truly incredible sequence of gorgeously shot video showing how creeping slime mold solves mazes and performs other other amazing feats of computation. Take a look at what simple one celled organisms can do: The whole video is really well done and shockingly revelatory. It’s the story of decay, how atoms created during the Big Bang and through countless supernova explosions are continually rearranged an
same-blog 2 0.94265872 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. There's a pattern, already begun, that has accelerated by the need for applications to scale and increase complexity, the end result of which will be that applications give up their independence and enter a kind of feudal relationship with their platform provider. To understand how this process works, like a glacier slowly and inevitably carving out a deep river valley, here's the type of question I get quite a lot: I've learned PHP and MySQL and I've built a web app that I HOPE will receive traffic comparable to eBay's with a similar database structure. I see all these different opinions and different techniques and languages being recommended and it's so confusing. All I want is perhaps one book or one website that focuses on PHP and MySQL and building a large database web app like eBay's. Does something like this exist? I'm always at a loss fo
3 0.94265056 506 high scalability-2009-02-03-10 More Rules for Even Faster Websites
Introduction: Update: How-To Minimize Load Time for Fast User Experiences . Shows how to analyze the bottlenecks preventing websites and blogs from loading quickly and how to resolve them. 80-90% of the end-user response time is spent on the frontend, so it makes sense to concentrate efforts there before heroically rewriting the backend. Take a shower before buying a Porsche, if you know what I mean. Steve Souders, author of High Performance Websites and Yslow , has ten more best practices to speed up your website : Split the initial payload Load scripts without blocking Don’t scatter scripts Split dominant content domains Make static content cookie-free Reduce cookie weight Minify CSS Optimize images Use iframes sparingly To www or not to www Sadly, according to String Theory, there are only 26.7 rules left, so get them while they're still in our dimension. Here are slides on the first few rules. Love the speeding dog slide. That's exactly what my dog looks like trav
Introduction: If you were going to design a next generation Internet at the physical layer that routes around the current Internet, what would it look like? What should it do? How should it work? Who should own it? How should it be paid for? How would you access it? It has long been said the Internet routes around obstacles. Snowden has revealed some major obstacles. The beauty of the current current app and web system is the physical network doesn't matter. We can just replace it with something else. Something that doesn't flow through choke points like backhaul networks , under sea cables , and cell towers . What might that something else look like? Google's Loon Project Project Loon was so named because the idea was thought to be loony. Maybe not. The idea is to float high-altitude balloons 20 miles in the air to create an aerial wireless network with up to 3G-like speeds. Signals travel through the balloon network from balloon to balloon, then to a ground-based station conne
5 0.92808563 869 high scalability-2010-07-30-Hot Scalability Links for July 30, 2010
Introduction: Jeremy Zawodny, while performing data alchemy in the dungeons of Craigslist, stored 1,250,000,000 Key/Value Pairs in Redis on a 32GB Machine . Data sorting world record: 1 terabyte, 1 minute . The system has 52 computer nodes, each node is a commodity server with two quad-core processors, 24 gigabytes (GB) memory and sixteen 500 GB disks . It's not just hardware though, they also built a software that utilized all their CPU and RAM. Tweets of Gold: wm : I am really getting the sense that none of you yokels waxing profound about scalability actually has anything factual to say joestump : I think you can do things to *mitigate* pain points up front. You don't need to over-engineer, but it's not hard to look forward. danielcrenna : I love it when I check in debug code accidentally and it turns into a three day hunt for a major scalability problem joestump : Your post also makes me think of another phrase I say often: Scaling == Specialization. Bigger scale =
6 0.9068132 1467 high scalability-2013-05-30-Google Finds NUMA Up to 20% Slower for Gmail and Websearch
7 0.90329814 956 high scalability-2010-12-08-How To Get Experience Working With Large Datasets
9 0.88627213 631 high scalability-2009-06-15-Large-scale Graph Computing at Google
10 0.88020027 1389 high scalability-2013-01-18-Stuff The Internet Says On Scalability For January 18, 2013
11 0.88019282 851 high scalability-2010-07-02-Hot Scalability Links for July 2, 2010
12 0.8799457 1333 high scalability-2012-10-04-LinkedIn Moved from Rails to Node: 27 Servers Cut and Up to 20x Faster
13 0.87916815 849 high scalability-2010-06-28-VoltDB Decapitates Six SQL Urban Myths and Delivers Internet Scale OLTP in the Process
14 0.87890118 1020 high scalability-2011-04-12-Caching and Processing 2TB Mozilla Crash Reports in memory with Hazelcast
15 0.87888998 507 high scalability-2009-02-03-Paper: Optimistic Replication
16 0.87823784 1170 high scalability-2012-01-06-Stuff The Internet Says On Scalability For January 6, 2012
17 0.87785697 1559 high scalability-2013-12-06-Stuff The Internet Says On Scalability For December 6th, 2013
18 0.87768555 1382 high scalability-2013-01-07-Analyzing billions of credit card transactions and serving low-latency insights in the cloud
20 0.8739869 1603 high scalability-2014-02-28-Stuff The Internet Says On Scalability For February 28th, 2014