high_scalability high_scalability-2010 high_scalability-2010-768 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. If datacenters are the new castles, then what will be the new gunpowder? As soon as gunpowder came on the scene, castles, which are defensive structures, quickly became the future's cold, drafty hotels. Gunpowder fueled cannon balls make short work of castle walls. There's a long history of "gunpowder" type inventions in the tech industry. PCs took out the timeshare model. The cloud is taking out the PC model. There must be something that will take out the cloud. Right now it's hard to believe the cloud will one day be no more. They seem so much the future, but something will transcend the cloud. We even have a law that says so: Bell's Law of Computer Classes which holds that roughly every decade a new, lower priced computer class forms based on a new programming platform, network, and interface resulting in new usage and the establishment of
sentIndex sentText sentNum sentScore
1 We even have a law that says so: Bell's Law of Computer Classes which holds that roughly every decade a new, lower priced computer class forms based on a new programming platform, network, and interface resulting in new usage and the establishment of a new industry. [sent-11, score-0.319]
2 These are dust sized chips with a relatively small numbers of transistors enable the creation of ubiquitous, radio networked, implantable, sensing platforms to be part of everything and everybody as a wireless sensor network class. [sent-17, score-0.377]
3 The cloud treats large clusters of powerful networked computers like a single abstraction, thus the Datacenter is the Computer. [sent-30, score-0.312]
4 While we have cloud masters creating exquisite platforms for developers, we haven't had much progress on mastering smartphones, wireless sensor networks, and body area networks. [sent-37, score-0.668]
5 What would kill the cloud is to move the characteristics of the cloud outside the datacenter. [sent-43, score-0.282]
6 Create super low latency, super high bandwidth networks, using super fast CPUs and super dense storage - that would be a cannon shot straight through the castle walls of the datacenter. [sent-44, score-0.92]
7 What we, in the US at least, won't have are wide spread super low latency and super high bandwidth connections. [sent-47, score-0.449]
8 These exist between within datacenters, between datacenters, and as backhaul networks to datacenters, but on a point-to-point basis the US bandwidth picture is grim. [sent-48, score-0.363]
9 A seemingly small difference, but over a 10-year time period this means computer power grows 100x, but bandwidth only grows at 57x. [sent-50, score-0.515]
10 The Amazing Collective Compute Power of the Ambient Cloud Earlier we talked about how a single botnet could harness more compute power than our largest super computers. [sent-62, score-0.41]
11 Their computing capabilities will only increase as smartphones are fit with more processors and more graphics processors . [sent-79, score-0.428]
12 But all the compute power in the world is of little use if the cores can't talk to each other. [sent-89, score-0.351]
13 The Cell Network Bandwidth and Latency Question Aren't cell phone networks slow and have high latencies? [sent-90, score-0.326]
14 For cell connections your milleage may vary considerably as cell performance changes on a cell-by-cell manner according to the individual site's local demographics, projected traffic demand and the target coverage area of the cell. [sent-94, score-0.547]
15 Current bandwidth rates and latencies for cell networks don't make them a sturdy a platform on which to build a cloud. [sent-109, score-0.522]
16 True HPC (high performance computing) low-latency-interconnect applications won't find a cell based cloud attractive at all. [sent-111, score-0.313]
17 Another potential source of distributed compute power are sensors on smart grids . [sent-124, score-0.307]
18 BANs are sensor networks in and around the human body . [sent-137, score-0.387]
19 Let's compare the collective power of PCs + smartphones + smart grid + smart everything + BANs with the traditional cloud: it's trillions against many 10s of millions. [sent-145, score-0.604]
20 And there are many many more distributed computing projects like SETI@home supplying huge amounts of compute power to their users. [sent-161, score-0.299]
wordName wordTfidf (topN-words)
[('smartphones', 0.202), ('ambient', 0.197), ('super', 0.173), ('cell', 0.172), ('pcs', 0.165), ('networks', 0.154), ('computer', 0.142), ('sensor', 0.142), ('cloud', 0.141), ('area', 0.127), ('compute', 0.121), ('bell', 0.119), ('power', 0.116), ('cores', 0.114), ('exist', 0.106), ('gunpowder', 0.106), ('mainframes', 0.106), ('wireless', 0.105), ('bandwidth', 0.103), ('computers', 0.099), ('plura', 0.097), ('datacenters', 0.096), ('latencies', 0.093), ('body', 0.091), ('specialty', 0.089), ('processors', 0.082), ('collective', 0.08), ('personal', 0.079), ('grows', 0.077), ('projected', 0.076), ('networked', 0.072), ('minis', 0.07), ('smart', 0.07), ('cpus', 0.067), ('network', 0.066), ('grid', 0.066), ('bans', 0.065), ('cannon', 0.065), ('chips', 0.064), ('castles', 0.063), ('symbian', 0.063), ('home', 0.062), ('progress', 0.062), ('computing', 0.062), ('future', 0.061), ('palm', 0.061), ('seti', 0.061), ('castle', 0.06), ('era', 0.059), ('new', 0.059)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999982 768 high scalability-2010-02-01-What Will Kill the Cloud?
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. If datacenters are the new castles, then what will be the new gunpowder? As soon as gunpowder came on the scene, castles, which are defensive structures, quickly became the future's cold, drafty hotels. Gunpowder fueled cannon balls make short work of castle walls. There's a long history of "gunpowder" type inventions in the tech industry. PCs took out the timeshare model. The cloud is taking out the PC model. There must be something that will take out the cloud. Right now it's hard to believe the cloud will one day be no more. They seem so much the future, but something will transcend the cloud. We even have a law that says so: Bell's Law of Computer Classes which holds that roughly every decade a new, lower priced computer class forms based on a new programming platform, network, and interface resulting in new usage and the establishment of
2 0.81770724 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Earlier we talked about how a single botnet could harness more compute power than our largest super computers. Well, that's just the start of it. The amount of computer power available to the Ambient Cloud will be truly astounding. 2 Billion Personal Computers The number of personal computers is still growing. By 2014 one estimate is there will be 2 billion PCs . That's a giant reservoir of power to exploit, especially considering these new boxes are stuffed with multiple powerful processors and gigabytes of memory. 7 Billion Smartphones By now it's common wisdom smartphones are the computing platform of the future. It's plausible to assume the total number of mobile phones in use will roughly equal the number of people on earth. That's 7 billion smartphones. Smartphones aren't just tiny little wannabe computers anymore either. They are rea
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
5 0.27114633 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. The future looks many, big, complex, and adaptive: Many clouds. Many servers. Many operating systems. Many languages. Many storage services. Many database services. Many software services. Many adjunct human networks (like Mechanical Turk). Many fast interconnects. Many CDNs. Many cache memory pools. Many application profiles (simple request-response, live streaming, computationally complex, sensor driven, memory intensive, storage intensive, monolithic, decomposable, etc). Many legal jurisdictions. Don't want to perform a function on Patriot Act "protected" systems then move the function elsewhere. Many SLAs. Many data driven pricing policies that like airplane pricing algorithms will price "seats" to maximize profit using multi-variate time sensitive pricing models. Many competitive products. The need t
6 0.27060553 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
7 0.20642608 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
8 0.19013752 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
9 0.18568622 882 high scalability-2010-08-18-Misco: A MapReduce Framework for Mobile Systems - Start of the Ambient Cloud?
10 0.18123828 758 high scalability-2010-01-11-Have We Reached the End of Scaling?
11 0.16816975 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
12 0.1632349 823 high scalability-2010-05-05-How will memristors change everything?
13 0.15578671 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
14 0.15561077 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
15 0.15443255 636 high scalability-2009-06-23-Learn How to Exploit Multiple Cores for Better Performance and Scalability
16 0.14598794 1242 high scalability-2012-05-09-Cell Architectures
17 0.14595526 661 high scalability-2009-07-25-Latency is Everywhere and it Costs You Sales - How to Crush it
18 0.14290106 1519 high scalability-2013-09-18-If You're Programming a Cell Phone Like a Server You're Doing it Wrong
19 0.13842261 1451 high scalability-2013-05-03-Stuff The Internet Says On Scalability For May 3, 2013
20 0.13314749 1603 high scalability-2014-02-28-Stuff The Internet Says On Scalability For February 28th, 2014
topicId topicWeight
[(0, 0.282), (1, 0.117), (2, 0.118), (3, 0.18), (4, -0.17), (5, -0.13), (6, -0.011), (7, 0.086), (8, -0.088), (9, 0.055), (10, -0.009), (11, 0.01), (12, 0.035), (13, 0.179), (14, 0.107), (15, 0.089), (16, -0.193), (17, -0.04), (18, -0.104), (19, 0.061), (20, -0.174), (21, 0.108), (22, -0.14), (23, -0.012), (24, 0.093), (25, -0.025), (26, -0.067), (27, 0.041), (28, 0.051), (29, -0.133), (30, -0.072), (31, 0.022), (32, 0.016), (33, -0.015), (34, 0.094), (35, -0.073), (36, -0.042), (37, 0.012), (38, -0.098), (39, -0.02), (40, 0.072), (41, -0.003), (42, -0.001), (43, -0.091), (44, 0.044), (45, 0.014), (46, 0.078), (47, 0.094), (48, -0.02), (49, 0.028)]
simIndex simValue blogId blogTitle
1 0.95956922 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Earlier we talked about how a single botnet could harness more compute power than our largest super computers. Well, that's just the start of it. The amount of computer power available to the Ambient Cloud will be truly astounding. 2 Billion Personal Computers The number of personal computers is still growing. By 2014 one estimate is there will be 2 billion PCs . That's a giant reservoir of power to exploit, especially considering these new boxes are stuffed with multiple powerful processors and gigabytes of memory. 7 Billion Smartphones By now it's common wisdom smartphones are the computing platform of the future. It's plausible to assume the total number of mobile phones in use will roughly equal the number of people on earth. That's 7 billion smartphones. Smartphones aren't just tiny little wannabe computers anymore either. They are rea
same-blog 2 0.95459604 768 high scalability-2010-02-01-What Will Kill the Cloud?
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. If datacenters are the new castles, then what will be the new gunpowder? As soon as gunpowder came on the scene, castles, which are defensive structures, quickly became the future's cold, drafty hotels. Gunpowder fueled cannon balls make short work of castle walls. There's a long history of "gunpowder" type inventions in the tech industry. PCs took out the timeshare model. The cloud is taking out the PC model. There must be something that will take out the cloud. Right now it's hard to believe the cloud will one day be no more. They seem so much the future, but something will transcend the cloud. We even have a law that says so: Bell's Law of Computer Classes which holds that roughly every decade a new, lower priced computer class forms based on a new programming platform, network, and interface resulting in new usage and the establishment of
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
5 0.75586247 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. We tend to think compute of resources as residing primarily in datacenters. Given the fast pace of innovation we will likely see compute resources become pervasive. Some will reside in datacenters, but compute resources can be anywhere, not just in the datacenter, we'll actually see the bulk of compute resources live outside of datacenters in the future. Given the diversity of compute resources it's reasonable to assume they won't be homogeneous or conform to a standard API. They will specialize by service. Programmers will have to use those specialized service interfaces to build applications that are adaptive enough to take advantage of whatever leverage they can find, whenever and wherever they can find it. Once found the application will have to reorganize on the fly to use whatever new resources it has found and let go of whatever resources it doe
6 0.74911046 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
7 0.69676381 790 high scalability-2010-03-09-Applications as Virtual States
8 0.66209036 826 high scalability-2010-05-12-The Rise of the Virtual Cellular Machines
9 0.63351196 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
10 0.63166589 460 high scalability-2008-12-05-Scalability Perspectives #4: Kevin Kelly – One Machine
11 0.62774426 823 high scalability-2010-05-05-How will memristors change everything?
12 0.61947352 1584 high scalability-2014-01-22-How would you build the next Internet? Loons, Drones, Copters, Satellites, or Something Else?
13 0.61208457 758 high scalability-2010-01-11-Have We Reached the End of Scaling?
14 0.60288918 1091 high scalability-2011-08-02-How Will DIDO Wireless Networking Change Everything?
15 0.60105693 839 high scalability-2010-06-09-Paper: Propagation Networks: A Flexible and Expressive Substrate for Computation
16 0.58914995 439 high scalability-2008-11-10-Scalability Perspectives #1: Nicholas Carr – The Big Switch
17 0.56112874 953 high scalability-2010-12-03-GPU vs CPU Smackdown : The Rise of Throughput-Oriented Architectures
19 0.5557062 753 high scalability-2009-12-21-Hot Holiday Scalability Links for 2009
20 0.54800749 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
topicId topicWeight
[(1, 0.1), (2, 0.158), (5, 0.013), (10, 0.027), (30, 0.021), (40, 0.21), (47, 0.03), (61, 0.075), (77, 0.021), (79, 0.125), (85, 0.04), (94, 0.033), (95, 0.078)]
simIndex simValue blogId blogTitle
1 0.94499004 860 high scalability-2010-07-17-Hot Scalability Links for July 17, 2010
Introduction: And by hot I also mean temperature. Summer has arrived. It's sizzling here in Silicon Valley. Thank you air conditioning! Scale the web by appointing a Crawler Czar? Tom Foremski has the idea that Google should open up their index so sites wouldn't have to endure the constant pounding by ravenous crawler bots. Don MacAskill of SmugMug estimates 50% of our web server CPU resources are spent serving crawlers. What a waste. How this would all work with real-time feeds, paid feeds (Twitter, movies, ...), etc. is unknown, but does it make sense for all that money to be spent on extracting the same data over and over again? Tweets of Gold: jamesurquhart : Key to applications is architecture. Key for infrastructure supporting archs is configurability. Configurability==features . tjake : People who choose their datastore based oh hearsay and not their own evaluation are doomed . b6n : No global lock ever goes unpunished. MichaelSurt
2 0.93195313 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Earlier we talked about how a single botnet could harness more compute power than our largest super computers. Well, that's just the start of it. The amount of computer power available to the Ambient Cloud will be truly astounding. 2 Billion Personal Computers The number of personal computers is still growing. By 2014 one estimate is there will be 2 billion PCs . That's a giant reservoir of power to exploit, especially considering these new boxes are stuffed with multiple powerful processors and gigabytes of memory. 7 Billion Smartphones By now it's common wisdom smartphones are the computing platform of the future. It's plausible to assume the total number of mobile phones in use will roughly equal the number of people on earth. That's 7 billion smartphones. Smartphones aren't just tiny little wannabe computers anymore either. They are rea
3 0.93150479 1419 high scalability-2013-03-07-It's a VM Wasteland - A Near Optimal Packing of VMs to Machines Reduces TCO by 22%
Introduction: In Algorithm Design for Performance Aware VM Consolidation we learn some shocking facts (gambling in Casablanca?): Average server utilization in many data centers is low, estimated between 5% and 15%. This is wasteful because an idle server often consumes more than 50% of peak power. Surely that's just for old style datacenters? Nope. In Google data centers, workloads that are consolidated use only 50% of the processor cores. Every other processor core is left unused simply to ensure that performance does not degrade. It's a VM wasteland. The goal is to reduce waste by packing VMs onto machines without hurting performance or wasting resources. The idea is to select VMs that interfere the least with each other and places them together on the same server. It's a NP-Complete problem, but this paper describes a practical method that performs provably close to the optimal. Interestingly they can optimize for performance or power efficiency, so you can use different algorithm
4 0.92738789 330 high scalability-2008-05-27-Should Twitter be an All-You-Can-Eat Buffet or a Vending Machine?
Introduction: Om proposes one solution to the Twitter Problem is to limit followers to three square meals a day. The reasonable idea being that lower limits should mean fewer scaling problems. And as a kicker raising those limits is a good way to raise much needed revenue. Scoble thinks users should consume without limit and will drive to another buffet if all-you-can-eat privileges are revoked. The reasonable idea being that if an internet service can't solve internet scale problems then there's not much use for it. Dave says comp power users a top floor suite and shower them with free passes to the buffet. Let the good times roll! The reasonable idea being that power users help create popular restaurants, er, services in the first place and limiting them starves users and starved users won't come back. So, should web services like Twitter be a buffet, a fixed eight course fine dining experience, a small plate restaurant, a family style joint, or a vending machine? Or
5 0.92550308 1466 high scalability-2013-05-29-Amazon: Creating a Customer Utopia One Culture Hack at a Time
Introduction: If you don’t cannibalize yourself, someone else will. -- Steve Jobs America as the New World has a long history of inspiring Utopian communities . Early experiments were famously religious. But there have been many others as new waves of thought have inspired people to organize and try something different. In the 1840s Transcendentalists, believing the true path lay in the perfection of the individual, created intentional communities like Brook Farm . We've also seen socialist , anarchist , hippy , and virtually every other kind of Utopia in-between. Psychologist B.F. Skinner wrote an infamous book, Walden Two , with a more "scientific" take on creating a Utopian community and Ayn Rand in Atlas Shrugged had her free market version. I believe in startup organizations we see the modern version of Utopian energy in action. We now call it by names like "culture hacking", but the goal is the same: to create a new form of human community to achieve a profound go
same-blog 6 0.9123379 768 high scalability-2010-02-01-What Will Kill the Cloud?
8 0.89430231 338 high scalability-2008-06-02-Total Cost of Ownership for different web development frameworks
9 0.89195961 879 high scalability-2010-08-12-Think of Latency as a Pseudo-permanent Network Partition
10 0.88535994 300 high scalability-2008-04-07-Scalr - Open Source Auto-scaling Hosting on Amazon EC2
11 0.88413674 848 high scalability-2010-06-25-Hot Scalability Links for June 25, 2010
12 0.882792 1414 high scalability-2013-03-01-Stuff The Internet Says On Scalability For February 29, 2013
13 0.87093157 402 high scalability-2008-10-05-Paper: Scalability Design Patterns
14 0.86792189 1471 high scalability-2013-06-06-Paper: Memory Barriers: a Hardware View for Software Hackers
15 0.86681157 27 high scalability-2007-07-25-Product: 3 PAR REMOTE COPY
16 0.85417891 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
18 0.84634233 1492 high scalability-2013-07-17-How do you create a 100th Monkey software development culture?
20 0.7828545 1647 high scalability-2014-05-14-Google Says Cloud Prices Will Follow Moore’s Law: Are We All Renters Now?