high_scalability high_scalability-2010 high_scalability-2010-758 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Have we reached the end of scaling? That's what I asked myself one day after noticing a bunch of "The End of" headlines. We've reached The End of History because the Western liberal democracy is the "end point of humanity's sociocultural evolution and the final form of human government." We've reached The End of Science because of the "fact that there aren't going to be any obvious, cataclysmic revolutions." We've even reached The End of Theory because all answers can be found in the continuous stream of data we're collecting. And doesn't always seem like we're at The End of the World ? Motivated by the prospect of everything ending, I began to wonder: have we really reached The End of Scaling? For a while I thought this might be true. The reason I thought the End of Scaling might be near is because of the slow down of potential articles at m
sentIndex sentText sentNum sentScore
1 We've reached The End of History because the Western liberal democracy is the "end point of humanity's sociocultural evolution and the final form of human government. [sent-4, score-0.225]
2 Then I got to thinking, if I really don't think any of those other "end of" ideas are true, then we probably haven't really reached the end of scaling either. [sent-18, score-0.441]
3 More than likely what we've reached is the end of my vision. [sent-19, score-0.349]
4 No social networking site will support 7 billion friends . [sent-36, score-0.543]
5 No social networking site supports updating presence to 7 billion friends. [sent-37, score-0.543]
6 No site can handle a comment thread of 7 billion people. [sent-38, score-0.338]
7 No photography site will allow 7 billion people to upload pictures. [sent-39, score-0.55]
8 No lifestreaming site can wireless stream HD input from 7 billion eye glass mounted cams. [sent-40, score-0.406]
9 No virtual world allows 7 billion people to roam free and unhindered by meat conventions. [sent-41, score-0.504]
10 No music site allows 7 billion people listen to music at the same time. [sent-42, score-0.648]
11 I hope it's not so the entire world can know how another interchangeable starlet is having plastic surgery, while in rehab, during an multiple adoption process. [sent-53, score-0.263]
12 As a world we are facing worldwide problems that can only be solved with worldwide cooperation. [sent-54, score-0.435]
13 It would seem plausible that 7 billion people might want to follow them, read about their fresh insights, and what they had for lunch. [sent-60, score-0.377]
14 It seems reasonable that if there's some grand planet impacting scheme to combat global warming that a worldwide poll of 7 billion people would be run. [sent-61, score-0.693]
15 And if the newest boy robot band sensation released a new song, shouldn't 7 billion people know that immediately? [sent-62, score-0.377]
16 So planet sized systems are reasonable to think about, they just aren't even vaguely possible at the moment. [sent-63, score-0.244]
17 Imagine creating your own weather forecast by following billions of weather sensors. [sent-69, score-0.295]
18 Now we are not talking about a lousy 7 billion people anymore, we are talking about designing systems where trillions of entities follow each other, talk to each other, read each other, post to each other, often at prodigious data rates. [sent-72, score-0.744]
19 This is what often drags down social networking sites. [sent-74, score-0.273]
20 We are talking about systems 4 or 5 magnitudes (10,000 to 100,000 times) larger than our largest systems today. [sent-84, score-0.252]
wordName wordTfidf (topN-words)
[('billion', 0.233), ('reachedthe', 0.203), ('end', 0.192), ('reached', 0.157), ('worldwide', 0.154), ('people', 0.144), ('imagine', 0.135), ('social', 0.129), ('world', 0.127), ('trillions', 0.109), ('site', 0.105), ('following', 0.099), ('weather', 0.098), ('runner', 0.098), ('sensors', 0.096), ('ambient', 0.094), ('autonomic', 0.092), ('scaling', 0.092), ('planet', 0.091), ('blade', 0.091), ('talking', 0.088), ('music', 0.083), ('meets', 0.082), ('systems', 0.082), ('smart', 0.08), ('facebook', 0.078), ('photos', 0.076), ('networking', 0.076), ('reasonable', 0.071), ('noticing', 0.068), ('applicationsbecause', 0.068), ('audaciously', 0.068), ('cooperationworks', 0.068), ('densely', 0.068), ('drags', 0.068), ('followerscause', 0.068), ('historybecause', 0.068), ('isvery', 0.068), ('lifestreaming', 0.068), ('mahatma', 0.068), ('medicines', 0.068), ('networksfar', 0.068), ('phew', 0.068), ('photography', 0.068), ('plastic', 0.068), ('politicians', 0.068), ('rehab', 0.068), ('sciencebecause', 0.068), ('sociocultural', 0.068), ('starlet', 0.068)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000002 758 high scalability-2010-01-11-Have We Reached the End of Scaling?
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Have we reached the end of scaling? That's what I asked myself one day after noticing a bunch of "The End of" headlines. We've reached The End of History because the Western liberal democracy is the "end point of humanity's sociocultural evolution and the final form of human government." We've reached The End of Science because of the "fact that there aren't going to be any obvious, cataclysmic revolutions." We've even reached The End of Theory because all answers can be found in the continuous stream of data we're collecting. And doesn't always seem like we're at The End of the World ? Motivated by the prospect of everything ending, I began to wonder: have we really reached The End of Scaling? For a while I thought this might be true. The reason I thought the End of Scaling might be near is because of the slow down of potential articles at m
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
4 0.1949144 721 high scalability-2009-10-13-Why are Facebook, Digg, and Twitter so hard to scale?
Introduction: Real-time social graphs (connectivity between people, places, and things). That's why scaling Facebook is hard says Jeff Rothschild , Vice President of Technology at Facebook. Social networking sites like Facebook, Digg, and Twitter are simply harder than traditional websites to scale. Why is that? Why would social networking sites be any more difficult to scale than traditional web sites? Let's find out. Traditional websites are easier to scale than social networking sites for two reasons: They usually access only their own data and common cached data. Only 1-2% of users are active on the site at one time. Imagine a huge site like Yahoo. When you come to Yahoo they can get your profile record with one get and that's enough to build your view of the website for you. It's relatively straightforward to scale systems based around single records using distributed hashing schemes . And since only a few percent of the people are on the site at once it takes comparatively little
5 0.18592249 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Earlier we talked about how a single botnet could harness more compute power than our largest super computers. Well, that's just the start of it. The amount of computer power available to the Ambient Cloud will be truly astounding. 2 Billion Personal Computers The number of personal computers is still growing. By 2014 one estimate is there will be 2 billion PCs . That's a giant reservoir of power to exploit, especially considering these new boxes are stuffed with multiple powerful processors and gigabytes of memory. 7 Billion Smartphones By now it's common wisdom smartphones are the computing platform of the future. It's plausible to assume the total number of mobile phones in use will roughly equal the number of people on earth. That's 7 billion smartphones. Smartphones aren't just tiny little wannabe computers anymore either. They are rea
6 0.18322884 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
7 0.18123828 768 high scalability-2010-02-01-What Will Kill the Cloud?
8 0.16808036 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
9 0.14614354 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
10 0.14128141 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
11 0.13866287 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
12 0.13735794 1008 high scalability-2011-03-22-Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
13 0.1333859 106 high scalability-2007-10-02-Secrets to Fotolog's Scaling Success
14 0.13192418 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
15 0.12957376 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
16 0.12574644 638 high scalability-2009-06-26-PlentyOfFish Architecture
17 0.12568149 1361 high scalability-2012-11-22-Gone Fishin': PlentyOfFish Architecture
18 0.12482902 837 high scalability-2010-06-07-Six Ways Twitter May Reach its Big Hairy Audacious Goal of One Billion Users
19 0.12401946 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
20 0.12335104 1 high scalability-2007-07-06-Start Here
topicId topicWeight
[(0, 0.254), (1, 0.093), (2, 0.03), (3, 0.028), (4, 0.038), (5, -0.113), (6, -0.108), (7, 0.081), (8, 0.024), (9, 0.041), (10, 0.009), (11, 0.067), (12, -0.012), (13, 0.111), (14, 0.049), (15, 0.038), (16, -0.064), (17, 0.014), (18, -0.011), (19, 0.131), (20, -0.061), (21, 0.077), (22, 0.003), (23, 0.029), (24, 0.025), (25, -0.055), (26, -0.039), (27, 0.046), (28, 0.036), (29, -0.112), (30, -0.031), (31, -0.022), (32, -0.022), (33, 0.008), (34, 0.023), (35, -0.039), (36, -0.023), (37, -0.024), (38, -0.027), (39, 0.017), (40, 0.03), (41, -0.012), (42, -0.043), (43, -0.016), (44, 0.018), (45, 0.029), (46, 0.019), (47, 0.017), (48, -0.012), (49, -0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.9630242 758 high scalability-2010-01-11-Have We Reached the End of Scaling?
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Have we reached the end of scaling? That's what I asked myself one day after noticing a bunch of "The End of" headlines. We've reached The End of History because the Western liberal democracy is the "end point of humanity's sociocultural evolution and the final form of human government." We've reached The End of Science because of the "fact that there aren't going to be any obvious, cataclysmic revolutions." We've even reached The End of Theory because all answers can be found in the continuous stream of data we're collecting. And doesn't always seem like we're at The End of the World ? Motivated by the prospect of everything ending, I began to wonder: have we really reached The End of Scaling? For a while I thought this might be true. The reason I thought the End of Scaling might be near is because of the slow down of potential articles at m
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
4 0.77407205 976 high scalability-2011-01-20-75% Chance of Scale - Leveraging the New Scaleogenic Environment for Growth
Introduction: "I'll never need to scale so why bother? We aren't Twitter or Facebook or Google after all." This is the most common email I get, a question in the form of a thinly disguised rationalization for not having to worry about scaling. And in these days of giant transformer-like machines they are probably right. But what if there are Barry Bonds enhancing type forces at work that argue for the chances of your needing to scale being higher than you think? And if that happens, how will you cross the scalability chasm? Will you want to completely change your architecture or evolve it from a tool-chain that was meant to scale from the start? Architecturally, that's the question you have to answer. Today's tool-chains are making it possible to grow a system from small to large without needing to implement complete architectural phase changes at various scale inflection points, but that's a different topic. We're trying to think about why you may actually need to scale, that is the question.
5 0.75039834 460 high scalability-2008-12-05-Scalability Perspectives #4: Kevin Kelly – One Machine
Introduction: Scalability Perspectives is a series of posts that highlights the ideas that will shape the next decade of IT architecture. Each post is dedicated to a thought leader of the information age and his vision of the future. Be warned though – the journey into the minds and perspectives of these people requires an open mind. Warning #2: this post is wild. Kevin Kelly Kevin Kelly is Senior Maverick at Wired magazine. He helped launch Wired in 1993, and served as its Executive Editor until January 1999. He co-founded the ongoing Hackers' Conference, and was involved with the launch of the WELL, a pioneering online service started in 1985. He authored the best-selling New Rules for the New Economy and the classic book on decentralized emergent systems, Out of Control One Machine There is only one time in the history of each planet when its inhabitants first wire up its innumerable parts to make one large Machine. Later that Machine may run faster, but there is only on
6 0.74770516 768 high scalability-2010-02-01-What Will Kill the Cloud?
7 0.74587995 153 high scalability-2007-11-13-Friendster Lost Lead Because of a Failure to Scale
8 0.73021311 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
9 0.72849065 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
10 0.71354985 790 high scalability-2010-03-09-Applications as Virtual States
11 0.7109794 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
12 0.70540857 721 high scalability-2009-10-13-Why are Facebook, Digg, and Twitter so hard to scale?
13 0.70515066 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
14 0.70365316 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
15 0.69827801 1091 high scalability-2011-08-02-How Will DIDO Wireless Networking Change Everything?
16 0.69781995 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
17 0.69639373 1377 high scalability-2012-12-26-Ask HS: What will programming and architecture look like in 2020?
18 0.69411951 330 high scalability-2008-05-27-Should Twitter be an All-You-Can-Eat Buffet or a Vending Machine?
19 0.68481976 129 high scalability-2007-10-23-Hire Facebook, Ning, and Salesforce to Scale for You
20 0.68349415 1484 high scalability-2013-06-28-Stuff The Internet Says On Scalability For June 28, 2013
topicId topicWeight
[(1, 0.147), (2, 0.192), (5, 0.012), (10, 0.039), (13, 0.222), (30, 0.024), (47, 0.021), (51, 0.015), (61, 0.042), (77, 0.019), (79, 0.124), (85, 0.022), (94, 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 0.89462018 758 high scalability-2010-01-11-Have We Reached the End of Scaling?
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Have we reached the end of scaling? That's what I asked myself one day after noticing a bunch of "The End of" headlines. We've reached The End of History because the Western liberal democracy is the "end point of humanity's sociocultural evolution and the final form of human government." We've reached The End of Science because of the "fact that there aren't going to be any obvious, cataclysmic revolutions." We've even reached The End of Theory because all answers can be found in the continuous stream of data we're collecting. And doesn't always seem like we're at The End of the World ? Motivated by the prospect of everything ending, I began to wonder: have we really reached The End of Scaling? For a while I thought this might be true. The reason I thought the End of Scaling might be near is because of the slow down of potential articles at m
2 0.8924495 1282 high scalability-2012-07-12-4 Strategies for Punching Down Traffic Spikes
Introduction: Travis Reeder in Spikability - An Application's Ability to Handle Unknown and/or Inconsistent Load gives four good ways of handling spikey loads: Have more resources than you'll ever need . Estimate the maximum traffic you'll need and keep that many servers running. Downside is you are paying for capacity you aren't using. Disable features during high loads . Reduce load by disabling features or substituting in lighter weight features. Downside is users to have access to features. Auto scaling . Launch new servers in response to load. Downsides are it's complicated to setup and slow to respond. Random spikes will cause cycling of instances going up and down. Use message queues . Queues soak up work requests during traffic spikes. More servers can be started to process work from the queue. Resources aren't wasted and features are disabled. Downside is increased latency.
3 0.87456834 34 high scalability-2007-07-27-Product: Munin Monitoriting Tool
Introduction: Munin the monitoring tool surveys all your computers and remembers what it saw. It presents all the information in graphs through a web interface. Its emphasis is on plug and play capabilities. After completing a installation a high number of monitoring plugins will be playing with no more effort. Using Munin you can easily monitor the performance of your computers, networks, SANs, applications, weather measurements and whatever comes to mind. It makes it easy to determine "what's different today" when a performance problem crops up. It makes it easy to see how you're doing capacity-wise on any resources.
4 0.86802435 391 high scalability-2008-09-23-The 7 Stages of Scaling Web Apps
Introduction: By John Engales CTO, Rackspace. Good presentation of the stages a typical successful website goes through: Stage 1 - The Beginning: Simple architecture, low complexity. no redundancy. Firewall, load balancer, a pair of web servers, database server, and internal storage. Stage 2 - More of the same, just bigger. Stage 3 - The Pain Begins: publicity hits. Use reverse proxy, cache static content, load balancers, more databases, re-coding. Stage 4 - The Pain Intensifies: caching with memcached, writes overload and replication takes too long, start database partitioning, shared storage makes sense for content, significant re-architecting for DB. Stage 5 - This Really Hurts!: rethink entire application, partition on geography user ID, etc, create user clusters, using hashing scheme for locating which user belongs to which cluster. Stage 6 - Getting a little less painful: scalable application and database architecture, acceptable performance, starting to add ne features again, op
5 0.86799985 477 high scalability-2008-12-29-100% on Amazon Web Services: Soocial.com - a lesson of porting your service to Amazon
Introduction: Simone Brunozzi, technology evangelist for Amazon Web Services in Europe, describes how Soocial.com was fully ported to Amazon web services. ---------------- This period of the year I decided to dedicate some time to better understand how our customers use AWS, therefore I spent some online time with Stefan Fountain and the nice guys at Soocial.com, a "one address book solution to contact management", and I would like to share with you some details of their IT infrastructure, which now runs 100% on Amazon Web Services! In the last few months, they've been working hard to cope with tens of thousands of users and to get ready to easily scale to millions. To make this possible, they decided to move ALL their architecture to Amazon Web Services. Despite the fact that they were quite happy with their previous hosting provider, Amazon proved to be the way to go. ----------------- Read the rest of the article here .
6 0.86357731 1576 high scalability-2014-01-10-Stuff The Internet Says On Scalability For January 10th, 2014
7 0.85560471 1070 high scalability-2011-06-29-Second Hand Seizure : A New Cause of Site Death
8 0.85538435 112 high scalability-2007-10-04-You Can Now Store All Your Stuff on Your Own Google Like File System
9 0.83669388 664 high scalability-2009-07-29-Strategy: Devirtualize for More Vroom
10 0.82132107 1431 high scalability-2013-03-29-Stuff The Internet Says On Scalability For March 29, 2013
11 0.80323273 1446 high scalability-2013-04-25-Paper: Making reliable distributed systems in the presence of software errors
12 0.80243605 890 high scalability-2010-09-01-Paper: The Case for Determinism in Database Systems
13 0.78371602 624 high scalability-2009-06-10-Hive - A Petabyte Scale Data Warehouse using Hadoop
14 0.78197449 327 high scalability-2008-05-27-How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale
15 0.77864259 1654 high scalability-2014-06-05-Cloud Architecture Revolution
16 0.77855068 1275 high scalability-2012-07-02-C is for Compute - Google Compute Engine (GCE)
17 0.77839202 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
18 0.77700198 517 high scalability-2009-02-21-Google AppEngine - A Second Look
19 0.77670401 1304 high scalability-2012-08-14-MemSQL Architecture - The Fast (MVCC, InMem, LockFree, CodeGen) and Familiar (SQL)
20 0.77649307 1479 high scalability-2013-06-21-Stuff The Internet Says On Scalability For June 21, 2013