high_scalability high_scalability-2009 high_scalability-2009-753 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud . The only independent platform most of us will have access to capable of hosting planet-scale applications is the Ambient Cloud. It forms a sort of digital potluck where everyone contributes memory, network, and other compute resources from whatever they happen to have available. Top 10 Internet Startup Scalability Killers . Strategies taken from The Art of Scalability . 1. Thinking Scalability Is Just About Technology; 2. Overuse of Synchronous Calls; 3. Failure to Weed or Seed Soon Enough; 4. Inappropriate Use of Databases; 5. Cesspools Instead of Swim Lanes; 6. Reliance on Vertical Scale; 7. Failure to Learn from History; 8. Changing Development Methodologies to Fix Problems; 9. Too Little Caching, Too Late; 10. Overreliance on Third Parties to Scale. The New Google: Internet Giant Opens Up About Real-Time and Local Search, Cloud Computing, and Data Liberation . In four separat
sentIndex sentText sentNum sentScore
1 It forms a sort of digital potluck where everyone contributes memory, network, and other compute resources from whatever they happen to have available. [sent-3, score-0.316]
2 In four separate interviews, Google delved into some of the most important topics of the day, from its advances in real-time and local search to cloud computing and a “data. [sent-18, score-0.208]
3 liberation” effort to help consumers export their files and digital information from Google products Ask HN: What are the best technologies you've worked with this year? [sent-20, score-0.199]
4 We have been amazed at how much code we cut out of Harmony with the switch from MySQL to Mongo. [sent-25, score-0.11]
5 ” The first three paradigms were experimental, theoretical and, more recently, computational science. [sent-29, score-0.177]
6 He explained this paradigm as an evolving era in which an “exaflood” of observational data was threatening to overwhelm scientists. [sent-30, score-0.348]
7 Using SSD reduced the headcount of their heavy load servers from 80 to 30. [sent-32, score-0.115]
8 Using Jaiku's reimplementation on Google App Engine is a good reference. [sent-36, score-0.122]
9 It seems to me that the responsibility of building an indexing and query mechanism lands on the NoSQL user. [sent-38, score-0.106]
10 There are some really neat software projects emerging at the moment, and as a developer I always find it interesting to take a look at the implementation details, because there is often a lot to be learned. [sent-40, score-0.1]
11 By the end of the day, his computers will have bought and sold about 60 million to 80 million shares . [sent-44, score-0.131]
wordName wordTfidf (topN-words)
[('transcendent', 0.27), ('clojure', 0.174), ('ambient', 0.161), ('shares', 0.131), ('era', 0.129), ('reimplementation', 0.122), ('honorable', 0.122), ('delved', 0.122), ('observational', 0.122), ('overreliance', 0.122), ('rails', 0.115), ('headcount', 0.115), ('swim', 0.115), ('fromthe', 0.115), ('amqp', 0.115), ('potluck', 0.115), ('overuse', 0.115), ('digital', 0.115), ('inappropriate', 0.11), ('amazed', 0.11), ('lands', 0.106), ('gray', 0.106), ('google', 0.103), ('deluge', 0.102), ('harmony', 0.102), ('lanes', 0.1), ('shapes', 0.1), ('disrupt', 0.1), ('neat', 0.1), ('hoped', 0.097), ('xmpp', 0.097), ('overwhelm', 0.097), ('replaces', 0.097), ('hn', 0.095), ('parties', 0.091), ('seed', 0.09), ('paradigms', 0.09), ('fourth', 0.088), ('jquery', 0.088), ('theoretical', 0.087), ('computing', 0.086), ('contributes', 0.086), ('methodologies', 0.086), ('export', 0.084), ('interviews', 0.084), ('cloudfront', 0.084), ('mongo', 0.084), ('mentions', 0.083), ('appengine', 0.083), ('runner', 0.083)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 753 high scalability-2009-12-21-Hot Holiday Scalability Links for 2009
Introduction: Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud . The only independent platform most of us will have access to capable of hosting planet-scale applications is the Ambient Cloud. It forms a sort of digital potluck where everyone contributes memory, network, and other compute resources from whatever they happen to have available. Top 10 Internet Startup Scalability Killers . Strategies taken from The Art of Scalability . 1. Thinking Scalability Is Just About Technology; 2. Overuse of Synchronous Calls; 3. Failure to Weed or Seed Soon Enough; 4. Inappropriate Use of Databases; 5. Cesspools Instead of Swim Lanes; 6. Reliance on Vertical Scale; 7. Failure to Learn from History; 8. Changing Development Methodologies to Fix Problems; 9. Too Little Caching, Too Late; 10. Overreliance on Third Parties to Scale. The New Google: Internet Giant Opens Up About Real-Time and Local Search, Cloud Computing, and Data Liberation . In four separat
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
4 0.17282833 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. The future looks many, big, complex, and adaptive: Many clouds. Many servers. Many operating systems. Many languages. Many storage services. Many database services. Many software services. Many adjunct human networks (like Mechanical Turk). Many fast interconnects. Many CDNs. Many cache memory pools. Many application profiles (simple request-response, live streaming, computationally complex, sensor driven, memory intensive, storage intensive, monolithic, decomposable, etc). Many legal jurisdictions. Don't want to perform a function on Patriot Act "protected" systems then move the function elsewhere. Many SLAs. Many data driven pricing policies that like airplane pricing algorithms will price "seats" to maximize profit using multi-variate time sensitive pricing models. Many competitive products. The need t
5 0.14382701 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. We tend to think compute of resources as residing primarily in datacenters. Given the fast pace of innovation we will likely see compute resources become pervasive. Some will reside in datacenters, but compute resources can be anywhere, not just in the datacenter, we'll actually see the bulk of compute resources live outside of datacenters in the future. Given the diversity of compute resources it's reasonable to assume they won't be homogeneous or conform to a standard API. They will specialize by service. Programmers will have to use those specialized service interfaces to build applications that are adaptive enough to take advantage of whatever leverage they can find, whenever and wherever they can find it. Once found the application will have to reorganize on the fly to use whatever new resources it has found and let go of whatever resources it doe
6 0.12931156 768 high scalability-2010-02-01-What Will Kill the Cloud?
7 0.12250391 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
8 0.11895707 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
9 0.11781593 301 high scalability-2008-04-08-Google AppEngine - A First Look
10 0.10803782 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
11 0.10417011 1389 high scalability-2013-01-18-Stuff The Internet Says On Scalability For January 18, 2013
12 0.10281459 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
13 0.10145588 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
14 0.1000993 459 high scalability-2008-12-03-Java World Interview on Scalability and Other Java Scalability Secrets
15 0.098717302 758 high scalability-2010-01-11-Have We Reached the End of Scaling?
16 0.097892202 823 high scalability-2010-05-05-How will memristors change everything?
17 0.095232375 679 high scalability-2009-08-11-13 Scalability Best Practices
18 0.092748016 517 high scalability-2009-02-21-Google AppEngine - A Second Look
19 0.087508008 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
20 0.087151624 803 high scalability-2010-04-05-Intercloud: How Will We Scale Across Multiple Clouds?
topicId topicWeight
[(0, 0.186), (1, 0.074), (2, 0.048), (3, 0.073), (4, -0.028), (5, -0.035), (6, -0.075), (7, -0.005), (8, 0.021), (9, 0.045), (10, -0.02), (11, -0.058), (12, -0.005), (13, 0.058), (14, -0.018), (15, -0.017), (16, -0.065), (17, -0.044), (18, 0.024), (19, 0.015), (20, -0.027), (21, 0.013), (22, -0.039), (23, 0.006), (24, 0.074), (25, 0.001), (26, -0.029), (27, 0.019), (28, -0.046), (29, -0.012), (30, -0.004), (31, -0.026), (32, -0.012), (33, -0.008), (34, 0.016), (35, -0.054), (36, -0.063), (37, -0.004), (38, -0.053), (39, 0.033), (40, 0.067), (41, -0.05), (42, -0.011), (43, 0.047), (44, 0.011), (45, -0.024), (46, 0.007), (47, 0.079), (48, -0.012), (49, -0.019)]
simIndex simValue blogId blogTitle
same-blog 1 0.95178419 753 high scalability-2009-12-21-Hot Holiday Scalability Links for 2009
Introduction: Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud . The only independent platform most of us will have access to capable of hosting planet-scale applications is the Ambient Cloud. It forms a sort of digital potluck where everyone contributes memory, network, and other compute resources from whatever they happen to have available. Top 10 Internet Startup Scalability Killers . Strategies taken from The Art of Scalability . 1. Thinking Scalability Is Just About Technology; 2. Overuse of Synchronous Calls; 3. Failure to Weed or Seed Soon Enough; 4. Inappropriate Use of Databases; 5. Cesspools Instead of Swim Lanes; 6. Reliance on Vertical Scale; 7. Failure to Learn from History; 8. Changing Development Methodologies to Fix Problems; 9. Too Little Caching, Too Late; 10. Overreliance on Third Parties to Scale. The New Google: Internet Giant Opens Up About Real-Time and Local Search, Cloud Computing, and Data Liberation . In four separat
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
4 0.7131176 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. There's a pattern, already begun, that has accelerated by the need for applications to scale and increase complexity, the end result of which will be that applications give up their independence and enter a kind of feudal relationship with their platform provider. To understand how this process works, like a glacier slowly and inevitably carving out a deep river valley, here's the type of question I get quite a lot: I've learned PHP and MySQL and I've built a web app that I HOPE will receive traffic comparable to eBay's with a similar database structure. I see all these different opinions and different techniques and languages being recommended and it's so confusing. All I want is perhaps one book or one website that focuses on PHP and MySQL and building a large database web app like eBay's. Does something like this exist? I'm always at a loss fo
5 0.70724177 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. We tend to think compute of resources as residing primarily in datacenters. Given the fast pace of innovation we will likely see compute resources become pervasive. Some will reside in datacenters, but compute resources can be anywhere, not just in the datacenter, we'll actually see the bulk of compute resources live outside of datacenters in the future. Given the diversity of compute resources it's reasonable to assume they won't be homogeneous or conform to a standard API. They will specialize by service. Programmers will have to use those specialized service interfaces to build applications that are adaptive enough to take advantage of whatever leverage they can find, whenever and wherever they can find it. Once found the application will have to reorganize on the fly to use whatever new resources it has found and let go of whatever resources it doe
6 0.70381039 768 high scalability-2010-02-01-What Will Kill the Cloud?
7 0.69543946 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
8 0.68920767 439 high scalability-2008-11-10-Scalability Perspectives #1: Nicholas Carr – The Big Switch
9 0.68608773 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
10 0.68347365 1057 high scalability-2011-06-10-Stuff The Internet Says On Scalability For June 10, 2011
11 0.68265569 1084 high scalability-2011-07-22-Stuff The Internet Says On Scalability For July 22, 2011
12 0.68080348 1182 high scalability-2012-01-27-Stuff The Internet Says On Scalability For January 27, 2012
13 0.67664516 1045 high scalability-2011-05-20-Stuff The Internet Says On Scalability For May 20, 2011
14 0.67528319 1147 high scalability-2011-11-25-Stuff The Internet Says On Scalability For November 25, 2011
15 0.66686678 1117 high scalability-2011-09-16-Stuff The Internet Says On Scalability For September 16, 2011
16 0.66645581 984 high scalability-2011-02-04-Stuff The Internet Says On Scalability For February 4, 2011
17 0.6608482 999 high scalability-2011-03-04-Stuff The Internet Says On Scalability For March 4, 2011
18 0.66063708 1019 high scalability-2011-04-08-Stuff The Internet Says On Scalability For April 8, 2011
19 0.65875155 1315 high scalability-2012-08-30-Stuff The Internet Says On Scalability For August 31, 2012
topicId topicWeight
[(1, 0.109), (2, 0.135), (10, 0.037), (47, 0.024), (61, 0.11), (77, 0.342), (79, 0.137), (94, 0.03)]
simIndex simValue blogId blogTitle
Introduction: James Hamilton in Counting Servers is Hard has an awesome breakdown of what one million plus servers really means in terms of resource usage. The summary from his calculations are eye popping: Facilities: 15 to 30 large datacenters Capital expense: $4.25 Billion Total power: 300MW Power Consumption: 2.6TWh annually The power consumption is about the same as used by Nicaragua and the capital cost is about a third of what Americans spent on video games in 2012. Now that's web scale.
same-blog 2 0.89209282 753 high scalability-2009-12-21-Hot Holiday Scalability Links for 2009
Introduction: Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud . The only independent platform most of us will have access to capable of hosting planet-scale applications is the Ambient Cloud. It forms a sort of digital potluck where everyone contributes memory, network, and other compute resources from whatever they happen to have available. Top 10 Internet Startup Scalability Killers . Strategies taken from The Art of Scalability . 1. Thinking Scalability Is Just About Technology; 2. Overuse of Synchronous Calls; 3. Failure to Weed or Seed Soon Enough; 4. Inappropriate Use of Databases; 5. Cesspools Instead of Swim Lanes; 6. Reliance on Vertical Scale; 7. Failure to Learn from History; 8. Changing Development Methodologies to Fix Problems; 9. Too Little Caching, Too Late; 10. Overreliance on Third Parties to Scale. The New Google: Internet Giant Opens Up About Real-Time and Local Search, Cloud Computing, and Data Liberation . In four separat
Introduction: Successful software design is all about trade-offs. In the typical (if there is such a thing) distributed system, recognizing the importance of trade-offs within the design of your architecture is integral to the success of your system. Despite this reality, I see time and time again, developers choosing a particular solution based on an ill-placed belief in their solution as a “silver bullet”, or a solution that conquers all, despite the inevitable occurrence of changing requirements. Regardless of the reasons behind this phenomenon, I’d like to outline a few of the methods I use to ensure that I’m making good scalable decisions without losing sight of the trade-offs that accompany them. I’d also like to compile (pun intended) the issues at hand, by formulating a simple theorem that we can use to describe this oft occurring situation.
4 0.87130892 258 high scalability-2008-02-24-Yandex Architecture
Introduction: Update: Anatomy of a crash in a new part of Yandex written in Django . Writing to a magic session variable caused an unexpected write into an InnoDB database on every request. Writes took 6-7 seconds because of index rebuilding. Lots of useful details on the sizing of their system, what went wrong, and how they fixed it. Yandex is a Russian search engine with 3.5 billion pages in their search index. We only know a few fun facts about how they do things, nothing at a detailed architecture level. Hopefully we'll learn more later, but I thought it would still be interesting. From Allen Stern's interview with Yandex's CTO Ilya Segalovich, we learn: 3.5 billion pages in the search index. Over several thousand servers. 35 million searches a day. Several data centers around Russia. Two-layer architecture. The database is split in pieces and when a search is requested, it pulls the bits from the different database servers and brings it together for the user. Languages
5 0.86682284 1116 high scalability-2011-09-15-Paper: It's Time for Low Latency - Inventing the 1 Microsecond Datacenter
Introduction: In It's Time for Low Latency Stephen Rumble et al. explore the idea that it's time to rearchitect our stack to live in the modern era of low-latency datacenter instead of high-latency WANs. The implications for program architectures will be revolutionary . Luiz André Barroso , Distinguished Engineer at Google, sees ultra low latency as a way to make computer resources, to be as much as possible, fungible, that is they are interchangeable and location independent, effectively turning a datacenter into single computer. Abstract from the paper: The operating systems community has ignored network latency for too long. In the past, speed-of-light delays in wide area networks and unoptimized network hardware have made sub-100µs round-trip times impossible. However, in the next few years datacenters will be deployed with low-latency Ethernet. Without the burden of propagation delays in the datacenter campus and network delays in the Ethernet devices, it will be up to us to finish
6 0.86303979 1195 high scalability-2012-02-17-Stuff The Internet Says On Scalability For February 17, 2012
7 0.85535759 211 high scalability-2008-01-13-Google Reveals New MapReduce Stats
8 0.85418028 525 high scalability-2009-03-05-Product: Amazon Simple Storage Service
9 0.84043586 959 high scalability-2010-12-17-Stuff the Internet Says on Scalability For December 17th, 2010
10 0.84038824 439 high scalability-2008-11-10-Scalability Perspectives #1: Nicholas Carr – The Big Switch
11 0.83595377 766 high scalability-2010-01-26-Product: HyperGraphDB - A Graph Database
12 0.82870096 1531 high scalability-2013-10-13-AIDA: Badoo’s journey into Continuous Integration
13 0.81970769 1059 high scalability-2011-06-14-A TripAdvisor Short
14 0.80926186 1377 high scalability-2012-12-26-Ask HS: What will programming and architecture look like in 2020?
15 0.77734649 222 high scalability-2008-01-25-Application Database and DAL Architecture
16 0.7650528 1158 high scalability-2011-12-16-Stuff The Internet Says On Scalability For December 16, 2011
17 0.76060873 212 high scalability-2008-01-14-OpenSpaces.org community site launched - framework for building scale-out applications
18 0.75319701 200 high scalability-2008-01-02-WEB hosting Select
19 0.74046874 1188 high scalability-2012-02-06-The Design of 99designs - A Clean Tens of Millions Pageviews Architecture
20 0.73027509 1567 high scalability-2013-12-20-Stuff The Internet Says On Scalability For December 20th, 2013