high_scalability high_scalability-2012 high_scalability-2012-1240 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
sentIndex sentText sentNum sentScore
1 It’s Not All About the Cloud, but It’s Mostly About the Cloud These days the story of startups primarily revolves around the cloud in one way or another. [sent-21, score-0.47]
2 Not all startups choose the cloud, many do not , but even if a startup doesn’t join a formal cloud, we still see the development of cloud-like infrastructures and the deployment of cloud inspired tool chains. [sent-26, score-0.589]
3 So we’ll just skip all the old arguments about OpEx vs CapEx, IaaS vs PaaS vs SaaS, virtualization vs bare metal, public vs private vs hybrid clouds, and open vs closed clouds. [sent-27, score-0.592]
4 By cloud I do not mean any particular cloud provider or technology. [sent-30, score-0.372]
5 Here are a few examples from startups of how pretty much everything has changed: Instagram : Give me a place to stand and with a lever I will move the whole world . [sent-72, score-0.385]
6 What makes it possible is the leverage gained by an IT programming fabric that treats a datacenter and its contained services as being software scriptable. [sent-133, score-0.437]
7 Here are a few examples of how startups are making use of these new capabilities to innovate: Robert Scoble talks about how the flood of new startups is just starting . [sent-141, score-0.606]
8 Each of these teams has their own business objectives, and each team is able to, and responsible for, all aspects of their business. [sent-149, score-0.396]
9 Netflix : "We built a completely cloud based infrastructure in the US and did some work extracting it so we could actually deploy it anywhere. [sent-152, score-0.403]
10 ” Playfish : The cloud allows Playfish to innovate and try new features and new game with very low friction, which is key in a fast moving market. [sent-156, score-0.416]
11 By unshackling developers from IT infrastructure people it opens up the possibility space and developers can do new things they could never do before. [sent-164, score-0.393]
12 As we’ll see, services are not just a software architecture feature anymore, but they’ve become the organizing principle around how teams and software are constructed. [sent-192, score-0.531]
13 Amazon : “If you think about infrastructure as a service and platform as a service (PaaS), what we've built is a PaaS over the top of the AWS infrastructure, which is as thin a layer as we could build, leveraging as many Amazon features as seemed interesting and useful. [sent-223, score-0.448]
14 ” Netflix : “If you think about infrastructure as a service and platform as a service (PaaS), what we've built is a PaaS over the top of the AWS infrastructure, which is as thin a layer as we could build, leveraging as many Amazon features as seemed interesting and useful. [sent-225, score-0.448]
15 Many small teams of 3-5 person teams are completely responsible for their service: development, support, deployment. [sent-228, score-0.515]
16 Even if you aren't in a public cloud it's likely you'll conceptualize your architecture in this way because that's how the infrastructure tools will be patterned. [sent-277, score-0.383]
17 Here are a few examples of startups using firehose architectures: Tumblr : Internally applications need access to the activity stream of information about users creating/deleting posts, liking/unliking posts, etc. [sent-282, score-0.482]
18 The firehose model is very flexible, not like Twitter’s firehose in which data is assumed to be lost. [sent-288, score-0.408]
19 Before firehose style architectures the easiest path was to create monolithic applications because information was accessible only in one place. [sent-299, score-0.377]
20 Users are homed to a cell and all cells consume all posts via firehose updates. [sent-316, score-0.406]
wordName wordTfidf (topN-words)
[('startups', 0.218), ('firehose', 0.204), ('cloud', 0.186), ('teams', 0.165), ('datacenter', 0.148), ('cell', 0.142), ('services', 0.137), ('developers', 0.117), ('architectures', 0.113), ('service', 0.105), ('world', 0.104), ('business', 0.096), ('startup', 0.094), ('paas', 0.092), ('source', 0.092), ('development', 0.091), ('playfish', 0.091), ('open', 0.088), ('netflix', 0.087), ('software', 0.086), ('new', 0.085), ('thin', 0.083), ('built', 0.081), ('team', 0.078), ('responsibility', 0.076), ('infrastructure', 0.074), ('message', 0.074), ('vs', 0.072), ('independent', 0.07), ('zynga', 0.068), ('pod', 0.066), ('way', 0.066), ('programming', 0.066), ('small', 0.066), ('oriented', 0.064), ('innovation', 0.063), ('everything', 0.063), ('completely', 0.062), ('system', 0.062), ('applications', 0.06), ('innovate', 0.06), ('kafka', 0.06), ('cells', 0.06), ('amazon', 0.06), ('sophisticated', 0.057), ('code', 0.057), ('responsible', 0.057), ('idea', 0.057), ('architecture', 0.057), ('created', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999875 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
2 0.36278656 1242 high scalability-2012-05-09-Cell Architectures
Introduction: A consequence of Service Oriented Architectures is the burning need to provide services at scale. The architecture that has evolved to satisfy these requirements is a little known technique called the Cell Architecture. A Cell Architecture is based on the idea that massive scale requires parallelization and parallelization requires components be isolated from each other. These islands of isolation are called cells. A cell is a self-contained installation that can satisfy all the operations for a shard . A shard is a subset of a much larger dataset, typically a range of users, for example. Cell Architectures have several advantages: Cells provide a unit of parallelization that can be adjusted to any size as the user base grows. Cell are added in an incremental fashion as more capacity is required. Cells isolate failures. One cell failure does not impact other cells. Cells provide isolation as the storage and application horsepower to process requests is independent of othe
3 0.35844985 1461 high scalability-2013-05-20-The Tumblr Architecture Yahoo Bought for a Cool Billion Dollars
Introduction: It's being reportedYahoo bought Tumblr for $1.1 billion. You may recallInstagram was profiled on HighScalabilityand they were also bought by Facebook for a ton of money. A coincidence? You be the judge.Just what is Yahoo buying? The business acumen of the deal is not something I can judge, but if you are doing due diligence on the technology then Tumblr would probably get a big thumbs up. To see why, please keep on reading...With over 15 billion page views a month Tumblr has become an insanely popular blogging platform. Users may like Tumblr for its simplicity, its beauty, its strong focus on user experience, or its friendly and engaged community, but like it they do.Growing at over 30% a month has not been without challenges. Some reliability problems among them. It helps to realize that Tumblr operates at surprisingly huge scales: 500 million page views a day, a peak rate of ~40k requests per second, ~3TB of new data to store a day, all running on 1000+ servers.One of the common patt
4 0.35823888 1191 high scalability-2012-02-13-Tumblr Architecture - 15 Billion Page Views a Month and Harder to Scale than Twitter
Introduction: With over 15 billion page views a month Tumblr has become an insanely popular blogging platform. Users may like Tumblr for its simplicity, its beauty, its strong focus on user experience, or its friendly and engaged community, but like it they do.Growing at over 30% a month has not been without challenges. Some reliability problems among them. It helps to realize that Tumblr operates at surprisingly huge scales: 500 million page views a day, a peak rate of ~40k requests per second, ~3TB of new data to store a day, all running on 1000+ servers.One of the common patterns across successful startups is the perilous chasm crossing from startup to wildly successful startup. Finding people, evolving infrastructures, servicing old infrastructures, while handling huge month over month increases in traffic, all with only four engineers, means you have to make difficult choices about what to work on. This was Tumblr's situation. Now with twenty engineers there's enough energy to work on issues an
Introduction: With over 15 billion page views a month Tumblr has become an insanely popular blogging platform. Users may like Tumblr for its simplicity, its beauty, its strong focus on user experience, or its friendly and engaged community, but like it they do.Growing at over 30% a month has not been without challenges. Some reliability problems among them. It helps to realize that Tumblr operates at surprisingly huge scales: 500 million page views a day, a peak rate of ~40k requests per second, ~3TB of new data to store a day, all running on 1000+ servers.One of the common patterns across successful startups is the perilous chasm crossing from startup to wildly successful startup. Finding people, evolving infrastructures, servicing old infrastructures, while handling huge month over month increases in traffic, all with only four engineers, means you have to make difficult choices about what to work on. This was Tumblr's situation. Now with twenty engineers there's enough energy to work on issues an
6 0.31785831 96 high scalability-2007-09-18-Amazon Architecture
9 0.26577228 1042 high scalability-2011-05-17-Facebook: An Example Canonical Architecture for Scaling Billions of Messages
10 0.26181957 1155 high scalability-2011-12-12-Netflix: Developing, Deploying, and Supporting Software According to the Way of the Cloud
11 0.25688401 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
12 0.24219623 904 high scalability-2010-09-21-Playfish's Social Gaming Architecture - 50 Million Monthly Users and Growing
13 0.2408561 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
14 0.22502276 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
17 0.21791933 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
19 0.21269114 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
20 0.21028614 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
topicId topicWeight
[(0, 0.468), (1, 0.112), (2, 0.092), (3, 0.124), (4, 0.017), (5, -0.186), (6, 0.012), (7, -0.104), (8, -0.011), (9, -0.029), (10, -0.001), (11, 0.216), (12, 0.023), (13, -0.002), (14, -0.042), (15, 0.06), (16, -0.072), (17, -0.073), (18, -0.018), (19, -0.058), (20, -0.056), (21, 0.026), (22, -0.097), (23, -0.038), (24, -0.052), (25, 0.065), (26, -0.025), (27, -0.018), (28, 0.002), (29, -0.003), (30, -0.04), (31, 0.098), (32, 0.036), (33, -0.075), (34, -0.012), (35, 0.076), (36, -0.041), (37, -0.006), (38, 0.055), (39, -0.013), (40, -0.016), (41, 0.016), (42, 0.002), (43, -0.072), (44, -0.034), (45, -0.016), (46, -0.01), (47, -0.058), (48, 0.042), (49, -0.052)]
simIndex simValue blogId blogTitle
same-blog 1 0.97035795 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
2 0.82416117 1628 high scalability-2014-04-08-Microservices - Not a free lunch!
Introduction: This is a guest post by Benjamin Wootton , CTO of Contino , a London based consultancy specialising in applying DevOps and Continuous Delivery to software delivery projects. Microservices are a style of software architecture that involves delivering systems as a set of very small, granular, independent collaborating services. Though they aren't a particularly new idea, Microservices seem to have exploded in popularity this year, with articles, conference tracks, and Twitter streams waxing lyrical about the benefits of building software systems in this style. This popularity is partly off the back of trends such as Cloud, DevOps and Continuous Delivery coming together as enablers for this kind of approach, and partly off the back of great work at companies such as Netflix who have very visibly applied the pattern to great effect. Let me say up front that I am a fan of the approach. Microservices architectures have lots of very real and significant benefits: The service
3 0.78444541 96 high scalability-2007-09-18-Amazon Architecture
Introduction: This is a wonderfully informative Amazon update based on Joachim Rohde's discovery of an interview with Amazon's CTO. You'll learn about how Amazon organizes their teams around services, the CAP theorem of building scalable systems, how they deploy software, and a lot more. Many new additions from the ACM Queue article have also been included. Amazon grew from a tiny online bookstore to one of the largest stores on earth. They did it while pioneering new and interesting ways to rate, review, and recommend products. Greg Linden shared is version of Amazon's birth pangs in a series of blog articles Site: http://amazon.com Information Sources Early Amazon by Greg Linden How Linux saved Amazon millions Interview Werner Vogels - Amazon's CTO Asynchronous Architectures - a nice summary of Werner Vogels' talk by Chris Loosley Learning from the Amazon technology platform - A Conversation with Werner Vogels Werner Vogels' Weblog - building scalable and robus
4 0.78170615 1636 high scalability-2014-04-23-Here's a 1300 Year Old Solution to Resilience - Rebuild, Rebuild, Rebuild
Introduction: How is it possible that a wooden Shinto shrine built in the 7th century is still standing? The answer depends on how you answer this philosophical head scratcher: With nearly every cell in your body continually being replaced, are you still the same person? The Ise Grand Shrine has been in continuous existence for over 1300 years because every twenty years an exact replica has been rebuilt on an adjacent footprint. The former temple is then dismantled. Now that's resilience. If you want something to last make it a living part of a culture. It's not so much the building that is remade, what is rebuilt and passed down from generation to generation is the meme that the shrine is important and worth preserving. The rest is an unfolding of that imperative. You can see echoes of this same process in Open Source projects like Linux and the libraries and frameworks that get themselves reconstructed in each new environment. The patterns of recurrence in software are the result of Darw
5 0.7739628 1522 high scalability-2013-09-25-Great Open Source Solution for Boring HA and Scalability Problems
Introduction: This is a guest post about how boring and repetitive HA and scalability problems can be solved via Open Source so you can focus on the interesting tasks. The post was written by Maarten Ectors , responsible for Cloud Strategy and Frank Mueller , a Juju Core developer, at Ubuntu / Canonical . High-availability and scalability are exciting in general but there are certain problems that experts see over and over again. The list is long but examples are setting up MySQL clustering, sharding Mongo, adding data nodes to a Hadoop cluster, monitoring with Ganglia, building continuous deployment solutions, integrating Memcached / Varnish / Nginx,… Why are we reinventing the wheel? At Ubuntu we made it our goal to have the community solve these repetitive and often boring tasks. How often have you had to set-up MySQL replication and scale it? What if the next time you just simply do: juju deploy mysql juju deploy mysql mysql-slave juju add-relation mysql:master mysql-slave:slav
6 0.76715624 1461 high scalability-2013-05-20-The Tumblr Architecture Yahoo Bought for a Cool Billion Dollars
7 0.76520479 1191 high scalability-2012-02-13-Tumblr Architecture - 15 Billion Page Views a Month and Harder to Scale than Twitter
9 0.75500804 1575 high scalability-2014-01-08-Under Snowden's Light Software Architecture Choices Become Murky
10 0.75190371 1042 high scalability-2011-05-17-Facebook: An Example Canonical Architecture for Scaling Billions of Messages
11 0.75115335 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
12 0.74855816 1630 high scalability-2014-04-11-Stuff The Internet Says On Scalability For April 11th, 2014
14 0.73822492 807 high scalability-2010-04-09-Vagrant - Build and Deploy Virtualized Development Environments Using Ruby
15 0.73392123 1155 high scalability-2011-12-12-Netflix: Developing, Deploying, and Supporting Software According to the Way of the Cloud
16 0.73121291 976 high scalability-2011-01-20-75% Chance of Scale - Leveraging the New Scaleogenic Environment for Growth
17 0.73021793 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
18 0.72911441 1242 high scalability-2012-05-09-Cell Architectures
19 0.72702414 1354 high scalability-2012-11-05-Are we seeing the renaissance of enterprises in the cloud?
20 0.7256031 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
topicId topicWeight
[(1, 0.157), (2, 0.21), (10, 0.043), (30, 0.018), (40, 0.019), (56, 0.012), (61, 0.089), (77, 0.017), (79, 0.241), (85, 0.033), (94, 0.05)]
simIndex simValue blogId blogTitle
1 0.99370939 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. The future looks many, big, complex, and adaptive: Many clouds. Many servers. Many operating systems. Many languages. Many storage services. Many database services. Many software services. Many adjunct human networks (like Mechanical Turk). Many fast interconnects. Many CDNs. Many cache memory pools. Many application profiles (simple request-response, live streaming, computationally complex, sensor driven, memory intensive, storage intensive, monolithic, decomposable, etc). Many legal jurisdictions. Don't want to perform a function on Patriot Act "protected" systems then move the function elsewhere. Many SLAs. Many data driven pricing policies that like airplane pricing algorithms will price "seats" to maximize profit using multi-variate time sensitive pricing models. Many competitive products. The need t
Introduction: This is a guest post by BugSense Founder/CTO Jon Vlachogiannis and Head of Infrastructure at BugSense Panagiotis Papadomitsos . There has been a lot of speculation and assumptions around whether PRISM exists and if it is cost effective. I don't know whether it exists or not, but I can tell you if it could be built. Short answer: It can. If you believe it would be impossible for someone with access to a social "datapool" to find out more about you (if they really want to track you down) in the tsunami of data, you need to think again. Devices, apps and websites are transmitting data. Lots of data. The questions are could the data compiled and searched and how costly would it be to search for your targeted data. (hint: It is not $4.56 trillion). Let's experiment and try to build PRISM by ourselves with a few assumptions: Assumption 1 : We have all the appropriate "data connectors" that will provide us with data. Assumption 2 : These connectors provide dire
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
5 0.9846701 763 high scalability-2010-01-22-How BuddyPoke Scales on Facebook Using Google App Engine
Introduction: How do you scale a viral Facebook app that has skyrocketed to a mind boggling 65 million installs (the population of France)? That's the fortunate problem BuddyPoke co-founder Dave Westwood has and he talked about his solution at Wednesday's Facebook Meetup . Slides for the complete talk are here . For those not quite sure what BuddyPoke is, it's a social network application that lets users show their mood, hug, kiss, and poke their friends through on-line avatars. In many ways BuddyPoke is the quintessentially modern web application. It thrives off the energy of social network driven ecosystems. Game play mechanics, viral loops, and creative monetization strategies are all part of if its everyday conceptualization. It mashes together different technologies, not in a dark Frankensteining sort of way, but in a smart way that gets the most bang for the buck. Part of it runs on Facebook servers (free). Part of it runs on flash in a browser (free). Part of it runs on a storage clou
6 0.98315132 1494 high scalability-2013-07-19-Stuff The Internet Says On Scalability For July 19, 2013
7 0.9824475 1420 high scalability-2013-03-08-Stuff The Internet Says On Scalability For March 8, 2013
8 0.98160309 448 high scalability-2008-11-22-Google Architecture
9 0.98130566 1630 high scalability-2014-04-11-Stuff The Internet Says On Scalability For April 11th, 2014
same-blog 10 0.98125786 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
11 0.98082268 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
12 0.98075497 380 high scalability-2008-09-05-Product: Tungsten Replicator
13 0.97728485 289 high scalability-2008-03-27-Amazon Announces Static IP Addresses and Multiple Datacenter Operation
14 0.97697258 1048 high scalability-2011-05-27-Stuff The Internet Says On Scalability For May 27, 2011
15 0.97687399 129 high scalability-2007-10-23-Hire Facebook, Ning, and Salesforce to Scale for You
16 0.9757821 803 high scalability-2010-04-05-Intercloud: How Will We Scale Across Multiple Clouds?
17 0.9751057 1403 high scalability-2013-02-08-Stuff The Internet Says On Scalability For February 8, 2013
18 0.97416198 1328 high scalability-2012-09-24-Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In
19 0.97305715 1275 high scalability-2012-07-02-C is for Compute - Google Compute Engine (GCE)
20 0.97201627 288 high scalability-2008-03-25-Paper: On Designing and Deploying Internet-Scale Services