high_scalability high_scalability-2007 high_scalability-2007-96 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is a wonderfully informative Amazon update based on Joachim Rohde's discovery of an interview with Amazon's CTO. You'll learn about how Amazon organizes their teams around services, the CAP theorem of building scalable systems, how they deploy software, and a lot more. Many new additions from the ACM Queue article have also been included. Amazon grew from a tiny online bookstore to one of the largest stores on earth. They did it while pioneering new and interesting ways to rate, review, and recommend products. Greg Linden shared is version of Amazon's birth pangs in a series of blog articles Site: http://amazon.com Information Sources Early Amazon by Greg Linden How Linux saved Amazon millions Interview Werner Vogels - Amazon's CTO Asynchronous Architectures - a nice summary of Werner Vogels' talk by Chris Loosley Learning from the Amazon technology platform - A Conversation with Werner Vogels Werner Vogels' Weblog - building scalable and robus
sentIndex sentText sentNum sentScore
1 You'll learn about how Amazon organizes their teams around services, the CAP theorem of building scalable systems, how they deploy software, and a lot more. [sent-2, score-0.316]
2 Between 100-150 services are accessed to build a page. [sent-9, score-0.265]
3 The databases were split into small parts and around each part and created a services interface that was the only way to access the data. [sent-19, score-0.365]
4 So are the applications that serve the Web-services interface, the customer service application, and the seller interface. [sent-27, score-0.386]
5 If you use a middleware package you get lock-in around the software patterns they have chosen. [sent-37, score-0.253]
6 They are assigned authority and empowered to solve a problem as a service in anyway they see fit. [sent-67, score-0.374]
7 This team built a separate service interface for that feature and they had authority to do what they needed. [sent-69, score-0.307]
8 - Force developers to focus on value delivered to the customer instead of building technology first and then figuring how to use it. [sent-80, score-0.483]
9 - Consistency: write a value and then you read the value you get the same value back. [sent-94, score-0.339]
10 Errors are hidden from the customer and sorted out later. [sent-103, score-0.321]
11 - When a customer submits an order you favor consistency because several services--credit card processing, shipping and handling, reporting--are simultaneously accessing the data. [sent-104, score-0.312]
12 A service oriented architecture allows the creation of a parallel and isolated development process that scales feature development to match your growth. [sent-115, score-0.265]
13 Keep things simple by making sure there are no hidden requirements and hidden dependencies in the design. [sent-118, score-0.288]
14 Create an infrastructure that allows services to be built very fast. [sent-124, score-0.261]
15 com available through a Web services interface to any developer in the world free of charge has also been a major success because it has driven so much innovation that they couldn't have thought of or built on their own. [sent-189, score-0.335]
16 Support an environment around the service development that never gets in the way of the development itself. [sent-196, score-0.366]
17 Developers should spend some time with customer service every two years. [sent-201, score-0.386]
18 Their they'll actually listen to customer service calls, answer customer service e-mails, and really understand the impact of the kinds of things they do as technologists. [sent-202, score-0.843]
19 They can build very complex applications out of primitive services that are by themselves relatively simple. [sent-207, score-0.265]
20 They can scale their operation independently, maintain unparalleled system availability, and introduce new services quickly without the need for massive reconfiguration. [sent-208, score-0.315]
wordName wordTfidf (topN-words)
[('customer', 0.243), ('amazon', 0.213), ('services', 0.175), ('service', 0.143), ('soap', 0.121), ('obedience', 0.115), ('wsdl', 0.115), ('value', 0.113), ('internally', 0.105), ('around', 0.101), ('shared', 0.096), ('solve', 0.094), ('werner', 0.093), ('build', 0.09), ('interface', 0.089), ('units', 0.088), ('teams', 0.088), ('independently', 0.087), ('middleware', 0.087), ('infrastructure', 0.086), ('checkout', 0.079), ('hidden', 0.078), ('properties', 0.075), ('authority', 0.075), ('system', 0.074), ('must', 0.074), ('book', 0.073), ('work', 0.072), ('innovation', 0.071), ('things', 0.071), ('rest', 0.07), ('backwards', 0.07), ('consistency', 0.069), ('scale', 0.066), ('embrace', 0.066), ('theorem', 0.065), ('vogels', 0.065), ('use', 0.065), ('measurement', 0.064), ('decentralized', 0.064), ('availability', 0.064), ('web', 0.064), ('organized', 0.062), ('building', 0.062), ('anyway', 0.062), ('dependencies', 0.061), ('development', 0.061), ('want', 0.06), ('books', 0.06), ('three', 0.059)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000006 96 high scalability-2007-09-18-Amazon Architecture
Introduction: This is a wonderfully informative Amazon update based on Joachim Rohde's discovery of an interview with Amazon's CTO. You'll learn about how Amazon organizes their teams around services, the CAP theorem of building scalable systems, how they deploy software, and a lot more. Many new additions from the ACM Queue article have also been included. Amazon grew from a tiny online bookstore to one of the largest stores on earth. They did it while pioneering new and interesting ways to rate, review, and recommend products. Greg Linden shared is version of Amazon's birth pangs in a series of blog articles Site: http://amazon.com Information Sources Early Amazon by Greg Linden How Linux saved Amazon millions Interview Werner Vogels - Amazon's CTO Asynchronous Architectures - a nice summary of Werner Vogels' talk by Chris Loosley Learning from the Amazon technology platform - A Conversation with Werner Vogels Werner Vogels' Weblog - building scalable and robus
2 0.31785831 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
3 0.30602852 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
Introduction: Can you really create an infinitely scalable infrastructure for less than $100 using Amazon's storage, grid, and queuing services platform? It appears so, at least for the right application. Amazon beams a spot light on the future battle of the roll-your-own versus the connect-the-dots approach to building next generation websites using core external services. Their argument is strong. Using Amazon's platform you can quickly build an infrastructure that would otherwise take an eternity to make, a pile of money to create, and an unbounded mass of people to implement and maintain. Yet Amazon doesn't provide SLAs, so you can you really trust them with your crown jewels? Facebook recently leap frogged Amazon's vision with an even more comprehensive set of services. The battle for the future is on. Site: http://aws.amazon.com/ Information Sources Slides: Building Highly Scalable Web Applications Podcast: Technometria: Amazon Web Services Amazon Services Home . Platform
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
6 0.20925339 480 high scalability-2008-12-30-Scalability Perspectives #5: Werner Vogels – The Amazon Technology Platform
7 0.19935867 1508 high scalability-2013-08-28-Sean Hull's 20 Biggest Bottlenecks that Reduce and Slow Down Scalability
8 0.19849499 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
9 0.19578692 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
12 0.18894687 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
17 0.17797391 1037 high scalability-2011-05-10-Viddler Architecture - 7 Million Embeds a Day and 1500 Req-Sec Peak
18 0.17783178 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
19 0.17461435 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
topicId topicWeight
[(0, 0.401), (1, 0.105), (2, 0.022), (3, 0.069), (4, 0.044), (5, -0.125), (6, 0.013), (7, -0.1), (8, -0.017), (9, -0.037), (10, -0.009), (11, 0.088), (12, -0.034), (13, -0.06), (14, 0.057), (15, -0.038), (16, 0.018), (17, -0.022), (18, 0.047), (19, -0.014), (20, 0.05), (21, 0.005), (22, 0.034), (23, 0.021), (24, -0.049), (25, -0.022), (26, 0.006), (27, 0.02), (28, 0.061), (29, 0.046), (30, -0.008), (31, 0.016), (32, 0.063), (33, -0.072), (34, -0.016), (35, -0.047), (36, -0.029), (37, 0.048), (38, 0.052), (39, 0.004), (40, -0.035), (41, -0.046), (42, 0.025), (43, -0.04), (44, -0.008), (45, 0.005), (46, -0.016), (47, -0.066), (48, -0.008), (49, -0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.99080896 96 high scalability-2007-09-18-Amazon Architecture
Introduction: This is a wonderfully informative Amazon update based on Joachim Rohde's discovery of an interview with Amazon's CTO. You'll learn about how Amazon organizes their teams around services, the CAP theorem of building scalable systems, how they deploy software, and a lot more. Many new additions from the ACM Queue article have also been included. Amazon grew from a tiny online bookstore to one of the largest stores on earth. They did it while pioneering new and interesting ways to rate, review, and recommend products. Greg Linden shared is version of Amazon's birth pangs in a series of blog articles Site: http://amazon.com Information Sources Early Amazon by Greg Linden How Linux saved Amazon millions Interview Werner Vogels - Amazon's CTO Asynchronous Architectures - a nice summary of Werner Vogels' talk by Chris Loosley Learning from the Amazon technology platform - A Conversation with Werner Vogels Werner Vogels' Weblog - building scalable and robus
2 0.89805967 1628 high scalability-2014-04-08-Microservices - Not a free lunch!
Introduction: This is a guest post by Benjamin Wootton , CTO of Contino , a London based consultancy specialising in applying DevOps and Continuous Delivery to software delivery projects. Microservices are a style of software architecture that involves delivering systems as a set of very small, granular, independent collaborating services. Though they aren't a particularly new idea, Microservices seem to have exploded in popularity this year, with articles, conference tracks, and Twitter streams waxing lyrical about the benefits of building software systems in this style. This popularity is partly off the back of trends such as Cloud, DevOps and Continuous Delivery coming together as enablers for this kind of approach, and partly off the back of great work at companies such as Netflix who have very visibly applied the pattern to great effect. Let me say up front that I am a fan of the approach. Microservices architectures have lots of very real and significant benefits: The service
3 0.88021439 139 high scalability-2007-10-30-Paper: Dynamo: Amazon’s Highly Available Key-value Store
Introduction: Update 2 : Read/WriteWeb has a good article talking about the scalability issues of relational databases and how Dynamo solves them: Amazon Dynamo: The Next Generation Of Virtual Distributed Storage . But since Dynamo is just another frustrating walled garden protected by barbed wire and guard dogs, its relevance is somewhat overstated. Update : Greg Linden has a take on the paper where he questions some of Amazon's design choices: emphasizing write availability over fast reads, a lack of indexing support, use of random distribution for load balancing, and punting on some scalability issues. Werner Vogels, Amazon's avuncular CTO, just announced a new paper on the internal database technology Amazon uses to handle tens of millions customers. I'll dive into more details later, but I thought you'd want to read it hot off the blog. The bad news is it won't be a service. They are keeping this tech not so secret, but very safe. Happily, it's another real-life example to learn from.
4 0.86177659 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
5 0.83221227 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
Introduction: Can you really create an infinitely scalable infrastructure for less than $100 using Amazon's storage, grid, and queuing services platform? It appears so, at least for the right application. Amazon beams a spot light on the future battle of the roll-your-own versus the connect-the-dots approach to building next generation websites using core external services. Their argument is strong. Using Amazon's platform you can quickly build an infrastructure that would otherwise take an eternity to make, a pile of money to create, and an unbounded mass of people to implement and maintain. Yet Amazon doesn't provide SLAs, so you can you really trust them with your crown jewels? Facebook recently leap frogged Amazon's vision with an even more comprehensive set of services. The battle for the future is on. Site: http://aws.amazon.com/ Information Sources Slides: Building Highly Scalable Web Applications Podcast: Technometria: Amazon Web Services Amazon Services Home . Platform
6 0.82732075 1090 high scalability-2011-08-01-Peecho Architecture - scalability on a shoestring
7 0.8248688 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
8 0.8179881 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
10 0.80915803 1431 high scalability-2013-03-29-Stuff The Internet Says On Scalability For March 29, 2013
11 0.80753565 674 high scalability-2009-08-07-The Canonical Cloud Architecture
12 0.80444342 1271 high scalability-2012-06-25-StubHub Architecture: The Surprising Complexity Behind the World’s Largest Ticket Marketplace
13 0.80353826 1030 high scalability-2011-04-27-Heroku Emergency Strategy: Incident Command System and 8 Hour Ops Rotations for Fresh Minds
14 0.80050105 1466 high scalability-2013-05-29-Amazon: Creating a Customer Utopia One Culture Hack at a Time
15 0.79947472 1634 high scalability-2014-04-18-Stuff The Internet Says On Scalability For April 18th, 2014
16 0.79350513 1014 high scalability-2011-03-31-8 Lessons We Can Learn from the MySpace Incident - Balance, Vision, Fearlessness
17 0.79267162 1277 high scalability-2012-07-05-10 Golden Principles For Building Successful Mobile-Web Applications
18 0.79167819 1368 high scalability-2012-12-07-Stuff The Internet Says On Scalability For December 7, 2012
19 0.79071128 1206 high scalability-2012-03-09-Stuff The Internet Says On Scalability For March 9, 2012
topicId topicWeight
[(1, 0.189), (2, 0.25), (5, 0.013), (10, 0.037), (27, 0.012), (30, 0.021), (55, 0.063), (61, 0.101), (77, 0.025), (79, 0.117), (85, 0.046), (94, 0.034)]
simIndex simValue blogId blogTitle
Introduction: If you are Google you don't just complain about performance on the web, you do something about it. Doing something about web performance is the job of one Ilya Grigorik , Developer Advocate, Make the Web Fast at Google, and author of a great new book: High Performance Browser Networking: What every web developer should know about networking and web performance . That's a big topic you might be saying to yourself. And it is. The book is 400 plus information packed pages. But never fear. Ilya writes in a very straightforward style. It’s like a man page for the web. Which is a good thing. In case you are not familiar with Ilya, he's the perfect choice for writing such an ambitious book. For years Ilya has been producing excellent content on his blog and if you search YouTube you'll find presentation after presentation on the topics found in the book. Authority established. Reading the book I was struck by what a complicated beast or little World Wide Web has become. That's
Introduction: Google has released an epic second edition of their ground breaking The Datacenter as a Computer book. It's called an introduction, but at 156 pages I would love to see what the Advanced version would look like! John Fries in a G+ comment has what I think is a perfect summary of the ultimate sense of the book: It's funny, when I was at Google I was initially quite intimidated by interacting with an enormous datacenter, and then I started imagining the entire datacenter was shrunk down into a small box sitting on my desk, and realized it was just another machine and the physical size didn't matter anymore It's such a far ranging book that it's impossible to characterize simply. It covers an amazing diversity of topics, from an introduction to warehouse-scale computing; workloads and software infrastructure; hardware; datacenter architecture; energy and power efficiency; cost structures; how to deal with failures and repairs; and it closes with a discussion of key challenge
3 0.98015594 1008 high scalability-2011-03-22-Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
Introduction: Facebook did it again. They've built another system capable of doing something useful with ginormous streams of realtime data. Last time we saw Facebook release their New Real-Time Messaging System: HBase To Store 135+ Billion Messages A Month . This time it's a realtime analytics system handling over 20 billion events per day (200,000 events per second) with a lag of less than 30 seconds . Alex Himel, Engineering Manager at Facebook, explains what they've built ( video ) and the scale required: Social plugins have become an important and growing source of traffic for millions of websites over the past year. We released a new version of Insights for Websites last week to give site owners better analytics on how people interact with their content and to help them optimize their websites in real time. To accomplish this, we had to engineer a system that could process over 20 billion events per day (200,000 events per second) with a lag of less than 30 seconds. Alex does a
4 0.97976708 1606 high scalability-2014-03-05-10 Things You Should Know About Running MongoDB at Scale
Introduction: Guest post by Asya Kamsky , Principal Solutions Architect at MongoDB. This post outlines ten things you need to know for operating MongoDB at scale based on my experience working with MongoDB customers and open source users: MongoDB requires DevOps, too. MongoDB is a database. Like any other data store, it requires capacity planning, tuning, monitoring, and maintenance. Just because it's easy to install and get started and it fits the developer paradigm more naturally than a relational database, don't assume that MongoDB doesn't need proper care and feeding. And just because it performs super-fast on a small sample dataset in development doesn't mean you can get away without having a good schema and indexing strategy, as well as the right hardware resources in production! But if you prepare well and understand the best practices, operating large MongoDB clusters can be boring instead of nerve-wracking. Successful MongoDB users monitor everything and prepare for growth.
5 0.97932053 808 high scalability-2010-04-12-Poppen.de Architecture
Introduction: This is a guest a post by Alvaro Videla describing their architecture for Poppen.de , a popular German dating site. This site is very much NSFW, so be careful before clicking on the link. What I found most interesting is how they manage to sucessfully blend a little of the old with a little of the new, using technologies like Nginx, MySQL, CouchDB, and Erlang, Memcached, RabbitMQ, PHP, Graphite, Red5, and Tsung. What is Poppen.de? Poppen.de (NSFW) is the top dating website in Germany, and while it may be a small site compared to giants like Flickr or Facebook, we believe it's a nice architecture to learn from if you are starting to get some scaling problems. The Stats 2.000.000 users 20.000 concurrent users 300.000 private messages per day 250.000 logins per day We have a team of eleven developers, two designers and two sysadmins for this project. Business Model The site works with a freemium model, where users can do for free things like: Search
8 0.97702372 160 high scalability-2007-11-19-Tailrank Architecture - Learn How to Track Memes Across the Entire Blogosphere
9 0.97698706 1499 high scalability-2013-08-09-Stuff The Internet Says On Scalability For August 9, 2013
10 0.97670197 671 high scalability-2009-08-05-Stack Overflow Architecture
11 0.97629672 106 high scalability-2007-10-02-Secrets to Fotolog's Scaling Success
12 0.97592056 1302 high scalability-2012-08-10-Stuff The Internet Says On Scalability For August 10, 2012
13 0.97543132 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
14 0.97517973 1064 high scalability-2011-06-20-35+ Use Cases for Choosing Your Next NoSQL Database
same-blog 15 0.97500533 96 high scalability-2007-09-18-Amazon Architecture
16 0.97481024 254 high scalability-2008-02-19-Hadoop Getting Closer to 1.0 Release
17 0.97477806 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
18 0.97467065 711 high scalability-2009-09-22-How Ravelry Scales to 10 Million Requests Using Rails
19 0.9744727 1011 high scalability-2011-03-25-Did the Microsoft Stack Kill MySpace?
20 0.97445244 1508 high scalability-2013-08-28-Sean Hull's 20 Biggest Bottlenecks that Reduce and Slow Down Scalability