high_scalability high_scalability-2010 high_scalability-2010-853 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ) on the differences between using a cloud infrastructure and building your own. Frédéric was kind enough to translate the original French version of this article into English. I’ve been noticing many questions about the differences inherent in choosing between a Cloud infrastructure such as AWS (Amazon Web Services) and a traditional physical infrastructure. Firstly, there are a certain number of preconceived notions on this subject that I will attempt to decode for you. Then, it must be understood that each infrastructure has its advantages and disadvantages: a Cloud-type infrastructure does not necessarily fulfill your requirements in every case, however, it can satisfy some of them by optimizing or facilitating the features offered by a traditional physical infrastructure. I will therefore demonstrate the differences between the two that I have noticed, in order to help you make up your own mind. The Fram
sentIndex sentText sentNum sentScore
1 This is a guest post by Frédéric Faure (architect at Ysance ) on the differences between using a cloud infrastructure and building your own. [sent-1, score-0.435]
2 I’ve been noticing many questions about the differences inherent in choosing between a Cloud infrastructure such as AWS (Amazon Web Services) and a traditional physical infrastructure. [sent-3, score-0.615]
3 In fact, regarding the latter, we can’t speak for clients (they are the ones holding the credit card) about infrastructure management, because we upload our application using the APIs provided and leave the entire infrastructure management to the service provider. [sent-8, score-0.773]
4 Physical infrastructure As far as physical infrastructure is concerned, I will examine the notion of self-hosted infrastructure and equally the notion of infrastructure supported by a hosting provider. [sent-11, score-1.291]
5 A person with experience in the field, able to set up the infrastructure requirement is needed: the tool may change but the skills must still be available. [sent-25, score-0.364]
6 Indeed, being able to pull up resources on the fly will enable infrastructure management to be scheduled and automated via scripts which will call up the APIs provided by Amazon enabling communication with its web services. [sent-31, score-0.483]
7 The systems administrator’s craft is therefore evolving between a physical infrastructure and an AWS-type Cloud infrastructure: he is becoming more and more of a developer. [sent-34, score-0.555]
8 As I mentioned earlier, one of the crucial differences between the two types of infrastructure is the flexibility and dynamism provided by the Cloud solution, compared to a traditional physical architecture (whether it is based on virtualization or not). [sent-37, score-0.741]
9 You also have complete access to certain elements such as security groups (firewall) set for each instance… And that’s very useful. [sent-42, score-0.398]
10 All the safety standards that need to be met in terms of fire protection, computer cooling, redundant electrical power supply, physical security against break-ins, distributing the hardware across 2 or more physical datacenters for disaster recovery, etc. [sent-46, score-0.65]
11 To easily calculate how much your infrastructure will cost you, take advantage of the new calculator provided by Amazon: Simple Monthly Calculator . [sent-57, score-0.345]
12 Simple Monthly Calculator You can do the comparison with the cost of your local infrastructure or with that of your hosting provider. [sent-59, score-0.359]
13 However, it must be acknowledged that often the cosy aspects of “home”-hosted infrastructure can lead to a certain lack of rigor on many issues. [sent-67, score-0.465]
14 Total visibility of our infrastructure is therefore not possible on Cloud. [sent-76, score-0.351]
15 Unlike the specific contracts you may sign with your hosting providers, you must provide for these elements yourself, or regarding on-call support for example, you could out-source it to a facilities management company. [sent-90, score-0.448]
16 Some traditional hosting providers offer packaged monitoring with their services. [sent-93, score-0.34]
17 The first key point is that the level of security supplied in Amazon datacenters, not just physically but – equally importantly – in programmatically terms, will still be streets ahead of your average corporate computer room, even the datacenters of the smallest hosting providers. [sent-104, score-0.575]
18 Firstly because that is Amazon’s business: a security problem revealed in their infrastructure would have immediate implications in terms of user reactions (and thus in terms of business). [sent-105, score-0.383]
19 Conclusion The evolving duties of infrastructure management can be clearly seen in this first part: from handling physical resources by means of APIs, underlying mechanisms ensuring data durability, availability of services, etc. [sent-119, score-0.635]
20 right up to server power supply and the physical security of datacenters, all of which are supported transparently. [sent-120, score-0.368]
wordName wordTfidf (topN-words)
[('aws', 0.227), ('infrastructure', 0.219), ('physical', 0.204), ('administrator', 0.175), ('amazon', 0.167), ('security', 0.164), ('ebs', 0.163), ('hosting', 0.14), ('bcp', 0.139), ('cloud', 0.138), ('therefore', 0.132), ('provided', 0.126), ('traditional', 0.114), ('instances', 0.105), ('penalties', 0.099), ('certain', 0.094), ('firstly', 0.093), ('must', 0.087), ('monitoring', 0.086), ('pay', 0.083), ('elements', 0.082), ('services', 0.08), ('datacenters', 0.078), ('differences', 0.078), ('instance', 0.077), ('however', 0.076), ('responsible', 0.076), ('apis', 0.074), ('mechanisms', 0.074), ('regarding', 0.071), ('equally', 0.071), ('provider', 0.07), ('resources', 0.07), ('service', 0.07), ('management', 0.068), ('durability', 0.067), ('transparency', 0.066), ('lack', 0.065), ('keys', 0.065), ('frederic', 0.064), ('offered', 0.062), ('supplied', 0.062), ('haas', 0.062), ('credentials', 0.06), ('smallest', 0.06), ('french', 0.059), ('logistics', 0.059), ('set', 0.058), ('price', 0.058), ('editor', 0.056)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ) on the differences between using a cloud infrastructure and building your own. Frédéric was kind enough to translate the original French version of this article into English. I’ve been noticing many questions about the differences inherent in choosing between a Cloud infrastructure such as AWS (Amazon Web Services) and a traditional physical infrastructure. Firstly, there are a certain number of preconceived notions on this subject that I will attempt to decode for you. Then, it must be understood that each infrastructure has its advantages and disadvantages: a Cloud-type infrastructure does not necessarily fulfill your requirements in every case, however, it can satisfy some of them by optimizing or facilitating the features offered by a traditional physical infrastructure. I will therefore demonstrate the differences between the two that I have noticed, in order to help you make up your own mind. The Fram
2 0.39007011 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ), you can follow him on twitter . How do you scale an AWS (Amazon Web Services) infrastructure? This article will give you a detailed reply in two parts: the tools you can use to make the most of Amazon’s dynamic approach, and the architectural model you should adopt for a scalable infrastructure. I base my report on my experience gained in several AWS production projects in casual gaming (Facebook), e-commerce infrastructures and within the mainstream GIS (Geographic Information System). It’s true that my experience in gaming ( IsCool, The Game ) is currently the most representative in terms of scalability, due to the number of users (over 800 thousand DAU – daily active users – at peak usage and over 20 million page views every day), however my experiences in e-commerce and GIS (currently underway) provide a different view of scalability, taking into account the various problems of availability and da
3 0.2288985 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
Introduction: This is a guest post by Shawn Hsiao , Luke Massa , and Victor Luu . Shawn runs TripAdvisor ’s Technical Operations team, Luke and Victor interned on his team this past summer. This post is introduced by Andy Gelfond , TripAdvisor’s head of engineering. It's been a little over a year since our last post about the TripAdvisor architecture . It has been an exciting year. Our business and team continues to grow, we are now an independent public company, and we have continued to keep/scale our development process and culture as we have grown - we still run dozens of independent teams, and each team continues to work across the entire stack. All that has changed are the numbers: 56M visitors per month 350M+ pages requests a day 120TB+ of warehouse data running on a large Hadoop cluster, and quickly growing We also had a very successful college intern program that brought on over 60 interns this past summer, all who were quickly on boarded and doing the same kind of work a
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
6 0.21791933 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
7 0.20951407 1058 high scalability-2011-06-13-Automation on AWS with Ruby and Puppet
8 0.20912901 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
9 0.20385288 1543 high scalability-2013-11-05-10 Things You Should Know About AWS
10 0.20232478 1631 high scalability-2014-04-14-How do you even do anything without using EBS?
11 0.19849499 96 high scalability-2007-09-18-Amazon Architecture
12 0.18894985 1112 high scalability-2011-09-07-What Google App Engine Price Changes Say About the Future of Web Architecture
13 0.18476787 961 high scalability-2010-12-21-SQL + NoSQL = Yes !
16 0.1784351 841 high scalability-2010-06-14-How scalable could be a cPanel Hosting service?
20 0.17538022 517 high scalability-2009-02-21-Google AppEngine - A Second Look
topicId topicWeight
[(0, 0.344), (1, 0.061), (2, 0.045), (3, 0.133), (4, -0.153), (5, -0.148), (6, 0.083), (7, -0.207), (8, 0.046), (9, -0.112), (10, -0.001), (11, 0.038), (12, -0.013), (13, -0.067), (14, -0.049), (15, -0.017), (16, 0.011), (17, -0.011), (18, 0.057), (19, 0.021), (20, -0.012), (21, -0.023), (22, 0.031), (23, 0.056), (24, 0.001), (25, 0.002), (26, -0.055), (27, -0.055), (28, -0.007), (29, -0.042), (30, -0.018), (31, 0.023), (32, 0.058), (33, 0.035), (34, -0.026), (35, -0.024), (36, 0.083), (37, -0.017), (38, -0.002), (39, 0.013), (40, 0.015), (41, 0.025), (42, -0.02), (43, -0.035), (44, 0.026), (45, -0.019), (46, -0.026), (47, 0.007), (48, 0.038), (49, 0.081)]
simIndex simValue blogId blogTitle
same-blog 1 0.98116952 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ) on the differences between using a cloud infrastructure and building your own. Frédéric was kind enough to translate the original French version of this article into English. I’ve been noticing many questions about the differences inherent in choosing between a Cloud infrastructure such as AWS (Amazon Web Services) and a traditional physical infrastructure. Firstly, there are a certain number of preconceived notions on this subject that I will attempt to decode for you. Then, it must be understood that each infrastructure has its advantages and disadvantages: a Cloud-type infrastructure does not necessarily fulfill your requirements in every case, however, it can satisfy some of them by optimizing or facilitating the features offered by a traditional physical infrastructure. I will therefore demonstrate the differences between the two that I have noticed, in order to help you make up your own mind. The Fram
2 0.82660484 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ), you can follow him on twitter . How do you scale an AWS (Amazon Web Services) infrastructure? This article will give you a detailed reply in two parts: the tools you can use to make the most of Amazon’s dynamic approach, and the architectural model you should adopt for a scalable infrastructure. I base my report on my experience gained in several AWS production projects in casual gaming (Facebook), e-commerce infrastructures and within the mainstream GIS (Geographic Information System). It’s true that my experience in gaming ( IsCool, The Game ) is currently the most representative in terms of scalability, due to the number of users (over 800 thousand DAU – daily active users – at peak usage and over 20 million page views every day), however my experiences in e-commerce and GIS (currently underway) provide a different view of scalability, taking into account the various problems of availability and da
3 0.82031006 1543 high scalability-2013-11-05-10 Things You Should Know About AWS
Introduction: Authored by Chris Fregly : Former Netflix Streaming Platform Engineer, AWS Certified Solution Architect and Purveyor of fluxcapacitor.com. Ahead of the upcoming 2nd annual re:Invent conference, inspired by Simone Brunozzi’s recent presentation at an AWS Meetup in San Francisco, and collected from a few of my recent Fluxcapacitor.com consulting engagements, I’ve compiled a list of 10 useful time and clock-tick saving tips about AWS. 1) Query AWS resource metadata Can’t remember the EBS-Optimized IO throughput of your c1.xlarge cluster? How about the size limit of an S3 object on a single PUT? awsnow.info is the answer to all of your AWS-resource metadata questions. Interested in integrating awsnow.info with your application? You’re in luck. There’s now a REST API , as well! Note: These are default soft limits and will vary by account. 2) Tame your S3 buckets Delete an entire S3 bucket with a single CLI command:
4 0.80642444 1058 high scalability-2011-06-13-Automation on AWS with Ruby and Puppet
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ), you can follow him on twitter . Urbandive is an immersive view service launched by the French YellowPages which allows you to travel in cities in France thanks to a 360° view. Urbandive focuses on providing high definition pictures and accurate professional and social content. One of the biggest jobs was to enable a fast scalable architecture, because it was really difficult to forecast the traffic load at production time. Traffic load may be influenced if the service receives attention from users as a result of advertising. Below you will find a summary of the goals we achieve by using a Ruby scheduler built on top of Puppet on AWS to create a complete infrastructure. Workflow & XTR-Lucid Our scalability combo is : a home-made Ruby scheduler ( XTR-Lucid ) to deal with AWS APIs + the Puppet Master to install services and configure EC2 instances and keep them up-to-date during all the product
5 0.79573828 1343 high scalability-2012-10-18-Save up to 30% by Selecting Better Performing Amazon Instances
Introduction: If you like the idea of exploiting market inconsistencies to lower your costs then you will love this paper and video from the Hot Cloud '12 conference: Exploiting Hardware Heterogeneity within the Same Instance Type of Amazon EC2 . The conclusion is interesting and is a source of good guidance: Amazon EC2 uses diversified hardware to host the same type of instance. The hardware diversity results in performance variation. In general, the variation between the fast instances and slow instances can reach 40%. In some applications, the variation can even approach up to 60%. By selecting fast instances within the same instance type, Amazon EC2 users can acquire up to 30% of cost saving, if the fast instances have a relatively low probability. The abstract: Cloud computing providers might start with near-homogeneous hardware environment. Over time, the homogeneous environment will most likely evolve into heterogeneous one because of possible upgrades and replac
8 0.77099133 1448 high scalability-2013-04-29-AWS v GCE Face-off and Why Innovation Needs Lower Cost Infrastructures
9 0.76045728 1286 high scalability-2012-07-18-Strategy: Kill Off Multi-tenant Instances with High CPU Stolen Time
10 0.75267607 798 high scalability-2010-03-22-7 Secrets to Successfully Scaling with Scalr (on Amazon) by Sebastian Stadil
11 0.74904913 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
12 0.74381071 1006 high scalability-2011-03-17-Are long VM instance spin-up times in the cloud costing you money?
13 0.739923 1575 high scalability-2014-01-08-Under Snowden's Light Software Architecture Choices Become Murky
14 0.73964107 1624 high scalability-2014-04-01-The Mullet Cloud Selection Pattern
15 0.73510271 1212 high scalability-2012-03-21-The Conspecific Hybrid Cloud
16 0.73329997 1549 high scalability-2013-11-15-Stuff The Internet Says On Scalability For November 15th, 2013
17 0.73238534 1347 high scalability-2012-10-25-Not All Regions are Created Equal - South America Es Bueno
18 0.73154926 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
19 0.73095512 1631 high scalability-2014-04-14-How do you even do anything without using EBS?
20 0.72678876 1029 high scalability-2011-04-25-The Big List of Articles on the Amazon Outage
topicId topicWeight
[(1, 0.153), (2, 0.155), (6, 0.015), (10, 0.098), (30, 0.036), (47, 0.014), (55, 0.041), (56, 0.015), (61, 0.107), (77, 0.029), (79, 0.114), (85, 0.034), (94, 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.97294778 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ) on the differences between using a cloud infrastructure and building your own. Frédéric was kind enough to translate the original French version of this article into English. I’ve been noticing many questions about the differences inherent in choosing between a Cloud infrastructure such as AWS (Amazon Web Services) and a traditional physical infrastructure. Firstly, there are a certain number of preconceived notions on this subject that I will attempt to decode for you. Then, it must be understood that each infrastructure has its advantages and disadvantages: a Cloud-type infrastructure does not necessarily fulfill your requirements in every case, however, it can satisfy some of them by optimizing or facilitating the features offered by a traditional physical infrastructure. I will therefore demonstrate the differences between the two that I have noticed, in order to help you make up your own mind. The Fram
Introduction: There has been an update on Pinterest: Pinterest growth driven by Amazon cloud scalability since our last post: A Short on the Pinterest Stack for Handling 3+ Million Users . With Pinterest we see a story very similar to that of Instagram . Huge growth, lots of users, lots of data, with remarkably few employees, all on the cloud. While it's true that both Pinterest and Instagram are not making great advances in science and technology , that is more indicator of the easy power of today's commodity environments rather than a sign of Silicon Valley's lack of innovation. The numbers are so huge and the valuations are so high we naturally want some sort of fundamental technological revolution to underlie their growth. The revolution is more subtle. It really is just that easy to attain such growth these days, if you can execute on the right idea. Get used to it. This is the new normal. Here's what Pinterest looks like today: 80 million objects stored in S3 with 410 terabytes
3 0.96084094 1448 high scalability-2013-04-29-AWS v GCE Face-off and Why Innovation Needs Lower Cost Infrastructures
Introduction: This is a repost of part 2 ( part 1 ) of an interview I did for the Boundary blog . Boundary: There’s another battle coming down the pike between Amazon (AWS) and Google (GCE). How should the CTO decide which one’s best? Hoff: Given that GCE is still closed to public access we have very little common experience on which to judge. The best way to decide is as always, by running a few experiments. Pick a few representative projects, a representative team, implement the projects on both infrastructures, crunch some numbers, figure out the bigger picture and then select the one you wanted in the first place . Sebastian Stadil, founder of Scalr, recently wrote about his experiences on both platforms and found some interesting differences: AWS has a much richer set of services; GCE is on-demand only, so AWS can be cheaper; GCE has faster disk and faster network IO, especially between datacenters; GCE has faster boot times and can mount read-only partitions across multiple
4 0.95964473 1180 high scalability-2012-01-24-The State of NoSQL in 2012
Introduction: This is a guest post by Siddharth Anand , a senior member of LinkedIn's Distributed Data Systems team. Preamble Ramble If you’ve been working in the online (e.g. internet) space over the past 3 years, you are no stranger to terms like “the cloud” and “NoSQL”. In 2007, Amazon published a paper on Dynamo . The paper detailed how Dynamo, employing a collection of techniques to solve several problems in fault-tolerance, provided a resilient solution to the on-line shopping cart problem. A few years go by while engineers at AWS toil in relative obscurity at standing up their public cloud. It’s December 2008 and I am a member of Netflix’s Software Infrastructure team. We’ve just been told that there is something called the “CAP theorem” and because of it, we are to abandon our datacenter in hopes of leveraging Cloud Computing. Huh? A month into the investigation, we start wondering about our Oracle database. How are we are going to move it into the cloud? That’s when we are
Introduction: This is a guest post by Doug Judd , original creator of Hypertable and the CEO of Hypertable, Inc. Hypertable delivers 2X better throughput in most tests -- HBase fails 41 and 167 billion record insert tests, overwhelmed by garbage collection -- Both systems deliver similar results for random read uniform test We recently conducted a test comparing the performance of Hypertable ( @hypertable ) version 0.9.5.5 to that of HBase ( @HBase ) version 0.90.4 (CDH3u2) running Zookeeper 3.3.4. In this post, we summarize the results and offer explanations for the discrepancies. For the full test report, see Hypertable vs. HBase II . Introduction Hypertable and HBase are both open source, scalable databases modeled after Google's proprietary Bigtable database. The primary difference between the two systems is that Hypertable is written in C++, while HBase is written in Java. We modeled this test after the one described in section 7 of the Bigtable paper and tuned both systems fo
6 0.95411813 1109 high scalability-2011-09-02-Stuff The Internet Says On Scalability For September 2, 2011
7 0.95332462 576 high scalability-2009-04-21-What CDN would you recommend?
8 0.95211762 851 high scalability-2010-07-02-Hot Scalability Links for July 2, 2010
9 0.95055199 1036 high scalability-2011-05-06-Stuff The Internet Says On Scalability For May 6th, 2011
10 0.95015991 1289 high scalability-2012-07-23-State of the CDN: More Traffic, Stable Prices, More Products, Profits - Not So Much
11 0.94958937 1302 high scalability-2012-08-10-Stuff The Internet Says On Scalability For August 10, 2012
12 0.94885552 1028 high scalability-2011-04-22-Stuff The Internet Says On Scalability For April 22, 2011
13 0.94841427 1137 high scalability-2011-11-04-Stuff The Internet Says On Scalability For November 4, 2011
14 0.94813174 106 high scalability-2007-10-02-Secrets to Fotolog's Scaling Success
15 0.94764513 1626 high scalability-2014-04-04-Stuff The Internet Says On Scalability For April 4th, 2014
16 0.9457168 498 high scalability-2009-01-20-Product: Amazon's SimpleDB
17 0.94548988 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
18 0.94523782 1344 high scalability-2012-10-19-Stuff The Internet Says On Scalability For October 19, 2012
19 0.94482946 383 high scalability-2008-09-10-Shard servers -- go big or small?