high_scalability high_scalability-2014 high_scalability-2014-1647 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: After Google cut prices on their Google Cloud Platform Amazon quickly followed with their own price cuts . Even more interesting is what the future holds for pricing. The near future looks great. After that? We'll see. Adrian Cockcroft highlights that Google thinks prices should follow Moore’s law, which means we should expect prices to halve every 18-24 months. That's good news. Greater cost certainty means you can make much more aggressive build out plans. With the savings you can hire more people, handle more customers, and add those media rich features you thought you couldn't afford. Design is directly related to costs. Without Google competing with Amazon there's little doubt the price reduction curve would be much less favorable. As a late cloud entrant Google is now in a customer acquisition phase, so they are willing to pay for customers, which means lower prices are an acceptable cost of doing business. Profit and high margins are not the objective. Getting marke
sentIndex sentText sentNum sentScore
1 After Google cut prices on their Google Cloud Platform Amazon quickly followed with their own price cuts . [sent-1, score-0.497]
2 Adrian Cockcroft highlights that Google thinks prices should follow Moore’s law, which means we should expect prices to halve every 18-24 months. [sent-6, score-0.809]
3 As a late cloud entrant Google is now in a customer acquisition phase, so they are willing to pay for customers, which means lower prices are an acceptable cost of doing business. [sent-12, score-0.669]
4 Amazon on the other hand has been reaping the higher margins earned from recurring customers. [sent-15, score-0.335]
5 So Google's entrance into the early product life cycle phase is making Amazon eat into their margins and is forcing down prices over all. [sent-16, score-0.852]
6 In other words, hard disk prices are dropping much faster than datacenter prices. [sent-20, score-0.567]
7 This is because, right, datacenters have costs other than hard disks (power, cooling, bandwidth, building infrastructure, diesel backup generators, battery backup systems, fire suppression, staffing, etc). [sent-21, score-0.417]
8 Most of those costs do not follow Moore's Law -- in fact energy costs are on a long trend upwards. [sent-22, score-0.364]
9 So over time, the gap shown by the graph should continue to widen. [sent-23, score-0.175]
10 There is also some long term strategic advantage to building your own, e. [sent-25, score-0.185]
11 , GDrive dropped their price dramatically at will because Google owns their datacenters, but Dropbox couldn't do that without convincing Amazon to drop the price they pay for S3. [sent-27, score-0.35]
12 Costs other than hardware began dominating in datacenters several years ago, Moore's Law-like effects are dampened. [sent-28, score-0.213]
13 Energy/cooling and cooling costs do not follow Moore's Law, and those costs make up a huge component of the overall picture in datacenters. [sent-29, score-0.333]
14 This is only going to get worse, barring some radical new energy production technology arriving on the scene. [sent-30, score-0.153]
15 What we're [ Space Monkey ] interested in, long term, is dropping the floor out from underneath all of these, and I think that only happens if you get out of the datacenter entirely. [sent-31, score-0.291]
16 As the size of cloud market is still growing there will still be a fight for market share. [sent-32, score-0.422]
17 When growth slows and the market is divided between major players a collusionary pricing phase will take over. [sent-33, score-0.457]
18 The need for higher margins to justify the cash flow drain during the customer acquisition phase will reverse the favorable trends we are seeing now. [sent-36, score-0.612]
19 " The Cloud Is Not Green Brad Stone : “Bill Miller, the chief investment officer at Legg Mason Capital Management and a major Amazon shareholder, asked Bezos at the time about the profitability prospects for AWS. [sent-40, score-0.23]
20 Bezos predicted they would be good over the long term but said that he didn’t want to repeat “Steve Jobs’s mistake” of pricing the iPhone in a way that was so fantastically profitable that the smartphone market became a magnet for competition. [sent-41, score-0.644]
wordName wordTfidf (topN-words)
[('prices', 0.365), ('margins', 0.251), ('alen', 0.185), ('phase', 0.161), ('market', 0.156), ('moore', 0.149), ('datacenters', 0.143), ('pricing', 0.14), ('bezos', 0.137), ('price', 0.132), ('monkey', 0.123), ('cooling', 0.121), ('acquisition', 0.121), ('law', 0.119), ('term', 0.111), ('floor', 0.11), ('cloud', 0.11), ('amazon', 0.109), ('dropping', 0.107), ('google', 0.107), ('costs', 0.106), ('hard', 0.095), ('historical', 0.091), ('graph', 0.089), ('dropped', 0.086), ('shown', 0.086), ('reaping', 0.084), ('graphed', 0.084), ('staffing', 0.084), ('suppression', 0.084), ('prospects', 0.084), ('gdrive', 0.084), ('fantastically', 0.084), ('favorable', 0.079), ('magnet', 0.079), ('halve', 0.079), ('energy', 0.078), ('housed', 0.075), ('mason', 0.075), ('entrance', 0.075), ('arriving', 0.075), ('customers', 0.075), ('long', 0.074), ('diesel', 0.073), ('officer', 0.073), ('profitability', 0.073), ('entrant', 0.073), ('dominating', 0.07), ('graphic', 0.068), ('certainty', 0.068)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 1647 high scalability-2014-05-14-Google Says Cloud Prices Will Follow Moore’s Law: Are We All Renters Now?
Introduction: After Google cut prices on their Google Cloud Platform Amazon quickly followed with their own price cuts . Even more interesting is what the future holds for pricing. The near future looks great. After that? We'll see. Adrian Cockcroft highlights that Google thinks prices should follow Moore’s law, which means we should expect prices to halve every 18-24 months. That's good news. Greater cost certainty means you can make much more aggressive build out plans. With the savings you can hire more people, handle more customers, and add those media rich features you thought you couldn't afford. Design is directly related to costs. Without Google competing with Amazon there's little doubt the price reduction curve would be much less favorable. As a late cloud entrant Google is now in a customer acquisition phase, so they are willing to pay for customers, which means lower prices are an acceptable cost of doing business. Profit and high margins are not the objective. Getting marke
2 0.17625967 1289 high scalability-2012-07-23-State of the CDN: More Traffic, Stable Prices, More Products, Profits - Not So Much
Introduction: CDNs ( content delivery networks ) are the secret shadow super powers behind the web and Dan Rayburn at streamingmedia.com is the go to investigative reporter for quality information on CDNs. Every year Dan has a Content Delivery Summit on all things CDN and those videos are now available . Dan also gives a kind of state of the industry talk where he does something wonderful, he gives real numbers and prices. Dan really knows his stuff and is an excellent speaker, so watch the video, but here’s my gloss on the state of the CDN so far this year: Massive growth . Large customers are expecting 126% growth in video traffic over last year; medium size customers are seeing 48% traffic growth, small sized customer are seeing 73.3% traffic growth. More traffic != More profit . Traffic growth doesn’t lead to more profit because the traffic growth is concentrated in larger customers that can make the best deals. Video takes up the largest amount of traffic on a
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
5 0.13774902 1637 high scalability-2014-04-25-Stuff The Internet Says On Scalability For April 25th, 2014
Introduction: Hey, it's HighScalability time: New World Record BASE jumping from World's Tallest Building. #crazy 30 billion : total Pinterest pins; 500,000,000 : What'sApp users (700 million photos and 100 million videos every single day); 1 billion : Facebook active users on phones and tablets. Quotable Quotes: @jimplush : Google spent 2.3 billion on infrastructure in Q1. Remember that when you say you want to be "the Google of something" Clay Shirky: I think one of the things that happened to the P2P market for infrastructure is that users preference for predictable pricing vs resource-sensitive pricing is so overwhelming that they will overpay to anyone who can promise flat prices. And because the logic of centralization vs decentralization is so price sensitive, I don't think there is any logical reason to assume a broadly stable class of apps, separate from c
7 0.13391954 1466 high scalability-2013-05-29-Amazon: Creating a Customer Utopia One Culture Hack at a Time
8 0.12787585 1112 high scalability-2011-09-07-What Google App Engine Price Changes Say About the Future of Web Architecture
9 0.12481583 687 high scalability-2009-08-24-How Google Serves Data from Multiple Datacenters
10 0.12053291 1310 high scalability-2012-08-23-Economies of Scale in the Datacenter: Gmail is 100x Cheaper to Run than Your Own Server
11 0.12003309 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
12 0.11598663 200 high scalability-2008-01-02-WEB hosting Select
13 0.1159433 1621 high scalability-2014-03-28-Stuff The Internet Says On Scalability For March 28th, 2014
14 0.11258432 865 high scalability-2010-07-27-A Metric A$$-Ton of Joe Stump: The Cloud is Cheaper than Bare Metal
15 0.1115781 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
16 0.11103883 1206 high scalability-2012-03-09-Stuff The Internet Says On Scalability For March 9, 2012
17 0.10837165 768 high scalability-2010-02-01-What Will Kill the Cloud?
18 0.10475049 688 high scalability-2009-08-26-Hot Links for 2009-8-26
19 0.10205942 195 high scalability-2007-12-28-Amazon's EC2: Pay as You Grow Could Cut Your Costs in Half
20 0.10159272 1564 high scalability-2013-12-13-Stuff The Internet Says On Scalability For December 13th, 2013
topicId topicWeight
[(0, 0.164), (1, 0.07), (2, 0.053), (3, 0.123), (4, -0.091), (5, -0.091), (6, -0.023), (7, -0.014), (8, 0.067), (9, -0.036), (10, -0.003), (11, -0.086), (12, -0.027), (13, 0.038), (14, 0.022), (15, 0.012), (16, -0.069), (17, 0.058), (18, -0.002), (19, 0.021), (20, -0.034), (21, 0.013), (22, -0.011), (23, -0.043), (24, -0.046), (25, 0.046), (26, -0.015), (27, -0.016), (28, 0.012), (29, -0.025), (30, 0.012), (31, -0.016), (32, 0.005), (33, -0.054), (34, -0.0), (35, 0.004), (36, -0.019), (37, 0.026), (38, 0.013), (39, -0.005), (40, 0.006), (41, 0.022), (42, 0.002), (43, -0.019), (44, -0.008), (45, -0.052), (46, -0.017), (47, -0.065), (48, -0.013), (49, -0.076)]
simIndex simValue blogId blogTitle
same-blog 1 0.974738 1647 high scalability-2014-05-14-Google Says Cloud Prices Will Follow Moore’s Law: Are We All Renters Now?
Introduction: After Google cut prices on their Google Cloud Platform Amazon quickly followed with their own price cuts . Even more interesting is what the future holds for pricing. The near future looks great. After that? We'll see. Adrian Cockcroft highlights that Google thinks prices should follow Moore’s law, which means we should expect prices to halve every 18-24 months. That's good news. Greater cost certainty means you can make much more aggressive build out plans. With the savings you can hire more people, handle more customers, and add those media rich features you thought you couldn't afford. Design is directly related to costs. Without Google competing with Amazon there's little doubt the price reduction curve would be much less favorable. As a late cloud entrant Google is now in a customer acquisition phase, so they are willing to pay for customers, which means lower prices are an acceptable cost of doing business. Profit and high margins are not the objective. Getting marke
2 0.7792148 865 high scalability-2010-07-27-A Metric A$$-Ton of Joe Stump: The Cloud is Cheaper than Bare Metal
Introduction: Should you pay more in the cloud or pay less for bare metal in the datacenter? This is a crucial decision point facing startups today. Which way should you go? In this Webpulp.tv interview , Joe Stump, always a go-to guy when you need a metric ass-ton (a favorite expression of Joe’s) of good advice on cutting edge practices for the modern startup, laughs at conventional wisdom by saying the cloud is really not more expensive than bare metal. The argument for a cheaper cloud has a three main points: Raw Hardware Costs are Close . Buying hardware is cheaper than renting hardware from Amazon, but it’s close when you consider reserved instances . A reserved instance costs about $250 a year and a new server costs about $1500. Let’s say Amazon is 30% slower. That means for a cluster of 20 machines you would need a another 7 servers. This takes 10 minutes and 4 button clicks to provision on the cloud (the advantage of a scale-out architecture + automation + cloud aware tools).
3 0.76022905 1448 high scalability-2013-04-29-AWS v GCE Face-off and Why Innovation Needs Lower Cost Infrastructures
Introduction: This is a repost of part 2 ( part 1 ) of an interview I did for the Boundary blog . Boundary: There’s another battle coming down the pike between Amazon (AWS) and Google (GCE). How should the CTO decide which one’s best? Hoff: Given that GCE is still closed to public access we have very little common experience on which to judge. The best way to decide is as always, by running a few experiments. Pick a few representative projects, a representative team, implement the projects on both infrastructures, crunch some numbers, figure out the bigger picture and then select the one you wanted in the first place . Sebastian Stadil, founder of Scalr, recently wrote about his experiences on both platforms and found some interesting differences: AWS has a much richer set of services; GCE is on-demand only, so AWS can be cheaper; GCE has faster disk and faster network IO, especially between datacenters; GCE has faster boot times and can mount read-only partitions across multiple
4 0.71156377 527 high scalability-2009-03-06-Cloud Programming Directly Feeds Cost Allocation Back into Software Design
Introduction: Update 6 : CARS = Cost Aware Runtimes and Services by William Louth. Update 5 : Damn You Google, Damn You Yahoo! Why D'Ya Do This to Us? Free accounts on a cloud platform are a constant drain of money. Update 4: Caching becomes even more important in CPU based billing environments . Avoiding the CPU means saving money. Update 3: An interesting simple example of this idea showed up on the Google AppEngine list. With one paging algorithm and one use of AJAX the yearly cost of the site was $1000. By changing those algorithms the site went under quota and became free again. This will make life a lot more interesting for developers. Update 2: Business Model Influencing Software Architecture by Brandon Watson. The profitability of your project could disappear overnight on account of code behaving badly . Update: Amazon adds Elastic Block Store at $0.10 per 1 million I/O requests. Now I need some cost minimization storage algorithms! In the GAE Meetup yesterday a very in
Introduction: Urs Hoelzle , infrastructure guru and SVP at Google, made a really interesting statement about the economics of scale in the datacenter: We’ve shown that when you run a large application in the datacenter, like Gmail, you can, compared to a small organization running their own email server, you can save nearly a factor of 100 in terms of compute and energy, when you run it at scale. My first thought was shock at the magnitude of the difference. 100x is a chasm crosser. Then I thought about Gmail, it's horizontally scalable using technologies that are following Moore's Law (storage and compute), latency requirements are lax, a commodity network is sufficient, and it can be highly automated so management costs scale slower than users. After that it's a simple matter of software :-) Oh, and developing a market where it's "cheaper to run a large thing than a small thing."
6 0.69139707 1466 high scalability-2013-05-29-Amazon: Creating a Customer Utopia One Culture Hack at a Time
7 0.68265438 1380 high scalability-2013-01-02-Why Pinterest Uses the Cloud Instead of Going Solo - To Be Or Not To Be
8 0.67674607 284 high scalability-2008-03-19-RAD Lab is Creating a Datacenter Operating System
9 0.67613494 1603 high scalability-2014-02-28-Stuff The Internet Says On Scalability For February 28th, 2014
10 0.65892804 1053 high scalability-2011-06-06-Apple iCloud: Syncing and Distributed Storage Over Streaming and Centralized Storage
11 0.64983571 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
13 0.64733756 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
14 0.64655399 1464 high scalability-2013-05-24-Stuff The Internet Says On Scalability For May 24, 2013
15 0.64308405 1410 high scalability-2013-02-20-Smart Companies Fail Because they Do Everything Right - Staying Alive to Scale
16 0.62860227 768 high scalability-2010-02-01-What Will Kill the Cloud?
17 0.62583989 1575 high scalability-2014-01-08-Under Snowden's Light Software Architecture Choices Become Murky
18 0.62351978 1027 high scalability-2011-04-20-Packet Pushers: How to Build a Low Cost Data Center
19 0.62351686 1206 high scalability-2012-03-09-Stuff The Internet Says On Scalability For March 9, 2012
topicId topicWeight
[(1, 0.089), (2, 0.128), (10, 0.078), (20, 0.024), (30, 0.026), (35, 0.173), (40, 0.056), (47, 0.011), (56, 0.015), (61, 0.063), (79, 0.139), (85, 0.056), (94, 0.034), (96, 0.017)]
simIndex simValue blogId blogTitle
1 0.94049102 740 high scalability-2009-11-11-Hot Scalability Links for Nov 11 2009
Introduction: The Cost of Latency by James Hamilton. James summarizes latency info from Steve Souder , Greg Linden , and Marissa Mayer . Speed [is] an undervalued and under-discussed asset on the web. Dynamo - Part I: a followup and re-rebuttals . Dynamo under attack as having Design flaws and the resounding rebuttal in response. Programming Bits and Atoms . Thinking about programming and scaling as a problem in physics. Absolutely fascinating and inspiring. Scaling Servers with the Cloud: Amazon S3 . Build a static site using S3 for pennies. An oldly but still a goody idea. Are Wireless Road Trains the Cure for Traffic Congestion? The concept of road trains--up to eight vehicles zooming down the road together--has long been considered a faster, safer, and greener way of traveling long distances by car. Erlang at Facebook by Eugene Letuchy. How Facebook uses Erlang to implement Chat, AIM Presence, and Chat Jabber support. Yahoo Open Sources Traffic Server . Traffic Serv
same-blog 2 0.87774992 1647 high scalability-2014-05-14-Google Says Cloud Prices Will Follow Moore’s Law: Are We All Renters Now?
Introduction: After Google cut prices on their Google Cloud Platform Amazon quickly followed with their own price cuts . Even more interesting is what the future holds for pricing. The near future looks great. After that? We'll see. Adrian Cockcroft highlights that Google thinks prices should follow Moore’s law, which means we should expect prices to halve every 18-24 months. That's good news. Greater cost certainty means you can make much more aggressive build out plans. With the savings you can hire more people, handle more customers, and add those media rich features you thought you couldn't afford. Design is directly related to costs. Without Google competing with Amazon there's little doubt the price reduction curve would be much less favorable. As a late cloud entrant Google is now in a customer acquisition phase, so they are willing to pay for customers, which means lower prices are an acceptable cost of doing business. Profit and high margins are not the objective. Getting marke
3 0.83142895 101 high scalability-2007-09-27-Product: Ganglia Monitoring System
Introduction: Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
Introduction: Many enterprises' high-availability architecture is based on the assumption that you can prevent failure from happening by putting all your critical data in a centralized database, back it up with expensive storage, and replicate it somehow between the sites. As I argued in one of my previous posts ( Why Existing Databases (RAC) are So Breakable! ) many of those assumptions are broken at their core, as storage is doomed to failure just like any other device, expensive hardware doesn’t make things any better and database replication is often not enough. One of the main lessons that we can take from the likes of Amazon and Google is that the right way to ensure continuous high availability is by designing our system to cope with failure. We need to assume that what we tend to think of as unthinkable will probably happen, as that’s the nature of failure. So rather than trying to prevent failures, we need to build a system that will tolerate them. As we can learn from a recent outage
5 0.78029752 1153 high scalability-2011-12-08-Update on Scalable Causal Consistency For Wide-Area Storage With COPS
Introduction: Here are a few updates on the article Paper: Don’t Settle For Eventual: Scalable Causal Consistency For Wide-Area Storage With COPS from Mike Freedman and Wyatt Lloyd. Q: How software architectures could change in response to casual+ consistency? A : I don't really think they would much. Somebody would still run a two-tier architecture in their datacenter: a front-tier of webservers running both (say) PHP and our client library, and a back tier of storage nodes running COPS. (I'm not sure if it was obvious given the discussion of our "thick" client -- you should think of the COPS client dropping in where a memcache client library does...albeit ours has per-session state.) Q: Why not just use vector clocks? A : The problem with vector clocks and scalability has always been that the size of vector clocks in O(N), where N is the number of nodes. So if we want to scale to a datacenter with 10K nodes, each piece of metadata must have size O(10K). And in fact, vector
6 0.77464044 440 high scalability-2008-11-11-Arhcitecture for content management
8 0.76801705 716 high scalability-2009-10-06-Building a Unique Data Warehouse
9 0.7666055 364 high scalability-2008-08-14-Product: Terracotta - Open Source Network-Attached Memory
12 0.76417768 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
13 0.76388162 1248 high scalability-2012-05-21-Pinterest Architecture Update - 18 Million Visitors, 10x Growth,12 Employees, 410 TB of Data
14 0.76386726 1630 high scalability-2014-04-11-Stuff The Internet Says On Scalability For April 11th, 2014
15 0.76298004 763 high scalability-2010-01-22-How BuddyPoke Scales on Facebook Using Google App Engine
16 0.76290172 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
17 0.76185501 289 high scalability-2008-03-27-Amazon Announces Static IP Addresses and Multiple Datacenter Operation
18 0.75979662 851 high scalability-2010-07-02-Hot Scalability Links for July 2, 2010
19 0.7597602 768 high scalability-2010-02-01-What Will Kill the Cloud?