high_scalability high_scalability-2009 high_scalability-2009-517 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Update 6: : Back to the Future for Data Storage . We are in the middle of a renaissance in data storage with the application of many new ideas and techniques; there's huge potential for breaking out of thinking about data storage in just one way. Update 5 : Building Scalable Web Applications with Google App Engine by Brett Slatkin. Update 4 : Why Google App Engine is broken and what Google must do to fix it by Aral Balkan. We don't care that it can scale. We care that it does scale. And that it scales when you need it the most. Issues: 1MB limit on data structures; 1MB limit on data structures; the short-term high CPU quota; quotas in general; Admin? What's that? Update 3 : BigTable Blues . Catherine Devlin couldn't port an application to GAE because it can't do basic filtering and can't search 5,000 records without timing out: "Querying from 5000 records - too much for the mighty BigTable, apparently." Followup: not the future database . "90% of the work of this proje
sentIndex sentText sentNum sentScore
1 Catherine Devlin couldn't port an application to GAE because it can't do basic filtering and can't search 5,000 records without timing out: "Querying from 5000 records - too much for the mighty BigTable, apparently. [sent-11, score-0.269]
2 Concerns swirl over poor performance, lack of a roadmap, perpetual beta status, poor support, and a quota system as torture chamber model of scalability. [sent-16, score-0.337]
3 James here and here also likes the ease of use but finds the data model takes some getting used to and is concerned the API limits won't scale for a real site. [sent-21, score-0.227]
4 Brett Morgan expands our cultural education in a thread on slow GAE databases performance when he talks about why MySQL thinking won't work on BigTable: It might look almost look like a sql db when you squint, but it's optimized for a totally different goal. [sent-44, score-0.24]
5 If you think that each different entity you retrieve could be retrieving a different disk block from a different machine in the cluster, then suddenly things start to make sense. [sent-45, score-0.32]
6 So if you build things such that web requests are only ever pulling a single entity from DataStore - by always precomputing everything - then your app will fly on all the read requests. [sent-49, score-0.313]
7 Having been on the wrong side of trying to scale up web app code, I can honestly say it is better to push the requirements of scaling into the face of us developers so that we do the right thing from the beginning. [sent-53, score-0.277]
8 AWS has a VM model whereas GAE has an application centric model. [sent-74, score-0.29]
9 In GAE you have a high level application component and you build out your application using services. [sent-79, score-0.271]
10 That speaks volumes about the efficacy of the application centric development model. [sent-90, score-0.198]
11 With an application centric model the view source viral spread approach would also work. [sent-109, score-0.29]
12 I mostly agree with this take, but I also agree with a commenter who observed that it's a lot harder for a team of developers to turn on a dime and adopt a whole new everything. [sent-112, score-0.243]
13 Though the Joyent's building blocks approach is very different than Google's application centric approach. [sent-137, score-0.318]
14 Niall Kennedy in Google App Engine for developers does a great job contrasting the complexity of your normal website setup with an application approach. [sent-139, score-0.201]
15 This means usage limits are set a bit low and with no pricing model to work from it's reasonable to be concerned about the cost. [sent-143, score-0.239]
16 What problems and successes are real developers experiencing trying to get real work done? [sent-158, score-0.273]
17 People are hitting limits like the 1MB upload limit on all requests. [sent-173, score-0.232]
18 This is very different than the AWS model which advocates moving work to your CPU so it makes sense to return large sets of data. [sent-175, score-0.214]
19 The pure service model has profound limitations for certain application types. [sent-177, score-0.245]
20 You must make a service external to the GAE environment to get things to work which means you have to develop in multiple environments. [sent-196, score-0.245]
wordName wordTfidf (topN-words)
[('gae', 0.473), ('bigtable', 0.214), ('google', 0.207), ('app', 0.166), ('engine', 0.112), ('python', 0.111), ('developers', 0.111), ('centric', 0.108), ('brett', 0.1), ('model', 0.092), ('level', 0.091), ('entity', 0.09), ('application', 0.09), ('datastore', 0.089), ('limits', 0.085), ('ca', 0.085), ('limit', 0.082), ('romans', 0.082), ('aws', 0.075), ('gallery', 0.073), ('lack', 0.072), ('port', 0.071), ('pill', 0.07), ('external', 0.07), ('joyent', 0.067), ('agree', 0.066), ('upload', 0.065), ('language', 0.065), ('avg', 0.063), ('quota', 0.063), ('service', 0.063), ('slow', 0.063), ('bulk', 0.062), ('work', 0.062), ('rdbms', 0.061), ('mindset', 0.061), ('different', 0.06), ('blocks', 0.06), ('limitation', 0.058), ('pulling', 0.057), ('cpu', 0.056), ('thinking', 0.055), ('poor', 0.055), ('cons', 0.054), ('records', 0.054), ('mysql', 0.053), ('layer', 0.052), ('suddenly', 0.05), ('environment', 0.05), ('real', 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999899 517 high scalability-2009-02-21-Google AppEngine - A Second Look
Introduction: Update 6: : Back to the Future for Data Storage . We are in the middle of a renaissance in data storage with the application of many new ideas and techniques; there's huge potential for breaking out of thinking about data storage in just one way. Update 5 : Building Scalable Web Applications with Google App Engine by Brett Slatkin. Update 4 : Why Google App Engine is broken and what Google must do to fix it by Aral Balkan. We don't care that it can scale. We care that it does scale. And that it scales when you need it the most. Issues: 1MB limit on data structures; 1MB limit on data structures; the short-term high CPU quota; quotas in general; Admin? What's that? Update 3 : BigTable Blues . Catherine Devlin couldn't port an application to GAE because it can't do basic filtering and can't search 5,000 records without timing out: "Querying from 5000 records - too much for the mighty BigTable, apparently." Followup: not the future database . "90% of the work of this proje
2 0.48720634 1112 high scalability-2011-09-07-What Google App Engine Price Changes Say About the Future of Web Architecture
Introduction: When I was a child, I spake as a child, I understood as a child, I thought as a child: but when I became a man, I put away childish things . -- Corinthians With this new pricing, developments will be driven by the costs . I like to optimize my apps to make them better or faster, but to optimize them just to make them cheaper is a waste of time. -- Sylvain on Google Groups The dream is dead. Google App Engine's bold pay for what you use dream dies as it leaves childish things behind and becomes a real product . Pricing will change . Architectures will change. Customers will change. Hearts and minds will change. But Google App Engine will survive. Google is shutting down many of its projects . GAE is not among them. Do we have GAE's pricing change to thank for it surving the more wood behind more deadly arrows push? Without a radical and quick shift towards profitably GAE would no doubt be a historical footnote in the long scroll of good ideas. The urgency involve
3 0.34303439 327 high scalability-2008-05-27-How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale
Introduction: Update 3 : ReadWriteWeb says Google App Engine Announces New Pricing Plans, APIs, Open Access . Pricing is specified but I'm not sure what to make of it yet. An image manipulation library is added (thus the need to pay for more CPU :-) and memcached support has been added. Memcached will help resolve the can't write for every read problem that pops up when keeping counters. Update 2 : onGWT.com threw a GAE load party and a lot of people came. The results at Load test : Google App Engine = 1, Community = 0 . GAE handled a peak of 35 requests/second and a sustained 10 requests/second. Some think performance was good, others not so good. My GMT watch broke and I was late to arrive. Maybe next time. Also added a few new design rules from the post. Update : Added a few new rules gleaned from the GAE Meetup : Design By Explicit Cost Model and Puts are Precious. How do you structure your database using a distributed hash table like BigTable ? The answer isn't what you might expect. If
4 0.29317418 1055 high scalability-2011-06-08-Stuff to Watch from Google IO 2011
Introduction: With the Google IO Developer Conference completed there are dozens and dozens of information packed videos now available. While you won't get any of the nifty free swag the attendees rake in, it's great of Google to make these videos available so quickly after the conference. Let's say you don't want to watch all the videos on the pretense you have a life, here are just a dozen scalability and architecture related videos you might find interesting: App Engine Backends by Justin Haugh, Greg Darke. One of this biggest complaints about GAE are the request deadlines. Those are now gone when you rent a new Backend node. Scaling App Engine Applications by Justin Haugh, Guido van Rossum. Two parts: how App Engine scales internally and how you as a programmer make it scale using the tools. Good discussion of scaling and why it's hard; 10,000 queries per second is a large app that needs good architecting; a good discussion of GAE's predictive scaling formula of how it decides w
5 0.27956384 301 high scalability-2008-04-08-Google AppEngine - A First Look
Introduction: I haven't developed an AppEngine application yet, I'm just taking a look around their documentation and seeing what stands out for me. It's not the much speculated super cluster VM . AppEngine is solidly grounded in code and structure. It reminds me a little of the guy who ran a website out of S3 with a splash of Heroku thrown in as a chaser. The idea is clearly to take advantage of our massive multi-core future by creating a shared nothing infrastructure based firmly on a core set of infinitely scalable database, storage and CPU services. Don't forget Google also has a few other services to leverage: email, login, blogs, video, search, ads, metrics, and apps. A shared nothing request is a simple beast. By its very nature shared nothing architectures must be composed of services which are themselves already scalable and Google is signing up to supply that scalable infrastructure. Google has been busy creating a platform of out-of-the-box scalable services to build
7 0.26224872 789 high scalability-2010-03-05-Strategy: Planning for a Power Outage Google Style
8 0.2394134 1432 high scalability-2013-04-01-Khan Academy Checkbook Scaling to 6 Million Users a Month on GAE
9 0.23829684 1023 high scalability-2011-04-14-Strategy: Cache Application Start State to Reduce Spin-up Times
10 0.2240269 763 high scalability-2010-01-22-How BuddyPoke Scales on Facebook Using Google App Engine
11 0.21712849 641 high scalability-2009-06-29-Google App Engine plus Amazon AWS: Best of both worlds
13 0.21222639 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
14 0.19819871 776 high scalability-2010-02-12-Hot Scalability Links for February 12, 2010
15 0.19556661 1064 high scalability-2011-06-20-35+ Use Cases for Choosing Your Next NoSQL Database
16 0.19274458 514 high scalability-2009-02-18-Numbers Everyone Should Know
17 0.19131108 1135 high scalability-2011-10-31-15 Ways to Make Your Application Feel More Responsive under Google App Engine
18 0.19001861 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
19 0.18873982 949 high scalability-2010-11-29-Stuff the Internet Says on Scalability For November 29th, 2010
20 0.18542399 1472 high scalability-2013-06-07-Stuff The Internet Says On Scalability For June 7, 2013
topicId topicWeight
[(0, 0.346), (1, 0.157), (2, 0.017), (3, 0.035), (4, 0.001), (5, -0.026), (6, -0.021), (7, -0.006), (8, 0.085), (9, -0.082), (10, -0.033), (11, -0.048), (12, -0.03), (13, -0.005), (14, 0.064), (15, -0.127), (16, -0.194), (17, -0.084), (18, 0.18), (19, -0.086), (20, 0.083), (21, -0.072), (22, 0.066), (23, -0.17), (24, 0.075), (25, 0.063), (26, 0.097), (27, -0.098), (28, -0.007), (29, -0.038), (30, -0.016), (31, -0.135), (32, 0.087), (33, -0.0), (34, -0.063), (35, 0.027), (36, -0.021), (37, 0.0), (38, 0.074), (39, -0.026), (40, -0.032), (41, 0.003), (42, 0.001), (43, 0.041), (44, 0.076), (45, 0.02), (46, -0.059), (47, 0.108), (48, 0.032), (49, 0.034)]
simIndex simValue blogId blogTitle
same-blog 1 0.96367854 517 high scalability-2009-02-21-Google AppEngine - A Second Look
Introduction: Update 6: : Back to the Future for Data Storage . We are in the middle of a renaissance in data storage with the application of many new ideas and techniques; there's huge potential for breaking out of thinking about data storage in just one way. Update 5 : Building Scalable Web Applications with Google App Engine by Brett Slatkin. Update 4 : Why Google App Engine is broken and what Google must do to fix it by Aral Balkan. We don't care that it can scale. We care that it does scale. And that it scales when you need it the most. Issues: 1MB limit on data structures; 1MB limit on data structures; the short-term high CPU quota; quotas in general; Admin? What's that? Update 3 : BigTable Blues . Catherine Devlin couldn't port an application to GAE because it can't do basic filtering and can't search 5,000 records without timing out: "Querying from 5000 records - too much for the mighty BigTable, apparently." Followup: not the future database . "90% of the work of this proje
2 0.90040636 301 high scalability-2008-04-08-Google AppEngine - A First Look
Introduction: I haven't developed an AppEngine application yet, I'm just taking a look around their documentation and seeing what stands out for me. It's not the much speculated super cluster VM . AppEngine is solidly grounded in code and structure. It reminds me a little of the guy who ran a website out of S3 with a splash of Heroku thrown in as a chaser. The idea is clearly to take advantage of our massive multi-core future by creating a shared nothing infrastructure based firmly on a core set of infinitely scalable database, storage and CPU services. Don't forget Google also has a few other services to leverage: email, login, blogs, video, search, ads, metrics, and apps. A shared nothing request is a simple beast. By its very nature shared nothing architectures must be composed of services which are themselves already scalable and Google is signing up to supply that scalable infrastructure. Google has been busy creating a platform of out-of-the-box scalable services to build
3 0.88170689 1112 high scalability-2011-09-07-What Google App Engine Price Changes Say About the Future of Web Architecture
Introduction: When I was a child, I spake as a child, I understood as a child, I thought as a child: but when I became a man, I put away childish things . -- Corinthians With this new pricing, developments will be driven by the costs . I like to optimize my apps to make them better or faster, but to optimize them just to make them cheaper is a waste of time. -- Sylvain on Google Groups The dream is dead. Google App Engine's bold pay for what you use dream dies as it leaves childish things behind and becomes a real product . Pricing will change . Architectures will change. Customers will change. Hearts and minds will change. But Google App Engine will survive. Google is shutting down many of its projects . GAE is not among them. Do we have GAE's pricing change to thank for it surving the more wood behind more deadly arrows push? Without a radical and quick shift towards profitably GAE would no doubt be a historical footnote in the long scroll of good ideas. The urgency involve
4 0.8793028 1055 high scalability-2011-06-08-Stuff to Watch from Google IO 2011
Introduction: With the Google IO Developer Conference completed there are dozens and dozens of information packed videos now available. While you won't get any of the nifty free swag the attendees rake in, it's great of Google to make these videos available so quickly after the conference. Let's say you don't want to watch all the videos on the pretense you have a life, here are just a dozen scalability and architecture related videos you might find interesting: App Engine Backends by Justin Haugh, Greg Darke. One of this biggest complaints about GAE are the request deadlines. Those are now gone when you rent a new Backend node. Scaling App Engine Applications by Justin Haugh, Guido van Rossum. Two parts: how App Engine scales internally and how you as a programmer make it scale using the tools. Good discussion of scaling and why it's hard; 10,000 queries per second is a large app that needs good architecting; a good discussion of GAE's predictive scaling formula of how it decides w
5 0.8540585 763 high scalability-2010-01-22-How BuddyPoke Scales on Facebook Using Google App Engine
Introduction: How do you scale a viral Facebook app that has skyrocketed to a mind boggling 65 million installs (the population of France)? That's the fortunate problem BuddyPoke co-founder Dave Westwood has and he talked about his solution at Wednesday's Facebook Meetup . Slides for the complete talk are here . For those not quite sure what BuddyPoke is, it's a social network application that lets users show their mood, hug, kiss, and poke their friends through on-line avatars. In many ways BuddyPoke is the quintessentially modern web application. It thrives off the energy of social network driven ecosystems. Game play mechanics, viral loops, and creative monetization strategies are all part of if its everyday conceptualization. It mashes together different technologies, not in a dark Frankensteining sort of way, but in a smart way that gets the most bang for the buck. Part of it runs on Facebook servers (free). Part of it runs on flash in a browser (free). Part of it runs on a storage clou
6 0.78543997 789 high scalability-2010-03-05-Strategy: Planning for a Power Outage Google Style
7 0.7781201 1432 high scalability-2013-04-01-Khan Academy Checkbook Scaling to 6 Million Users a Month on GAE
8 0.76684177 527 high scalability-2009-03-06-Cloud Programming Directly Feeds Cost Allocation Back into Software Design
9 0.75468498 299 high scalability-2008-04-07-Rumors of Signs and Portents Concerning Freeish Google Cloud
10 0.74752426 641 high scalability-2009-06-29-Google App Engine plus Amazon AWS: Best of both worlds
11 0.74334413 1084 high scalability-2011-07-22-Stuff The Internet Says On Scalability For July 22, 2011
12 0.74299788 327 high scalability-2008-05-27-How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale
13 0.72478789 1275 high scalability-2012-07-02-C is for Compute - Google Compute Engine (GCE)
14 0.72069144 949 high scalability-2010-11-29-Stuff the Internet Says on Scalability For November 29th, 2010
15 0.71381992 1472 high scalability-2013-06-07-Stuff The Internet Says On Scalability For June 7, 2013
16 0.71211475 776 high scalability-2010-02-12-Hot Scalability Links for February 12, 2010
17 0.70395648 1023 high scalability-2011-04-14-Strategy: Cache Application Start State to Reduce Spin-up Times
18 0.69744009 307 high scalability-2008-04-21-Using Google AppEngine for a Little Micro-Scalability
19 0.69377476 1117 high scalability-2011-09-16-Stuff The Internet Says On Scalability For September 16, 2011
20 0.67883015 972 high scalability-2011-01-11-Google Megastore - 3 Billion Writes and 20 Billion Read Transactions Daily
topicId topicWeight
[(1, 0.118), (2, 0.191), (4, 0.058), (10, 0.037), (26, 0.011), (30, 0.03), (40, 0.028), (47, 0.011), (56, 0.017), (57, 0.014), (61, 0.087), (77, 0.036), (79, 0.136), (85, 0.032), (94, 0.099)]
simIndex simValue blogId blogTitle
Introduction: Facebook has been teasing us. While many of their recent acquisitions have been surprising, shocking is the only word adequately describing Facebook's 5 day whirlwind acquisition of Oculus , immersive virtual reality visionaries, for a now paltry sounding $2 billion. The backlash is a pandemic, jumping across social networks with the speed only a meme powered by the directly unaffected can generate. For more than 30 years VR has been the dream burning in the heart of every science fiction fan. Now that this future might finally be here, Facebook’s ownage makes it seem like a wonderful and hopeful timeline has been choked off, killing the Metaverse before it even had a chance to begin. For the many who voted for an open future with their Kickstarter dollars , there’s a deep and personal sense of betrayal, despite Facebook’s promise to leave Oculus alone. The intensity of the reaction is because Oculus matters to people. It's new, it's different, it create
same-blog 2 0.95863473 517 high scalability-2009-02-21-Google AppEngine - A Second Look
Introduction: Update 6: : Back to the Future for Data Storage . We are in the middle of a renaissance in data storage with the application of many new ideas and techniques; there's huge potential for breaking out of thinking about data storage in just one way. Update 5 : Building Scalable Web Applications with Google App Engine by Brett Slatkin. Update 4 : Why Google App Engine is broken and what Google must do to fix it by Aral Balkan. We don't care that it can scale. We care that it does scale. And that it scales when you need it the most. Issues: 1MB limit on data structures; 1MB limit on data structures; the short-term high CPU quota; quotas in general; Admin? What's that? Update 3 : BigTable Blues . Catherine Devlin couldn't port an application to GAE because it can't do basic filtering and can't search 5,000 records without timing out: "Querying from 5000 records - too much for the mighty BigTable, apparently." Followup: not the future database . "90% of the work of this proje
3 0.95718473 1402 high scalability-2013-02-07-Ask HighScalability: Web asset server concept - 3rd party software available?
Introduction: We are serving dynamic (PHP) websites and their assets (JS/CSS/images/videos/binary downloads) via the same apache hosts. The static files are only being used as origins for CDN services used to distribute those files. Yet, in the current development-deploy pipeline, these files are checked into the same version control repositories as the code is. This is what we would like to change, for several reasons (decouple asset deployment from development & developers, lessen size of code repositories, etc.) My idea is to do the following: Set up a media server (cluster) which serves as an API (REST e.g.). You can PUT files to it, and get back the URL the file is available through from the public. In between input and output, the media service deals with everything that's necessary to serve the files: Upload them to the CDN, create the public URL, write the meta data to a (relational?) database, assign a version number... This API can be used by a) the application/website directly to provid
4 0.95332694 1516 high scalability-2013-09-13-Stuff The Internet Says On Scalability For September 13, 2013
Introduction: Hey, it's HighScalability time (this week is a fall harvest basket overflowing with good nutritious wisdom): ( Voyager: 36 yrs & 11B miles has reached interstellar space. So it begins... ) 170 million : metrics Twitter collects every minute; 350 million : Snapchat daily photo shares Quotable Quotes: @blowmage : OH: “Guys, databases don't know how to sort things. That's why NoSQL uses JavaScript.” Nokia insider : I look back and I think Nokia was just a very big company that started to maintain its position more than innovate for new opportunities. Paulo Siqueira : Ignoring scalability is not as bad as it sounds—if you use the proper tools. David Rosenthal : The relationship between diversity and risk is very complex. Jaime Teevan : the exact same result list will seem more relevant to you if it is returned just a fraction of a second faster. @aphyr : I use Redis as a queue #leaveDBalone Hey, I've
5 0.95306659 301 high scalability-2008-04-08-Google AppEngine - A First Look
Introduction: I haven't developed an AppEngine application yet, I'm just taking a look around their documentation and seeing what stands out for me. It's not the much speculated super cluster VM . AppEngine is solidly grounded in code and structure. It reminds me a little of the guy who ran a website out of S3 with a splash of Heroku thrown in as a chaser. The idea is clearly to take advantage of our massive multi-core future by creating a shared nothing infrastructure based firmly on a core set of infinitely scalable database, storage and CPU services. Don't forget Google also has a few other services to leverage: email, login, blogs, video, search, ads, metrics, and apps. A shared nothing request is a simple beast. By its very nature shared nothing architectures must be composed of services which are themselves already scalable and Google is signing up to supply that scalable infrastructure. Google has been busy creating a platform of out-of-the-box scalable services to build
6 0.95214766 851 high scalability-2010-07-02-Hot Scalability Links for July 2, 2010
7 0.95190078 1389 high scalability-2013-01-18-Stuff The Internet Says On Scalability For January 18, 2013
8 0.95152128 1612 high scalability-2014-03-14-Stuff The Internet Says On Scalability For March 14th, 2014
9 0.95047575 1649 high scalability-2014-05-16-Stuff The Internet Says On Scalability For May 16th, 2014
10 0.95022082 976 high scalability-2011-01-20-75% Chance of Scale - Leveraging the New Scaleogenic Environment for Growth
11 0.94963127 289 high scalability-2008-03-27-Amazon Announces Static IP Addresses and Multiple Datacenter Operation
12 0.94917297 1112 high scalability-2011-09-07-What Google App Engine Price Changes Say About the Future of Web Architecture
13 0.94906563 266 high scalability-2008-03-04-Manage Downtime Risk by Connecting Multiple Data Centers into a Secure Virtual LAN
15 0.94639689 789 high scalability-2010-03-05-Strategy: Planning for a Power Outage Google Style
16 0.94623297 1174 high scalability-2012-01-13-Stuff The Internet Says On Scalability For January 13, 2012
17 0.94561869 119 high scalability-2007-10-10-WAN Accelerate Your Way to Lightening Fast Transfers Between Data Centers
18 0.9452405 863 high scalability-2010-07-22-How can we spark the movement of research out of the Ivory Tower and into production?
19 0.9446407 888 high scalability-2010-08-27-OpenStack - The Answer to: How do We Compete with Amazon?
20 0.94412756 1117 high scalability-2011-09-16-Stuff The Internet Says On Scalability For September 16, 2011