high_scalability high_scalability-2007 high_scalability-2007-149 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Michael Nygard talks about Two Ways To Boost Your Flagging Web Site . The idea behind cache farms is to move memory devoted to the various caching layers into one large farm of caches, as with memcached. The idea behind read pools is to allocate your database read requests to a pool of dedicated read servers, thus offloading the write server. Using a combination of the strategies you aren't forced to scale up the database tier to scale your website.
sentIndex sentText sentNum sentScore
1 Michael Nygard talks about Two Ways To Boost Your Flagging Web Site . [sent-1, score-0.14]
2 The idea behind cache farms is to move memory devoted to the various caching layers into one large farm of caches, as with memcached. [sent-2, score-1.778]
3 The idea behind read pools is to allocate your database read requests to a pool of dedicated read servers, thus offloading the write server. [sent-3, score-2.406]
4 Using a combination of the strategies you aren't forced to scale up the database tier to scale your website. [sent-4, score-0.931]
wordName wordTfidf (topN-words)
[('nygard', 0.312), ('offloading', 0.312), ('devoted', 0.294), ('farms', 0.24), ('allocate', 0.235), ('boost', 0.219), ('forced', 0.214), ('behind', 0.21), ('read', 0.2), ('pools', 0.191), ('michael', 0.182), ('layers', 0.167), ('idea', 0.165), ('farm', 0.163), ('caches', 0.162), ('tier', 0.158), ('pool', 0.154), ('thus', 0.148), ('strategies', 0.142), ('talks', 0.14), ('dedicated', 0.131), ('combination', 0.127), ('ways', 0.118), ('various', 0.117), ('scale', 0.101), ('move', 0.097), ('website', 0.093), ('requests', 0.092), ('database', 0.088), ('caching', 0.084), ('cache', 0.083), ('write', 0.08), ('memory', 0.07), ('large', 0.056), ('servers', 0.052), ('web', 0.044), ('using', 0.034), ('one', 0.032)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 149 high scalability-2007-11-12-Scaling Using Cache Farms and Read Pooling
Introduction: Michael Nygard talks about Two Ways To Boost Your Flagging Web Site . The idea behind cache farms is to move memory devoted to the various caching layers into one large farm of caches, as with memcached. The idea behind read pools is to allocate your database read requests to a pool of dedicated read servers, thus offloading the write server. Using a combination of the strategies you aren't forced to scale up the database tier to scale your website.
2 0.14265357 360 high scalability-2008-08-04-A Bunch of Great Strategies for Using Memcached and MySQL Better Together
Introduction: The primero recommendation for speeding up a website is almost always to add cache and more cache. And after that add a little more cache just in case. Memcached is almost always given as the recommended cache to use. What we don't often hear is how to effectively use a cache in our own products. MySQL hosted two excellent webinars (referenced below) on the subject of how to deploy and use memcached. The star of the show, other than MySQL of course, is Farhan Mashraqi of Fotolog. You may recall we did an earlier article on Fotolog in Secrets to Fotolog's Scaling Success , which was one of my personal favorites. Fotolog, as they themselves point out, is probably the largest site nobody has ever heard of, pulling in more page views than even Flickr. Fotolog has 51 instances of memcached on 21 servers with 175G in use and 254G available. As a large successful photo-blogging site they have very demanding performance and scaling requirements. To meet those requirements they've developed a
3 0.11601724 367 high scalability-2008-08-17-Strategy: Drop Memcached, Add More MySQL Servers
Introduction: Update 2 : Michael Galpin in Cache Money and Cache Discussions likes memcached for it's expiry policy, complex graph data, process data, but says MySQL has many advantages: SQL, Uniform Data Access, Write-through, Read-through, Replication, Management, Cold starts, LRU eviction. Update: Dormando asks Should you use memcached? Should you just shard mysql more? . The idea of caching is the most important part of caching as it transports you beyond a simple CRUD worldview. Plan for caching and sharding by properly abstracting data access methods. Brace for change. Be ready to shard, be ready to cache. React and change to what you push out which is actually popular, vs over planning and wasting valuable time. Feedster's François Schiettecatte wonders if Fotolog's 21 memcached servers wouldn't be better used to further shard data by adding more MySQL servers? He mentions Feedster was able to drop memcached once they partitioned their data across more servers. The algorithm:
4 0.10383919 389 high scalability-2008-09-23-How to Scale with Ruby on Rails
Introduction: By George Palmer of 3dogsbark.com. Covers: * How you start out: shared hosting, web server DB on same machine. Move two 2 machines. Minimal code changes. * Scaling the database. Add read slaves on their own machines. Then master-master setup. Still minimal code changes. * Scaling the web server. Load balance against multiple application servers. Application servers scale but the database doesn't. * User clusters. Partition and allocate users to their own dedicated cluster. Requires substantial code changes. * Caching. A large percentage of hits are read only. Use reverse proxy, memcached, and language specific cache. * Elastic architectures. Based on Amazon EC2. Start and stop instances on demand. For global applications keep a cache on each continent, assign users to clusters by location, maintain app servers on each continent, use transaction replication software if you must replicate your site globally.
5 0.09836147 877 high scalability-2010-08-12-Designing Web Applications for Scalability
Introduction: I can’t even count the number of times that I’ve heard this phrase: “don’t worry about scaling your web application, worry about visitor (or customer) acquisition.” My response to this is always that you don’t need to choose one or the other, you can do both! In this post, I’m going to go over some of the strategies I’ve used to architect web applications for scalability, right from the start of the design process, in such a way that I’m prepared to scale when I need to, but not forced into doing so before its necessary. Easing the transition from small scale to large scale can be made much easier by choosing the right technologies and implementing the right coding patterns up front. You can read the full store here .
6 0.0981213 317 high scalability-2008-05-10-Hitting 300 SimbleDB Requests Per Second on a Small EC2 Instance
7 0.097035438 1372 high scalability-2012-12-14-Stuff The Internet Says On Scalability For December 14, 2012
8 0.091560066 90 high scalability-2007-09-12-Technology behind mediatemple grid service
9 0.080701292 182 high scalability-2007-12-12-Oracle Can Do Read-Write Splitting Too
10 0.080648787 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day
11 0.080167353 426 high scalability-2008-10-22-Server load balancing architectures, Part 1: Transport-level load balancing
12 0.077824041 665 high scalability-2009-07-29-Strategy: Let Google and Yahoo Host Your Ajax Library - For Free
13 0.07747636 436 high scalability-2008-11-02-Strategy: How to Manage Sessions Using Memcached
14 0.077273928 1266 high scalability-2012-06-18-Google on Latency Tolerant Systems: Making a Predictable Whole Out of Unpredictable Parts
15 0.075538158 373 high scalability-2008-08-29-Product: ScaleOut StateServer is Memcached on Steroids
16 0.075267598 532 high scalability-2009-03-11-Sharding and Connection Pools
17 0.074891873 662 high scalability-2009-07-27-Handle 700 Percent More Requests Using Squid and APC Cache
18 0.072996914 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
19 0.071747564 511 high scalability-2009-02-12-MySpace Architecture
20 0.070849121 263 high scalability-2008-02-27-Product: System Imager - Automate Deployment and Installs
topicId topicWeight
[(0, 0.104), (1, 0.049), (2, -0.028), (3, -0.067), (4, 0.004), (5, 0.036), (6, 0.011), (7, -0.027), (8, -0.054), (9, 0.006), (10, -0.018), (11, -0.038), (12, -0.012), (13, 0.06), (14, -0.054), (15, -0.043), (16, 0.021), (17, -0.044), (18, 0.013), (19, 0.02), (20, -0.034), (21, 0.049), (22, 0.024), (23, 0.005), (24, 0.0), (25, 0.014), (26, 0.022), (27, 0.028), (28, -0.015), (29, 0.003), (30, -0.001), (31, -0.012), (32, 0.025), (33, -0.003), (34, -0.023), (35, 0.041), (36, -0.001), (37, -0.025), (38, 0.036), (39, -0.019), (40, 0.056), (41, 0.016), (42, -0.04), (43, -0.056), (44, -0.033), (45, 0.03), (46, 0.019), (47, -0.03), (48, -0.01), (49, 0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.96858478 149 high scalability-2007-11-12-Scaling Using Cache Farms and Read Pooling
Introduction: Michael Nygard talks about Two Ways To Boost Your Flagging Web Site . The idea behind cache farms is to move memory devoted to the various caching layers into one large farm of caches, as with memcached. The idea behind read pools is to allocate your database read requests to a pool of dedicated read servers, thus offloading the write server. Using a combination of the strategies you aren't forced to scale up the database tier to scale your website.
2 0.7994163 602 high scalability-2009-05-17-Scaling Django Web Apps by Mike Malone
Introduction: Film buffs will recognize Django as a classic 1966 spaghetti western that spawned hundreds of imitators. Web heads will certainly first think of Django as the classic Python based Web framework that has also spawned hundreds of imitators and has become the gold standard framework for the web. Mike Malone, who worked on Pownce, a blogging tool now owned by Six Apart, tells in this very informative EuroDjangoCon presentation how Pownce scaled using Django in the real world. I was surprised to learn how large Pounce was: hundreds of requests/sec, thousands of DB operations/sec, millions of user relationships, millions of notes, and terabytes of static data. Django has a lot of functionality in the box to help you scale, but if you want to scale large it turns out Django has some limitations and Mike tells you what these are and also provides some code to get around them. Mike's talk-although Django specific--will really help anyone creating applications on the web. There's
3 0.77963501 359 high scalability-2008-07-29-Ehcache - A Java Distributed Cache
Introduction: Ehcache is a pure Java cache with the following features: fast, simple, small foot print, minimal dependencies, provides memory and disk stores for scalability into gigabytes, scalable to hundreds of caches is a pluggable cache for Hibernate, tuned for high concurrent load on large multi-cpu servers, provides LRU, LFU and FIFO cache eviction policies, and is production tested. Ehcache is used by LinkedIn to cache member profiles. The user guide says it's possible to get at 2.5 times system speedup for persistent Object Relational Caching, a 1000 times system speedup for Web Page Caching, and a 1.6 times system speedup Web Page Fragment Caching. From the website: Introduction Ehcache is a cache library. Before getting into ehcache, it is worth stepping back and thinking about caching generally. About Caches Wiktionary defines a cache as A store of things that will be required in future, and can be retrieved rapidly . That is the nub of it. In computer science terms, a cac
4 0.77600968 360 high scalability-2008-08-04-A Bunch of Great Strategies for Using Memcached and MySQL Better Together
Introduction: The primero recommendation for speeding up a website is almost always to add cache and more cache. And after that add a little more cache just in case. Memcached is almost always given as the recommended cache to use. What we don't often hear is how to effectively use a cache in our own products. MySQL hosted two excellent webinars (referenced below) on the subject of how to deploy and use memcached. The star of the show, other than MySQL of course, is Farhan Mashraqi of Fotolog. You may recall we did an earlier article on Fotolog in Secrets to Fotolog's Scaling Success , which was one of my personal favorites. Fotolog, as they themselves point out, is probably the largest site nobody has ever heard of, pulling in more page views than even Flickr. Fotolog has 51 instances of memcached on 21 servers with 175G in use and 254G available. As a large successful photo-blogging site they have very demanding performance and scaling requirements. To meet those requirements they've developed a
5 0.77027833 192 high scalability-2007-12-25-IBMer Says LAMP Can't Scale
Introduction: A very entertaining and somewhat educational article on IBM Poopheads say LAMP Users Need to "grow up" . The physical three tier architecture turns out to be the root of all evil and shared nothing architectures brings simplicity and light. In the comments Simon Willison makes an insightful comment on why fine grained caching works for personalized pages and proxy's don't: Great post, but I have to disagree with you on the finely grained caching part. If you look at big LAMP deployments such as Flickr, LiveJournal and Facebook the common technology component that enables them to scale is memcached - a tool for finely grained caching. That's not to say that they aren't doing shared-nothing, it's just that memcached is critical for helping the database layer scale. LiveJournal serves around 50% of its page views "permission controlled" (friends only) so an HTTP proxy on the front end isn't the right solution - but memcached reduces their database hits by 90%.
6 0.76577431 436 high scalability-2008-11-02-Strategy: How to Manage Sessions Using Memcached
7 0.76102608 1346 high scalability-2012-10-24-Saving Cash Using Less Cache - 90% Savings in the Caching Tier
8 0.75556648 836 high scalability-2010-06-04-Strategy: Cache Larger Chunks - Cache Hit Rate is a Bad Indicator
9 0.74155313 927 high scalability-2010-10-26-Marrying memcached and NoSQL
10 0.73488623 1467 high scalability-2013-05-30-Google Finds NUMA Up to 20% Slower for Gmail and Websearch
11 0.72680706 248 high scalability-2008-02-13-What's your scalability plan?
12 0.72506887 911 high scalability-2010-09-30-More Troubles with Caching
13 0.7249614 174 high scalability-2007-12-05-Product: Tugela Cache
14 0.70613235 662 high scalability-2009-07-27-Handle 700 Percent More Requests Using Squid and APC Cache
15 0.69927573 673 high scalability-2009-08-07-Strategy: Break Up the Memcache Dog Pile
17 0.6954869 467 high scalability-2008-12-16-[ANN] New Open Source Cache System
18 0.68964589 577 high scalability-2009-04-22-Gear6 Web cache - the hardware solution for working with Memcache
19 0.6861577 603 high scalability-2009-05-19-Scaling Memcached: 500,000+ Operations-Second with a Single-Socket UltraSPARC T2
20 0.67738509 701 high scalability-2009-09-10-When optimizing - don't forget the Java Virtual Machine (JVM)
topicId topicWeight
[(1, 0.16), (2, 0.279), (3, 0.237), (61, 0.123), (85, 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 0.91193879 149 high scalability-2007-11-12-Scaling Using Cache Farms and Read Pooling
Introduction: Michael Nygard talks about Two Ways To Boost Your Flagging Web Site . The idea behind cache farms is to move memory devoted to the various caching layers into one large farm of caches, as with memcached. The idea behind read pools is to allocate your database read requests to a pool of dedicated read servers, thus offloading the write server. Using a combination of the strategies you aren't forced to scale up the database tier to scale your website.
2 0.88418198 1194 high scalability-2012-02-16-A Super Short on the Youporn Stack - 300K QPS and 100 Million Page Views Per Day
Introduction: Eric Pickup from Youporn.com posted on a news group that Youporn is now 100% Redis based and will soon be revealing more about their architecture at the ConFoo conference . Some stunning, but not surprising numbers were revealed: 100 million page views per day A cluster of Redis slaves are handling over 300k queries per second. Some additional nuggets: Additional Redis nodes were added because the network cards couldn't keep up with Redis. Impressed with Redis' performance. All reads come from Redis; we are maintaining MySQL just to allow us to build new sorted sets as our requirement change Most data is found in hashes with ordered sets used to know what data to show. A typical lookup would be an zInterStore on: videos:filters:released, Videos:filters:orientation:straight,Videos:filters:categories:{category_id}, Videos:ordering:rating Then perform a zRange to get the pages we want and get the list of video_ids back. Then start a pipeline and get
3 0.86189216 926 high scalability-2010-10-24-Hot Scalability Links For Oct 24, 2010
Introduction: On a cold and rainy Fall day, a day stolen from winter rather than our usual gorgeous Indian Summers , a day not even the SF Giants winning the pennant can help warm, here are some hot links to read by a digital flame: Using MySQL as a NoSQL - A story for exceeding 750,000 qps on a commodity server by Yoshinori Matsunobu. Wonderfully detailed post on how you can lookup a row by ID really fast if you bypass all the typical MySQL query parsing overhead. Minecraftwiki.net and minecraftforum.net now serve more traffic than Slashdot and Stackoverflow! 1 million pageviews and 100k uniques per day, per site; 10TB of bandwidth a month; 4+ machines running Varnish, HAProxy, PHP, MySQL, Nginx. Stuff the Internet Says: @ old_sound : Somebody make me a t-shirt that says "I've read the CAP theorem and I liked it" @dscape : How relevant do I think the CAP theorem is? Not at all. I honestly hate conversations where anyone talks about crap.. cap, sorry. @humidbei
Introduction: Awesome paper on how particular synchronization mechanisms scale on multi-core architectures: Everything You Always Wanted to Know About Synchronization but Were Afraid to Ask . The goal is to pick a locking approach that doesn't degrade as the number of cores increase. Like everything else in life, that doesn't appear to be generically possible: None of the nine locking schemes we consider consistently outperforms any other one, on all target architectures or workloads. Strictly speaking, to seek optimality, a lock algorithm should thus be selected based on the hardware platform and the expected workload . Abstract: This paper presents the most exhaustive study of synchronization to date. We span multiple layers, from hardware cache-coherence protocols up to high-level concurrent software. We do so on different types architectures, from single-socket – uniform and nonuniform – to multi-socket – directory and broadcastbased – many-cores. We draw a set of observations t
5 0.84542966 811 high scalability-2010-04-16-Hot Scalability Links for April 16, 2010
Introduction: Twitter gets a total of 3 billion requests a day via its API ; 105,779,710 registered users; 300,000 new registered users a day; 180 million unique visitors a month; 55 million tweets a day. Who has the most servers? Google 1 million+; Intel 100K; 1&1 Internet 70K; Facebook 30K; Akamai 61K; Rackspace 56k+. Cloud Computing Economies of Scale . James Hamilton gives a fabulous talk breaking down where the costs are in the cloud. It's not where you may think. Higher utilization is the key. More here . Erlang Factory: Andy Gross: Distributed Erlang Systems In Operation: Patterns and Pitfalls by Martin J. Logan. Great overview of architecting distributed systems in Erlang. Covers what you want and don't want in a distributed system and how to compromise those elements, what's common, system design, cluster membership, load balancing, upgrades, debugging, and more. Extreme Scale Computing by Irving Wladawsky-Berger . “An exascale supercomputer capable of a million tr
6 0.83752918 1178 high scalability-2012-01-20-Stuff The Internet Says On Scalability For January 20, 2012
7 0.83378404 189 high scalability-2007-12-21-Strategy: Limit Result Sets
8 0.82320154 365 high scalability-2008-08-16-Strategy: Serve Pre-generated Static Files Instead Of Dynamic Pages
9 0.81489074 248 high scalability-2008-02-13-What's your scalability plan?
10 0.8138901 476 high scalability-2008-12-28-How to Organize a Database Table’s Keys for Scalability
11 0.81365687 801 high scalability-2010-03-30-Running Large Graph Algorithms - Evaluation of Current State-of-the-Art and Lessons Learned
12 0.80382478 754 high scalability-2009-12-22-Incremental deployment
13 0.80289245 877 high scalability-2010-08-12-Designing Web Applications for Scalability
14 0.79886603 1250 high scalability-2012-05-23-Averages, web performance data, and how your analytics product is lying to you
15 0.79665154 1231 high scalability-2012-04-20-Stuff The Internet Says On Scalability For April 20, 2012
16 0.79636562 240 high scalability-2008-02-05-Handling of Session for a site running from more than 1 data center
17 0.79519492 764 high scalability-2010-01-25-Designing applications for cloud deployment
18 0.79516852 568 high scalability-2009-04-14-Designing a Scalable Twitter
19 0.79275167 64 high scalability-2007-08-10-How do we make a large real-time search engine?
20 0.79261893 821 high scalability-2010-05-03-MocoSpace Architecture - 3 Billion Mobile Page Views a Month