high_scalability high_scalability-2008 high_scalability-2008-389 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: By George Palmer of 3dogsbark.com. Covers: * How you start out: shared hosting, web server DB on same machine. Move two 2 machines. Minimal code changes. * Scaling the database. Add read slaves on their own machines. Then master-master setup. Still minimal code changes. * Scaling the web server. Load balance against multiple application servers. Application servers scale but the database doesn't. * User clusters. Partition and allocate users to their own dedicated cluster. Requires substantial code changes. * Caching. A large percentage of hits are read only. Use reverse proxy, memcached, and language specific cache. * Elastic architectures. Based on Amazon EC2. Start and stop instances on demand. For global applications keep a cache on each continent, assign users to clusters by location, maintain app servers on each continent, use transaction replication software if you must replicate your site globally.
sentIndex sentText sentNum sentScore
1 Covers: * How you start out: shared hosting, web server DB on same machine. [sent-3, score-0.29]
2 Partition and allocate users to their own dedicated cluster. [sent-14, score-0.371]
3 For global applications keep a cache on each continent, assign users to clusters by location, maintain app servers on each continent, use transaction replication software if you must replicate your site globally. [sent-22, score-1.208]
wordName wordTfidf (topN-words)
[('continent', 0.475), ('minimal', 0.285), ('palmer', 0.258), ('george', 0.195), ('allocate', 0.179), ('assign', 0.164), ('substantial', 0.164), ('percentage', 0.162), ('slaves', 0.146), ('code', 0.141), ('globally', 0.141), ('replicate', 0.14), ('covers', 0.139), ('start', 0.139), ('reverse', 0.138), ('proxy', 0.133), ('hits', 0.123), ('elastic', 0.118), ('location', 0.118), ('balance', 0.116), ('partition', 0.116), ('stop', 0.112), ('db', 0.109), ('maintain', 0.108), ('transaction', 0.107), ('hosting', 0.104), ('clusters', 0.104), ('read', 0.101), ('scaling', 0.101), ('master', 0.1), ('dedicated', 0.099), ('users', 0.093), ('specific', 0.091), ('instances', 0.087), ('language', 0.087), ('shared', 0.084), ('requires', 0.082), ('replication', 0.08), ('memcached', 0.079), ('servers', 0.079), ('global', 0.078), ('application', 0.076), ('move', 0.074), ('add', 0.074), ('app', 0.07), ('web', 0.067), ('must', 0.065), ('cache', 0.063), ('amazon', 0.059), ('site', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 389 high scalability-2008-09-23-How to Scale with Ruby on Rails
Introduction: By George Palmer of 3dogsbark.com. Covers: * How you start out: shared hosting, web server DB on same machine. Move two 2 machines. Minimal code changes. * Scaling the database. Add read slaves on their own machines. Then master-master setup. Still minimal code changes. * Scaling the web server. Load balance against multiple application servers. Application servers scale but the database doesn't. * User clusters. Partition and allocate users to their own dedicated cluster. Requires substantial code changes. * Caching. A large percentage of hits are read only. Use reverse proxy, memcached, and language specific cache. * Elastic architectures. Based on Amazon EC2. Start and stop instances on demand. For global applications keep a cache on each continent, assign users to clusters by location, maintain app servers on each continent, use transaction replication software if you must replicate your site globally.
2 0.14045875 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
Introduction: Skype uses PostgreSQL as their backend database . PostgreSQL doesn't get enough run in the database world so I was excited to see how PostgreSQL is used "as the main DB for most of [Skype's] business needs." Their approach is to use a traditional stored procedure interface for accessing data and on top of that layer proxy servers which hash SQL requests to a set of database servers that actually carry out queries. The result is a horizontally partitioned system that they think will scale to handle 1 billion users. Skype's goal is an architecture that can handle 1 billion plus users. This level of scale isn't practically solvable with one really big computer, so our masked superhero horizontal scaling comes to the rescue. Hardware is dual or quad Opterons with SCSI RAID. Followed common database progression: Start with one DB. Add new databases partitioned by functionality. Replicate read-mostly data for better read access. Then horizontally partition data across multiple nod
3 0.13370824 11 high scalability-2007-07-15-Coyote Point Load Balancing Systems
Introduction: Appliances that: Ensures Non-Stop application availability Improves network and server maintainability Delivers Enterprise-grade gigabit content switching Offers true Application Acceleration Provides maximum throughput at minimal cost
4 0.12207219 360 high scalability-2008-08-04-A Bunch of Great Strategies for Using Memcached and MySQL Better Together
Introduction: The primero recommendation for speeding up a website is almost always to add cache and more cache. And after that add a little more cache just in case. Memcached is almost always given as the recommended cache to use. What we don't often hear is how to effectively use a cache in our own products. MySQL hosted two excellent webinars (referenced below) on the subject of how to deploy and use memcached. The star of the show, other than MySQL of course, is Farhan Mashraqi of Fotolog. You may recall we did an earlier article on Fotolog in Secrets to Fotolog's Scaling Success , which was one of my personal favorites. Fotolog, as they themselves point out, is probably the largest site nobody has ever heard of, pulling in more page views than even Flickr. Fotolog has 51 instances of memcached on 21 servers with 175G in use and 254G available. As a large successful photo-blogging site they have very demanding performance and scaling requirements. To meet those requirements they've developed a
5 0.12047276 63 high scalability-2007-08-09-Lots of questions for high scalability - high availability
Introduction: Hey, I do have a website that I would like to scale. Right now we have 10 servers but this does not scale well. I know how to deal with my apache web servers but have problems with sql servers. I would like to use the "scale out" system and add servers when we need. We have over 100Gb of data for mysql and we tried to have around 20G per server. It works well except that if a server goes down then 1/5 of the user can't access the website. We could use replication but we would need to at least double sql servers to replicate each server. And maybe in the future it's not gonna be enough we would need maybe 3 slaves per master ... well I don't really like this idea. I would prefer to have 8 servers that all deal with data from the 5 servers we have right now and then we could add new servers when we need. I looked at NFS but that does not seem to be a good idea for SQL servers ? Can you confirm?
6 0.11056612 391 high scalability-2008-09-23-The 7 Stages of Scaling Web Apps
7 0.11031376 276 high scalability-2008-03-15-New Website Design Considerations
8 0.10971119 623 high scalability-2009-06-10-Dealing with multi-partition transactions in a distributed KV solution
9 0.10470539 72 high scalability-2007-08-22-Wikimedia architecture
10 0.10383919 149 high scalability-2007-11-12-Scaling Using Cache Farms and Read Pooling
11 0.10270455 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
12 0.097986713 315 high scalability-2008-05-05-HSCALE - Handling 200 Million Transactions Per Month Using Transparent Partitioning With MySQL Proxy
13 0.09512271 915 high scalability-2010-10-05-Sponsored Post: Box.net, Wiredrive, Joyent, DeviantART, CloudSigma, ManageEngine, Site24x7
14 0.094865039 392 high scalability-2008-09-24-Building a Scalable Architecture for Web Apps
15 0.094718724 7 high scalability-2007-07-12-FeedBurner Architecture
16 0.093371548 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
17 0.092510037 1345 high scalability-2012-10-22-Spanner - It's About Programmers Building Apps Using SQL Semantics at NoSQL Scale
18 0.089422017 1501 high scalability-2013-08-13-In Memoriam: Lavabit Architecture - Creating a Scalable Email Service
19 0.088900037 619 high scalability-2009-06-05-HotPads Shows the True Cost of Hosting on Amazon
20 0.087631643 824 high scalability-2010-05-06-Going global on EC2
topicId topicWeight
[(0, 0.155), (1, 0.056), (2, -0.013), (3, -0.12), (4, 0.008), (5, -0.011), (6, 0.033), (7, -0.139), (8, -0.009), (9, -0.011), (10, -0.027), (11, -0.002), (12, -0.026), (13, 0.029), (14, -0.037), (15, -0.023), (16, 0.01), (17, -0.013), (18, 0.041), (19, 0.033), (20, 0.022), (21, 0.024), (22, -0.027), (23, -0.034), (24, 0.026), (25, 0.016), (26, 0.047), (27, -0.001), (28, 0.034), (29, 0.004), (30, 0.041), (31, -0.047), (32, -0.072), (33, -0.026), (34, -0.012), (35, -0.039), (36, -0.011), (37, -0.042), (38, 0.012), (39, 0.027), (40, 0.035), (41, 0.013), (42, -0.081), (43, -0.067), (44, -0.004), (45, 0.015), (46, -0.016), (47, -0.02), (48, -0.035), (49, -0.012)]
simIndex simValue blogId blogTitle
same-blog 1 0.96668088 389 high scalability-2008-09-23-How to Scale with Ruby on Rails
Introduction: By George Palmer of 3dogsbark.com. Covers: * How you start out: shared hosting, web server DB on same machine. Move two 2 machines. Minimal code changes. * Scaling the database. Add read slaves on their own machines. Then master-master setup. Still minimal code changes. * Scaling the web server. Load balance against multiple application servers. Application servers scale but the database doesn't. * User clusters. Partition and allocate users to their own dedicated cluster. Requires substantial code changes. * Caching. A large percentage of hits are read only. Use reverse proxy, memcached, and language specific cache. * Elastic architectures. Based on Amazon EC2. Start and stop instances on demand. For global applications keep a cache on each continent, assign users to clusters by location, maintain app servers on each continent, use transaction replication software if you must replicate your site globally.
2 0.75492644 602 high scalability-2009-05-17-Scaling Django Web Apps by Mike Malone
Introduction: Film buffs will recognize Django as a classic 1966 spaghetti western that spawned hundreds of imitators. Web heads will certainly first think of Django as the classic Python based Web framework that has also spawned hundreds of imitators and has become the gold standard framework for the web. Mike Malone, who worked on Pownce, a blogging tool now owned by Six Apart, tells in this very informative EuroDjangoCon presentation how Pownce scaled using Django in the real world. I was surprised to learn how large Pounce was: hundreds of requests/sec, thousands of DB operations/sec, millions of user relationships, millions of notes, and terabytes of static data. Django has a lot of functionality in the box to help you scale, but if you want to scale large it turns out Django has some limitations and Mike tells you what these are and also provides some code to get around them. Mike's talk-although Django specific--will really help anyone creating applications on the web. There's
3 0.75370032 391 high scalability-2008-09-23-The 7 Stages of Scaling Web Apps
Introduction: By John Engales CTO, Rackspace. Good presentation of the stages a typical successful website goes through: Stage 1 - The Beginning: Simple architecture, low complexity. no redundancy. Firewall, load balancer, a pair of web servers, database server, and internal storage. Stage 2 - More of the same, just bigger. Stage 3 - The Pain Begins: publicity hits. Use reverse proxy, cache static content, load balancers, more databases, re-coding. Stage 4 - The Pain Intensifies: caching with memcached, writes overload and replication takes too long, start database partitioning, shared storage makes sense for content, significant re-architecting for DB. Stage 5 - This Really Hurts!: rethink entire application, partition on geography user ID, etc, create user clusters, using hashing scheme for locating which user belongs to which cluster. Stage 6 - Getting a little less painful: scalable application and database architecture, acceptable performance, starting to add ne features again, op
4 0.72694761 7 high scalability-2007-07-12-FeedBurner Architecture
Introduction: FeedBurner is a news feed management provider launched in 2004. FeedBurner provides custom RSS feeds and management tools to bloggers, podcasters, and other web-based content publishers. Services provided to publishers include traffic analysis and an optional advertising system. Site: http://www.feedburner.com Information Sources FeedBurner - Scalable Web Applications using MySQL and Java What the Web’s most popular sites are running on Platform Java MySQL Hibernate Spring Tomcat Cacti Load balancing: NetScaler Application Switches Routers, switches: HP, Cisco DNS: bind The Stats FeedBurner is growing faster than MySpace and Digg with 385% traffic growth. Total feeds: 808,707, Number of publishers: 471,686. 11 million subscribers in 190 countries Scaling History - July 2004: 300Kbps, 5,600 feeds, 3 app servers, 3 web servers 2 DB servers, Round Robin DNS - April 2005: 5Mbps, 47,700 feeds, 6 app servers, 6 web servers (same mac
5 0.71439523 637 high scalability-2009-06-24-Habits of Highly Scalable Web Applications
Introduction: Nick Belhomme wrote up a excellent summary of a talk given by Eli White on building scalable web applications. Eli worked at digg.com and is now the PHP Community Manager & DevZone Editor-in-Chief at Zend Technologies. Eli takes us on a grand tour through various proven scaling strategies. On the trip you'll visit: What is scalable application design Tip 1: load balancing the webserver Tip 2: scaling from a single DB server to a Master-Slave setup Tip 3: Partitioning, Vertical DB Scaling Tip 4: Partitioning, horizontal DB Scaling Tip 5: Application Level Partitioning Tip 6: Caching to get around your database Resources
6 0.70894372 392 high scalability-2008-09-24-Building a Scalable Architecture for Web Apps
7 0.69314837 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
8 0.66944063 798 high scalability-2010-03-22-7 Secrets to Successfully Scaling with Scalr (on Amazon) by Sebastian Stadil
9 0.6662237 928 high scalability-2010-10-26-Scaling DISQUS to 75 Million Comments and 17,000 RPS
11 0.66331029 1188 high scalability-2012-02-06-The Design of 99designs - A Clean Tens of Millions Pageviews Architecture
12 0.66322881 68 high scalability-2007-08-20-TypePad Architecture
13 0.66201007 1646 high scalability-2014-05-12-4 Architecture Issues When Scaling Web Applications: Bottlenecks, Database, CPU, IO
14 0.66105306 302 high scalability-2008-04-10-Mysql scalability and failover...
15 0.6597051 149 high scalability-2007-11-12-Scaling Using Cache Farms and Read Pooling
16 0.65577364 1593 high scalability-2014-02-10-13 Simple Tricks for Scaling Python and Django with Apache from HackerEarth
17 0.65476185 81 high scalability-2007-09-06-Scaling IMAP and POP3
18 0.65330458 554 high scalability-2009-04-04-Digg Architecture
19 0.65159231 72 high scalability-2007-08-22-Wikimedia architecture
20 0.64859271 1260 high scalability-2012-06-07-Case Study on Scaling PaaS infrastructure
topicId topicWeight
[(1, 0.212), (2, 0.082), (10, 0.108), (61, 0.067), (79, 0.123), (94, 0.107), (97, 0.181)]
simIndex simValue blogId blogTitle
same-blog 1 0.87066835 389 high scalability-2008-09-23-How to Scale with Ruby on Rails
Introduction: By George Palmer of 3dogsbark.com. Covers: * How you start out: shared hosting, web server DB on same machine. Move two 2 machines. Minimal code changes. * Scaling the database. Add read slaves on their own machines. Then master-master setup. Still minimal code changes. * Scaling the web server. Load balance against multiple application servers. Application servers scale but the database doesn't. * User clusters. Partition and allocate users to their own dedicated cluster. Requires substantial code changes. * Caching. A large percentage of hits are read only. Use reverse proxy, memcached, and language specific cache. * Elastic architectures. Based on Amazon EC2. Start and stop instances on demand. For global applications keep a cache on each continent, assign users to clusters by location, maintain app servers on each continent, use transaction replication software if you must replicate your site globally.
2 0.86235505 456 high scalability-2008-12-01-Sun's High-Performance and Reliable Web Proxy Solution
Introduction: As individuals and businesses depend on the Web more than ever to conduct business, rapid and reliable content retrieval is critical. Reducing wait time improves productivity and increases user satisfaction. Web proxy technology has emerged as an effective solution to improve performance, help ensure content availability and enhance network security by caching and filtering Web content. The combination of Sun SPARC Enterprise servers with CoolThreads technology and the Sun Java System Web Proxy Server software provides a compelling foundation for a robust Web proxy solution. Sun SPARC Enterprise T1000 and T2000 servers include the UltraSPARC T1 processor with CoolThreads technology, offering six or eight cores with four threads per core. The Sun Java System Web Proxy Server software is highly threaded and takes advantage of the large number of threads supported by Sun UltraSPARC T1 processors with CoolThreads technology. Together, these products provide a highly scalable solution that
3 0.83583176 1347 high scalability-2012-10-25-Not All Regions are Created Equal - South America Es Bueno
Introduction: Rodrigo Campos shared some interesting benchmark results for AWS instances in the South America Region (SĂŁo Paulo) and US East Region (North Virginia). He summarized the results in this thread on the Guerrilla Capacity Planning list: The same instance type shows a different behavior depending on the region that it is running, this is particularly critical if you depend on multiple regions for disaster recovery or geographical load balancing Generally speaking performance is better and more consistent in the South America region when compared to Virginia, this is probably due to the fact that SA region was the latest to be deployed. Maybe the SA datacenter uses new server models or it is just underutilized, but this is a wild guess Write performance for the medium instance type in Virginia abruptly decays, dropping from almost 400 call/s to something around 300 calls/s, this is not very clear in the scatter plot but if you draw a time-based chart you can clearly see this pa
4 0.79504246 583 high scalability-2009-04-26-Scale-up vs. Scale-out: A Case Study by IBM using Nutch-Lucene
Introduction: Scale-up solutions in the form of large SMPs have represented the mainstream of commercial computing for the past several years. The major server vendors continue to provide increasingly larger and more powerful machines. More recently, scale-out solutions, in the form of clusters of smaller machines, have gained increased acceptance for commercial computing. Scale-out solutions are particularly effective in high-throughput web-centric applications. In this paper, we investigate the behavior of two competing approaches to parallelism, scale-up and scale-out, in an emerging search application. Our conclusions show that a scale-out strategy can be the key to good performance even on a scale-up machine. Furthermore, scale-out solutions offer better price/performance, although at an increase in management complexity. Read more about scaling out/up and about the conclusions here (PDF - you can also download it)
5 0.79161239 1160 high scalability-2011-12-21-In Memory Data Grid Technologies
Introduction: After winning a CSC Leading Edge Forum (LEF) research grant, I (Paul Colmer) wanted to publish some of the highlights of my research to share with the wider technology community. What is an In Memory Data Grid? It is not an in-memory relational database, a NOSQL database or a relational database. It is a different breed of software datastore. In summary an IMDG is an ‘off the shelf’ software product that exhibits the following characteristics: The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. All servers can be active in each site. All data is stored in the RAM of the servers. Servers can be added or removed non-disruptively, to increase the amount of RAM available. The data model is non-relational and is object-based. Distributed applications written on the .NET and Java application platforms are s
6 0.78672248 450 high scalability-2008-11-24-Scalability Perspectives #3: Marc Andreessen – Internet Platforms
7 0.78625548 458 high scalability-2008-12-01-Web Consolidation on the Sun Fire T1000 using Solaris Containers
8 0.78616273 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day
9 0.78544754 706 high scalability-2009-09-16-The VeriScale Architecture - Elasticity and efficiency for private clouds
10 0.7827968 888 high scalability-2010-08-27-OpenStack - The Answer to: How do We Compete with Amazon?
11 0.78182065 239 high scalability-2008-02-04-Streaming Video on Amazon EC2?
12 0.78129262 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
13 0.78067738 1216 high scalability-2012-03-27-Big Data In the Cloud Using Cloudify
14 0.77975339 1056 high scalability-2011-06-09-Retrospect on recent AWS outage and Resilient Cloud-Based Architecture
15 0.77649856 688 high scalability-2009-08-26-Hot Links for 2009-8-26
16 0.77266419 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
17 0.77250135 355 high scalability-2008-07-21-Eucalyptus - Build Your Own Private EC2 Cloud
18 0.77207649 184 high scalability-2007-12-13-Amazon SimpleDB - Scalable Cloud Database
19 0.76943368 1448 high scalability-2013-04-29-AWS v GCE Face-off and Why Innovation Needs Lower Cost Infrastructures
20 0.76896363 444 high scalability-2008-11-14-Private-Public Cloud