high_scalability high_scalability-2011 high_scalability-2011-1121 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Sean Hull with two helpful posts: 5 Things That are Toxic to Scalability : Object Relational Mappers. Create complex queries that hard to optimize and tweak. Synchronous, Serial, Coupled or Locking Processes. Locks are like stop signs, traffic circles keep the traffic flowing. Row level locking is better than table level locking. Use async replication. Use eventual consistency for clusters. One Copy of Your Database. A single database server is a choke point. Create parallel databases and let a driver select between them. Having No Metrics. Visualize what's happening to your system using one of the many monitoring packages. Lack of Feature Flags. Be able to turn off features via a flag so when a spike hits features can be turned off to reduce load. 3 Ways to Boost Cloud Scalability : Use Auto-scaling. Spin-up new instances when a threshold is passed and back down again when traffic drops. Horizontally Scale the Database Tier. MySQL in a master
sentIndex sentText sentNum sentScore
1 Sean Hull with two helpful posts: 5 Things That are Toxic to Scalability : Object Relational Mappers. [sent-1, score-0.107]
2 Create complex queries that hard to optimize and tweak. [sent-2, score-0.08]
3 Locks are like stop signs, traffic circles keep the traffic flowing. [sent-4, score-0.586]
4 Row level locking is better than table level locking. [sent-5, score-0.551]
5 Create parallel databases and let a driver select between them. [sent-10, score-0.211]
6 Visualize what's happening to your system using one of the many monitoring packages. [sent-12, score-0.093]
7 Be able to turn off features via a flag so when a spike hits features can be turned off to reduce load. [sent-14, score-0.736]
8 Spin-up new instances when a threshold is passed and back down again when traffic drops. [sent-16, score-0.406]
9 MySQL in a master-master active passive cluster configuration. [sent-18, score-0.238]
10 Scale up to a larger EC2 instance is the load on a single master is a problem. [sent-21, score-0.219]
11 For better EBS performance use Linux's software raid technology. [sent-23, score-0.194]
12 EBS already has redundancy built in so you can stripe across a number of EBS volumes. [sent-24, score-0.27]
13 Please read the original articles for the full details. [sent-27, score-0.164]
wordName wordTfidf (topN-words)
[('ebs', 0.42), ('locking', 0.212), ('striped', 0.201), ('circles', 0.185), ('toxic', 0.179), ('stripe', 0.17), ('choke', 0.162), ('passive', 0.159), ('flag', 0.159), ('traffic', 0.157), ('signs', 0.157), ('sean', 0.148), ('serial', 0.135), ('threshold', 0.132), ('brief', 0.13), ('boost', 0.13), ('coupled', 0.124), ('async', 0.122), ('spike', 0.121), ('passed', 0.117), ('driver', 0.116), ('roll', 0.114), ('grows', 0.109), ('helpful', 0.107), ('row', 0.107), ('eventual', 0.105), ('raid', 0.1), ('redundancy', 0.1), ('level', 0.095), ('hits', 0.095), ('select', 0.095), ('use', 0.094), ('features', 0.094), ('posts', 0.093), ('happening', 0.093), ('turned', 0.093), ('alternative', 0.092), ('stop', 0.087), ('original', 0.087), ('optimize', 0.08), ('turn', 0.08), ('active', 0.079), ('articles', 0.077), ('master', 0.077), ('object', 0.077), ('copy', 0.076), ('better', 0.075), ('table', 0.074), ('larger', 0.071), ('single', 0.071)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1121 high scalability-2011-09-21-5 Scalability Poisons and 3 Cloud Scalability Antidotes
Introduction: Sean Hull with two helpful posts: 5 Things That are Toxic to Scalability : Object Relational Mappers. Create complex queries that hard to optimize and tweak. Synchronous, Serial, Coupled or Locking Processes. Locks are like stop signs, traffic circles keep the traffic flowing. Row level locking is better than table level locking. Use async replication. Use eventual consistency for clusters. One Copy of Your Database. A single database server is a choke point. Create parallel databases and let a driver select between them. Having No Metrics. Visualize what's happening to your system using one of the many monitoring packages. Lack of Feature Flags. Be able to turn off features via a flag so when a spike hits features can be turned off to reduce load. 3 Ways to Boost Cloud Scalability : Use Auto-scaling. Spin-up new instances when a threshold is passed and back down again when traffic drops. Horizontally Scale the Database Tier. MySQL in a master
2 0.44003043 1473 high scalability-2013-06-10-The 10 Deadly Sins Against Scalability
Introduction: In the moral realm there may be 7 deadly sins, but scalability maven Sean Hull has come up Five More Things Deadly to Scalability that when added to his earlier 5 Things That are Toxic to Scalability , make for a numerologically satisfying 10 sins again scalability: Slow Disk I/O – RAID 5 – Multi-tenant EBS . Use RAID 10, it provides good protection along with good read and write performance. The design of RAID 5 means poor performance and long repair times on failure. On AWS consider Provisioned IOPS as a way around IO bottlenecks. Using the database for Queuing. The database may seem like the perfect place to keep work queues, but under load locking and scanning overhead kills performance. Use specialized products like RabbitMQ and SQS to remove this bottleneck. Using Database for full-text searching. Search seems like another perfect database feature. At scale search doesn't perform well. Use specialized technologies like Solr or Sphinx. Insufficient Caching
3 0.3176496 1631 high scalability-2014-04-14-How do you even do anything without using EBS?
Introduction: In a recent thread on Hacker News discussing recent AWS price changes , seldo mentioned they use AWS for business, they just never use EBS on AWS. A good question was asked: How do you even do anything without using EBS? Amazon certainly makes using EBS the easiest path. And EBS has a better reliability record as of late, but it's still often recommended to not use EBS. This avoids a single point of failure at the cost of a lot of complexity, though as AWS uses EBS internally, not using EBS may not save you if you use other AWS services like RDS or ELB. If you don't want to use EBS, it's hard to know where to even start. A dilemma to which Kevin Nuckolls gives a great answer : Well, you break your services out onto stateless and stateful machines. After that, you make sure that each of your stateful services is resilient to individual node failure. I prefer to believe that if you can't roll your entire infrastructure over to new nodes monthly then you're unprepared fo
4 0.21085414 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ), you can follow him on twitter . How do you scale an AWS (Amazon Web Services) infrastructure? This article will give you a detailed reply in two parts: the tools you can use to make the most of Amazon’s dynamic approach, and the architectural model you should adopt for a scalable infrastructure. I base my report on my experience gained in several AWS production projects in casual gaming (Facebook), e-commerce infrastructures and within the mainstream GIS (Geographic Information System). It’s true that my experience in gaming ( IsCool, The Game ) is currently the most representative in terms of scalability, due to the number of users (over 800 thousand DAU – daily active users – at peak usage and over 20 million page views every day), however my experiences in e-commerce and GIS (currently underway) provide a different view of scalability, taking into account the various problems of availability and da
5 0.21007903 1348 high scalability-2012-10-26-Stuff The Internet Says On Scalability For October 26, 2012
Introduction: It's HighScalability Time: 1.5 Billion Pageviews : Etsy in September; 200 dedicated database servers : Tumblr Quotable Quotes: @rbranson : Datadog stays available where it counts (metrics injest) by using Cassandra, combined with an RDBMS for queries. Nice. @jmhodges : Few engineers know what modern hw is capable of, in part, because the only people that see the numbers are in orgs that had to care or die. @tinagroves : Storing the brain in the cloud might cost $38/month asserts Jim Adelius at #strataconf in talk on #bigdata and thought crimes. Why is it hard to scale a database, in layman’s terms? on Quora. Some really good answers. My answer would involve a cookie jar filled with all different kinds of cookies and a motley crew of kindergartners all trying to get cookies at the same time while keebler elves are trying to fill up the jar, all at the same time. Rackspace now has their own cloud block storage prod
6 0.18504642 831 high scalability-2010-05-26-End-To-End Performance Study of Cloud Services
7 0.1554265 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
8 0.15045346 1099 high scalability-2011-08-16-The 5 Biggest Ways to Boost MySQL Scalability
9 0.14858735 1029 high scalability-2011-04-25-The Big List of Articles on the Amazon Outage
10 0.14741434 1398 high scalability-2013-02-04-Is Provisioned IOPS Better? Yes, it Delivers More Consistent and Higher Performance IO
11 0.13489838 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
13 0.11874998 61 high scalability-2007-08-07-What qps should we design for in making a MySpace like site?
15 0.11806512 1565 high scalability-2013-12-16-22 Recommendations for Building Effective High Traffic Web Software
16 0.11702803 1224 high scalability-2012-04-09-The Instagram Architecture Facebook Bought for a Cool Billion Dollars
17 0.11482498 1291 high scalability-2012-07-25-Vertical Scaling Ascendant - How are SSDs Changing Architectures?
18 0.11416669 1306 high scalability-2012-08-16-Stuff The Internet Says On Scalability For August 17, 2012
19 0.11342033 1508 high scalability-2013-08-28-Sean Hull's 20 Biggest Bottlenecks that Reduce and Slow Down Scalability
20 0.10799427 1058 high scalability-2011-06-13-Automation on AWS with Ruby and Puppet
topicId topicWeight
[(0, 0.164), (1, 0.07), (2, -0.038), (3, -0.017), (4, -0.035), (5, 0.018), (6, 0.02), (7, -0.132), (8, 0.035), (9, -0.145), (10, -0.019), (11, -0.038), (12, -0.011), (13, -0.069), (14, -0.057), (15, 0.016), (16, 0.043), (17, 0.017), (18, -0.031), (19, 0.043), (20, 0.032), (21, -0.002), (22, -0.014), (23, 0.012), (24, -0.073), (25, -0.08), (26, -0.018), (27, 0.033), (28, -0.006), (29, -0.048), (30, -0.059), (31, 0.009), (32, 0.021), (33, 0.04), (34, -0.011), (35, -0.042), (36, 0.085), (37, -0.045), (38, -0.016), (39, -0.046), (40, 0.084), (41, 0.004), (42, -0.089), (43, -0.047), (44, -0.059), (45, -0.027), (46, 0.052), (47, 0.051), (48, 0.055), (49, 0.153)]
simIndex simValue blogId blogTitle
same-blog 1 0.94390631 1121 high scalability-2011-09-21-5 Scalability Poisons and 3 Cloud Scalability Antidotes
Introduction: Sean Hull with two helpful posts: 5 Things That are Toxic to Scalability : Object Relational Mappers. Create complex queries that hard to optimize and tweak. Synchronous, Serial, Coupled or Locking Processes. Locks are like stop signs, traffic circles keep the traffic flowing. Row level locking is better than table level locking. Use async replication. Use eventual consistency for clusters. One Copy of Your Database. A single database server is a choke point. Create parallel databases and let a driver select between them. Having No Metrics. Visualize what's happening to your system using one of the many monitoring packages. Lack of Feature Flags. Be able to turn off features via a flag so when a spike hits features can be turned off to reduce load. 3 Ways to Boost Cloud Scalability : Use Auto-scaling. Spin-up new instances when a threshold is passed and back down again when traffic drops. Horizontally Scale the Database Tier. MySQL in a master
2 0.81947565 1631 high scalability-2014-04-14-How do you even do anything without using EBS?
Introduction: In a recent thread on Hacker News discussing recent AWS price changes , seldo mentioned they use AWS for business, they just never use EBS on AWS. A good question was asked: How do you even do anything without using EBS? Amazon certainly makes using EBS the easiest path. And EBS has a better reliability record as of late, but it's still often recommended to not use EBS. This avoids a single point of failure at the cost of a lot of complexity, though as AWS uses EBS internally, not using EBS may not save you if you use other AWS services like RDS or ELB. If you don't want to use EBS, it's hard to know where to even start. A dilemma to which Kevin Nuckolls gives a great answer : Well, you break your services out onto stateless and stateful machines. After that, you make sure that each of your stateful services is resilient to individual node failure. I prefer to believe that if you can't roll your entire infrastructure over to new nodes monthly then you're unprepared fo
3 0.71095848 1543 high scalability-2013-11-05-10 Things You Should Know About AWS
Introduction: Authored by Chris Fregly : Former Netflix Streaming Platform Engineer, AWS Certified Solution Architect and Purveyor of fluxcapacitor.com. Ahead of the upcoming 2nd annual re:Invent conference, inspired by Simone Brunozzi’s recent presentation at an AWS Meetup in San Francisco, and collected from a few of my recent Fluxcapacitor.com consulting engagements, I’ve compiled a list of 10 useful time and clock-tick saving tips about AWS. 1) Query AWS resource metadata Can’t remember the EBS-Optimized IO throughput of your c1.xlarge cluster? How about the size limit of an S3 object on a single PUT? awsnow.info is the answer to all of your AWS-resource metadata questions. Interested in integrating awsnow.info with your application? You’re in luck. There’s now a REST API , as well! Note: These are default soft limits and will vary by account. 2) Tame your S3 buckets Delete an entire S3 bucket with a single CLI command:
Introduction: Amazon created a whole new class of service with their Provisioned IOPS for RDS, EBS, and DynamoDB. The idea is simple. If you want more performance, you turn a dial up. If you want less, you turn a dial down. A beautifully simple model. You pay for the performance you want, which is different than their previous cloud model, where performance varied, but you paid only for what you used. The question: Do these higher priced services really work better? Rodrigo Campos put this question to the test (only for EBS) by running a benchmark he describes in IOMelt Provisioned IOPS EBS Benchmark Results - December 2012 . The result? Yes, AWS Provisioned IOPS Volumes Really Deliver More Consistent and Higher Performance IO : It is clear that the provisioned IOPS EBS volumes offer a huge performance upgrade when compared to the non-optimized EBS volumes, but as data has to be spread among more underlying disks or systems, it seems that the volume is increasingly more susceptibl
5 0.68453884 1473 high scalability-2013-06-10-The 10 Deadly Sins Against Scalability
Introduction: In the moral realm there may be 7 deadly sins, but scalability maven Sean Hull has come up Five More Things Deadly to Scalability that when added to his earlier 5 Things That are Toxic to Scalability , make for a numerologically satisfying 10 sins again scalability: Slow Disk I/O – RAID 5 – Multi-tenant EBS . Use RAID 10, it provides good protection along with good read and write performance. The design of RAID 5 means poor performance and long repair times on failure. On AWS consider Provisioned IOPS as a way around IO bottlenecks. Using the database for Queuing. The database may seem like the perfect place to keep work queues, but under load locking and scanning overhead kills performance. Use specialized products like RabbitMQ and SQS to remove this bottleneck. Using Database for full-text searching. Search seems like another perfect database feature. At scale search doesn't perform well. Use specialized technologies like Solr or Sphinx. Insufficient Caching
6 0.66942954 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
7 0.66605377 1347 high scalability-2012-10-25-Not All Regions are Created Equal - South America Es Bueno
8 0.65512037 1058 high scalability-2011-06-13-Automation on AWS with Ruby and Puppet
9 0.63605738 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
10 0.62833178 1348 high scalability-2012-10-26-Stuff The Internet Says On Scalability For October 26, 2012
11 0.58472943 1165 high scalability-2011-12-28-Strategy: Guaranteed Availability Requires Reserving Instances in Specific Zones
12 0.58329225 1434 high scalability-2013-04-03-5 Steps to Benchmarking Managed NoSQL - DynamoDB vs Cassandra
13 0.58213216 798 high scalability-2010-03-22-7 Secrets to Successfully Scaling with Scalr (on Amazon) by Sebastian Stadil
14 0.57106423 1133 high scalability-2011-10-27-Strategy: Survive a Comet Strike in the East With Reserved Instances in the West
15 0.56488395 1098 high scalability-2011-08-15-Should any cloud be considered one availability zone? The Amazon experience says yes.
16 0.56448787 831 high scalability-2010-05-26-End-To-End Performance Study of Cloud Services
17 0.56178427 1286 high scalability-2012-07-18-Strategy: Kill Off Multi-tenant Instances with High CPU Stolen Time
18 0.56028575 1029 high scalability-2011-04-25-The Big List of Articles on the Amazon Outage
19 0.55749238 1099 high scalability-2011-08-16-The 5 Biggest Ways to Boost MySQL Scalability
20 0.55711436 1248 high scalability-2012-05-21-Pinterest Architecture Update - 18 Million Visitors, 10x Growth,12 Employees, 410 TB of Data
topicId topicWeight
[(1, 0.197), (2, 0.194), (10, 0.17), (38, 0.173), (61, 0.05), (79, 0.053), (85, 0.029), (94, 0.034)]
simIndex simValue blogId blogTitle
1 0.93469959 1511 high scalability-2013-09-04-Wide Fast SATA: the Recipe for Hot Performance
Introduction: This is a guest post by Brian Bulkowski , CTO and co-founder of Aerospike , a leading clustered NoSQL database, has worked in the area of high performance commodity systems since 1989. This blog post will tell you exactly how to build a multi-terabyte high throughput datacenter server. A fast, reliable multi-terrabyte data tier can be used for recent behavior (messages, tweets, plays, actions), or anywhere that today you use Redis or Memcache. You need to know: Which SSDs work Which chassis work How to configure your RAID cards Intel’s SATA solutions – combined with a high capacity storage server like the Dell R720xd and a host bus adapter based on the LSI 2208, and a Flash optimized database like Aerospike , enables high throughput and low latency. In a wide configuration, with 12 to 20 drives per 2U server, individual servers can cost-effectively serve at high throughput with 16T at $2.50 per GB with the s3700, or $1.25 with the s3500. Other SSD of
same-blog 2 0.92159426 1121 high scalability-2011-09-21-5 Scalability Poisons and 3 Cloud Scalability Antidotes
Introduction: Sean Hull with two helpful posts: 5 Things That are Toxic to Scalability : Object Relational Mappers. Create complex queries that hard to optimize and tweak. Synchronous, Serial, Coupled or Locking Processes. Locks are like stop signs, traffic circles keep the traffic flowing. Row level locking is better than table level locking. Use async replication. Use eventual consistency for clusters. One Copy of Your Database. A single database server is a choke point. Create parallel databases and let a driver select between them. Having No Metrics. Visualize what's happening to your system using one of the many monitoring packages. Lack of Feature Flags. Be able to turn off features via a flag so when a spike hits features can be turned off to reduce load. 3 Ways to Boost Cloud Scalability : Use Auto-scaling. Spin-up new instances when a threshold is passed and back down again when traffic drops. Horizontally Scale the Database Tier. MySQL in a master
3 0.8868717 1452 high scalability-2013-05-06-7 Not So Sexy Tips for Saving Money On Amazon
Introduction: Harish Ganesan CTO of 8KMiles has a very helpful blog, Cloud, Big Data and Mobile , where he shows a nice analytical bent which leads to a lot of practical advice and cost saving tips: Use SQS Batch Requests to reduce the number of requests hitting SQS which saves costs. Sending 10 messages in a single batch request which in the example save $30/month. Use SQS Long Polling to reduce extra polling requests, cutting down empty receives, which in the example saves ~$600 in empty receive leakage costs. Choose the right search technology choice to save costs in AWS by matching your activity pattern to the technology. For a small application with constant load or a heavily utilized search tier or seasonal loads Amazon Cloud Search looks like the cost efficient play. Use Amazon CloudFront Price Class to minimize costs by selecting the right Price Class for your audience to potentially reduce delivery costs by excluding Amazon CloudFront’s more expensive edge locatio
4 0.87906003 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day
Introduction: This is a guest post written by Claude Johnson , a Lead Site Reliability Engineer at salesforce.com . The following is an architectural overview of salesforce.com’s core platform and applications. Other systems such as Heroku's Dyno architecture or the subsystems of other products such as work.com and do.com are specifically not covered by this material, although database.com is. The idea is to share with the technology community some insight about how salesforce.com does what it does. Any mistakes or omissions are mine. This is by no means comprehensive but if there is interest, the author would be happy to tackle other areas of how salesforce.com works. Salesforce.com is interested in being more open with the technology communities that we have not previously interacted with. Here’s to the start of “Opening the Kimono” about how we work. Since 1999, salesforce.com has been singularly focused on building technologies for business that are delivered over the Internet, displaci
5 0.87212878 994 high scalability-2011-02-23-This stuff isn't taught, you learn it bit by bit as you solve each problem.
Introduction: "For the things we have to learn before we can do them, we learn by doing them." -- Aristotle A really nice Internet moment happened in the HackerNews thread Disqus: Scaling the World’s Largest Django Application , when David Kitchen crafted an awesome response to a question about how you learn to build scalable systems. It's so good I thought I would reproduce it here. Question : asked by grovulent : Not like this is a problem I have to worry about. But where on earth does one learn this stuff? The talk is useful - as an overview of what they use - but I know nothing of how to implement a single step. Answer : answered by David Kitchen of buro9 : It's called experience. Which perhaps sounds rude, but it's not meant to be. This stuff isn't taught per se, you learn it bit by bit as you solve each problem that you face. I learned about HAProxy when my site load exceeded that which a single web server could manage.
6 0.86299074 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
7 0.8624993 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
8 0.85962802 1585 high scalability-2014-01-24-Stuff The Internet Says On Scalability For January 24th, 2014
9 0.8567372 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
10 0.85320222 1046 high scalability-2011-05-23-Evernote Architecture - 9 Million Users and 150 Million Requests a Day
11 0.84909165 584 high scalability-2009-04-27-Some Questions from a newbie
12 0.84570199 1398 high scalability-2013-02-04-Is Provisioned IOPS Better? Yes, it Delivers More Consistent and Higher Performance IO
13 0.84512872 1482 high scalability-2013-06-26-Leveraging Cloud Computing at Yelp - 102 Million Monthly Vistors and 39 Million Reviews
14 0.8447125 1635 high scalability-2014-04-21-This is why Microsoft won. And why they lost.
15 0.83805484 1004 high scalability-2011-03-14-Twitter by the Numbers - 460,000 New Accounts and 140 Million Tweets Per Day
17 0.83210444 257 high scalability-2008-02-22-Kevin's Great Adventures in SSDland
18 0.83156717 1543 high scalability-2013-11-05-10 Things You Should Know About AWS
19 0.83148873 1369 high scalability-2012-12-10-Switch your databases to Flash storage. Now. Or you're doing it wrong.
20 0.83004153 1072 high scalability-2011-07-01-TripAdvisor Strategy: No Architects, Engineers Work Across the Entire Stack