high_scalability high_scalability-2009 high_scalability-2009-717 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Scale-Out is incrementally adding servers as needed to scale rather than buying larger servers. Here's the MySQL idea of what a scale-out architecture looks like: This MySQL article lists 5 problems to avoid when scaling out: Don't Think Synchronously . Introduce asynchronous communication, parallelization, and strategies to deal with approximate or slightly outdated data. Don't Think Vertically. Scaling by bigger machines won't work. Plan on horizontal scaling and asynchronous architectures form the start which make it easy to add capacity on demand. Don't Mix Transactions with Business Intelligence. Transactions and analytics are inherently different. Separate out different types of data onto different databases. Avoid Mixing Hot and Cold Data. Static and fast changing data are inherently different. Separate out different types of data onto different databases. Don't Forget the Power of Memory. Make data accessible in RA
sentIndex sentText sentNum sentScore
1 Scale-Out is incrementally adding servers as needed to scale rather than buying larger servers. [sent-1, score-0.6]
2 Here's the MySQL idea of what a scale-out architecture looks like: This MySQL article lists 5 problems to avoid when scaling out: Don't Think Synchronously . [sent-2, score-0.238]
3 Introduce asynchronous communication, parallelization, and strategies to deal with approximate or slightly outdated data. [sent-3, score-0.971]
4 Plan on horizontal scaling and asynchronous architectures form the start which make it easy to add capacity on demand. [sent-6, score-0.863]
5 Separate out different types of data onto different databases. [sent-9, score-0.855]
6 Static and fast changing data are inherently different. [sent-11, score-0.539]
7 Separate out different types of data onto different databases. [sent-12, score-0.855]
8 Make data accessible in RAM by smartly partitioning data across servers. [sent-14, score-0.642]
wordName wordTfidf (topN-words)
[('inherently', 0.345), ('onto', 0.261), ('outdated', 0.229), ('mixing', 0.222), ('smartly', 0.211), ('asynchronous', 0.206), ('approximate', 0.202), ('types', 0.183), ('parallelization', 0.177), ('separate', 0.164), ('different', 0.156), ('incrementally', 0.154), ('cold', 0.146), ('scaling', 0.146), ('businesses', 0.143), ('slightly', 0.141), ('forget', 0.132), ('introduce', 0.129), ('buying', 0.128), ('accessible', 0.127), ('mix', 0.125), ('horizontal', 0.124), ('practices', 0.114), ('hot', 0.108), ('partitioning', 0.106), ('bigger', 0.106), ('strategies', 0.104), ('communication', 0.102), ('data', 0.099), ('plan', 0.098), ('changing', 0.095), ('think', 0.094), ('analytics', 0.093), ('avoid', 0.092), ('form', 0.091), ('architectures', 0.09), ('deal', 0.089), ('larger', 0.088), ('growth', 0.087), ('adding', 0.083), ('wo', 0.079), ('replication', 0.077), ('transactions', 0.076), ('rather', 0.074), ('needed', 0.073), ('ram', 0.072), ('machines', 0.071), ('add', 0.071), ('capacity', 0.068), ('start', 0.067)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 717 high scalability-2009-10-07-How to Avoid the Top 5 Scale-Out Pitfalls
Introduction: Scale-Out is incrementally adding servers as needed to scale rather than buying larger servers. Here's the MySQL idea of what a scale-out architecture looks like: This MySQL article lists 5 problems to avoid when scaling out: Don't Think Synchronously . Introduce asynchronous communication, parallelization, and strategies to deal with approximate or slightly outdated data. Don't Think Vertically. Scaling by bigger machines won't work. Plan on horizontal scaling and asynchronous architectures form the start which make it easy to add capacity on demand. Don't Mix Transactions with Business Intelligence. Transactions and analytics are inherently different. Separate out different types of data onto different databases. Avoid Mixing Hot and Cold Data. Static and fast changing data are inherently different. Separate out different types of data onto different databases. Don't Forget the Power of Memory. Make data accessible in RA
2 0.13074909 142 high scalability-2007-11-05-Strategy: Diagonal Scaling - Don't Forget to Scale Out AND Up
Introduction: All the cool kids advocate scaling out as the secret sauce of scaling. And it is, but don't forget to serve some tasty "scaling up" as a side dish. Scaling up doesn't have to mean buying a jet propelled, liquid cooled, 128 core monster super computer. Scaling up can just mean buying at the high end of the commodity buffet by buying more cores, more memory and using a shared nothing architecture to take advantage of all that power without adding complexity. Scale out when you need to, but big beefy boxes can absorb a lot of load before it's necessary to hit up your data center for more rack space. Here are a few examples of scaling out and up: John Allspaw , Flickr's operations manager, coined the term diagonal scaling for this strategy. In Making a site faster by removing machines (and a comment on this post) John told how Flickr replaced 67 dual-cpu boxes with 18 dual quad-core machines and recovered almost 4x rack space and reduced costs by about 50 percent. Fotol
3 0.11078482 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
Introduction: We are on the edge of two potent technological changes: Clouds and Memory Based Architectures. This evolution will rip open a chasm where new players can enter and prosper. Google is the master of disk. You can't beat them at a game they perfected. Disk based databases like SimpleDB and BigTable are complicated beasts, typical last gasp products of any aging technology before a change. The next era is the age of Memory and Cloud which will allow for new players to succeed. The tipping point will be soon. Let's take a short trip down web architecture lane: It's 1993: Yahoo runs on FreeBSD, Apache, Perl scripts and a SQL database It's 1995: Scale-up the database. It's 1998: LAMP It's 1999: Stateless + Load Balanced + Database + SAN It's 2001: In-memory data-grid. It's 2003: Add a caching layer. It's 2004: Add scale-out and partitioning. It's 2005: Add asynchronous job scheduling and maybe a distributed file system. It's 2007: Move it all into the cloud. It's 2008: C
4 0.10813015 392 high scalability-2008-09-24-Building a Scalable Architecture for Web Apps
Introduction: By Bhavin Turakhia CEO, Directi. Covers: * Why scalability is important. Viral marketing can result in instant success. With RSS/Ajax/SOA number of requests grow exponentially with user base. Goal is to build a web 2.0 app that can server millions of users with zero downtime. * Introduction to the variables. Scalability, performance, responsiveness, availability, downtime impact, cost, maintenance effort. * Introduction to the factors. Platform selection, hardware, application design, database architecture, deployment architecture, storage architecture, abuse prevention, monitoring mechanisms, etc. * Building our own scalable architecture in incremental steps: vertical scaling, vertical partitioning, horizontal scaling, horizontal partitioning, etc. First buy bigger. Then deploy each service on a separate node. Then increase the number of nudes and load balance. Deal with session management. Remove single points of failure. Use a shared nothing cluster. Choice of master-slave m
5 0.10396709 672 high scalability-2009-08-06-An Unorthodox Approach to Database Design : The Coming of the Shard
Introduction: Update 4: Why you don’t want to shard. by Morgon on the MySQL Performance Blog. Optimize everything else first, and then if performance still isn’t good enough, it’s time to take a very bitter medicine. Update 3: Building Scalable Databases: Pros and Cons of Various Database Sharding Schemes by Dare Obasanjo. Excellent discussion of why and when you would choose a sharding architecture, how to shard, and problems with sharding. Update 2: Mr. Moore gets to punt on sharding by Alan Rimm-Kaufman of 37signals. Insightful article on design tradeoffs and the evils of premature optimization. With more memory, more CPU, and new tech like SSD, problems can be avoided before more exotic architectures like sharding are needed. Add features not infrastructure. Jeremy Zawodny says he's wrong wrong wrong. we're running multi-core CPUs at slower clock speeds. Moore won't save you. Update: Dan Pritchett shares some excellent Sharding Lessons : Size Your Shards, Use Math on Shard C
6 0.09049052 248 high scalability-2008-02-13-What's your scalability plan?
7 0.08769688 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
8 0.087452084 1000 high scalability-2011-03-08-Medialets Architecture - Defeating the Daunting Mobile Device Data Deluge
9 0.08628571 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
10 0.082851157 666 high scalability-2009-07-30-Learn How to Think at Scale
11 0.082691237 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
12 0.080960676 594 high scalability-2009-05-08-Eight Best Practices for Building Scalable Systems
13 0.079729348 331 high scalability-2008-05-27-eBay Architecture
14 0.078967623 1008 high scalability-2011-03-22-Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
15 0.077800654 448 high scalability-2008-11-22-Google Architecture
16 0.07748799 1199 high scalability-2012-02-27-Zen and the Art of Scaling - A Koan and Epigram Approach
18 0.076351166 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
19 0.073925063 801 high scalability-2010-03-30-Running Large Graph Algorithms - Evaluation of Current State-of-the-Art and Lessons Learned
20 0.073564485 5 high scalability-2007-07-10-mixi.jp Architecture
topicId topicWeight
[(0, 0.136), (1, 0.071), (2, 0.011), (3, -0.016), (4, -0.0), (5, 0.052), (6, -0.021), (7, -0.029), (8, -0.001), (9, -0.009), (10, -0.029), (11, 0.027), (12, -0.007), (13, 0.038), (14, 0.034), (15, 0.033), (16, 0.027), (17, -0.004), (18, 0.015), (19, 0.064), (20, -0.006), (21, 0.024), (22, -0.035), (23, -0.028), (24, -0.033), (25, 0.018), (26, -0.007), (27, -0.029), (28, 0.035), (29, 0.032), (30, 0.06), (31, -0.005), (32, -0.001), (33, 0.002), (34, -0.035), (35, -0.026), (36, -0.029), (37, -0.01), (38, 0.01), (39, -0.02), (40, 0.026), (41, -0.025), (42, 0.001), (43, -0.019), (44, 0.004), (45, -0.021), (46, -0.019), (47, -0.013), (48, -0.043), (49, -0.083)]
simIndex simValue blogId blogTitle
same-blog 1 0.92515433 717 high scalability-2009-10-07-How to Avoid the Top 5 Scale-Out Pitfalls
Introduction: Scale-Out is incrementally adding servers as needed to scale rather than buying larger servers. Here's the MySQL idea of what a scale-out architecture looks like: This MySQL article lists 5 problems to avoid when scaling out: Don't Think Synchronously . Introduce asynchronous communication, parallelization, and strategies to deal with approximate or slightly outdated data. Don't Think Vertically. Scaling by bigger machines won't work. Plan on horizontal scaling and asynchronous architectures form the start which make it easy to add capacity on demand. Don't Mix Transactions with Business Intelligence. Transactions and analytics are inherently different. Separate out different types of data onto different databases. Avoid Mixing Hot and Cold Data. Static and fast changing data are inherently different. Separate out different types of data onto different databases. Don't Forget the Power of Memory. Make data accessible in RA
2 0.74486929 142 high scalability-2007-11-05-Strategy: Diagonal Scaling - Don't Forget to Scale Out AND Up
Introduction: All the cool kids advocate scaling out as the secret sauce of scaling. And it is, but don't forget to serve some tasty "scaling up" as a side dish. Scaling up doesn't have to mean buying a jet propelled, liquid cooled, 128 core monster super computer. Scaling up can just mean buying at the high end of the commodity buffet by buying more cores, more memory and using a shared nothing architecture to take advantage of all that power without adding complexity. Scale out when you need to, but big beefy boxes can absorb a lot of load before it's necessary to hit up your data center for more rack space. Here are a few examples of scaling out and up: John Allspaw , Flickr's operations manager, coined the term diagonal scaling for this strategy. In Making a site faster by removing machines (and a comment on this post) John told how Flickr replaced 67 dual-cpu boxes with 18 dual quad-core machines and recovered almost 4x rack space and reduced costs by about 50 percent. Fotol
3 0.72780657 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
Introduction: This is a guest posting by Marty Abbott and Michael Fisher, authors of The Art of Scalability . I'm still reading their book and will have an interview with them a little later. If 2010 is the year that you’ve decided to kickoff your startup or if you’ve already got something off the ground and are expecting double or triple digit growth, this list is for you. We all want the attention of user s to achieve viral growth but as many can attest , too much attention can bring a startup to its knees. If you’ve used Twitter for any amount of time you’re sure to have seen the “Fail Whale”, which is so often seen that it has its own fan club . Take a look at the graph below from Compete.com showing Twitter’s unique visitors. One can argue that limitations in the product offering have as much to do with the flattening of growth over the past six months as does the availability , but it’s hard to beli
4 0.70487934 793 high scalability-2010-03-10-Saying Yes to NoSQL; Going Steady with Cassandra at Digg
Introduction: The last six months have been exciting for Digg's engineering team. We're working on a soup-to-nuts rewrite. Not only are we rewriting all our application code, but we're also rolling out a new client and server architecture. And if that doesn't sound like a big enough challenge, we're replacing most of our infrastructure components and moving away from LAMP. Perhaps our most significant infrastructure change is abandoning MySQL in favor of a NoSQL alternative. To someone like me who's been building systems almost exclusively on relational databases for almost 20 years, this feels like a bold move. What's Wrong with MySQL? Our primary motivation for moving away from MySQL is the increasing difficulty of building a high performance, write intensive, application on a data set that is growing quickly, with no end in sight. This growth has forced us into horizontal and vertical partitioning strategies that have eliminated most of the value of a relational database, while still incurr
5 0.70429254 869 high scalability-2010-07-30-Hot Scalability Links for July 30, 2010
Introduction: Jeremy Zawodny, while performing data alchemy in the dungeons of Craigslist, stored 1,250,000,000 Key/Value Pairs in Redis on a 32GB Machine . Data sorting world record: 1 terabyte, 1 minute . The system has 52 computer nodes, each node is a commodity server with two quad-core processors, 24 gigabytes (GB) memory and sixteen 500 GB disks . It's not just hardware though, they also built a software that utilized all their CPU and RAM. Tweets of Gold: wm : I am really getting the sense that none of you yokels waxing profound about scalability actually has anything factual to say joestump : I think you can do things to *mitigate* pain points up front. You don't need to over-engineer, but it's not hard to look forward. danielcrenna : I love it when I check in debug code accidentally and it turns into a three day hunt for a major scalability problem joestump : Your post also makes me think of another phrase I say often: Scaling == Specialization. Bigger scale =
6 0.67472035 185 high scalability-2007-12-13-Is premature scalation a real disease?
7 0.67040271 1172 high scalability-2012-01-10-A Perfect Fifth of Notes on Scalability
8 0.66442341 392 high scalability-2008-09-24-Building a Scalable Architecture for Web Apps
9 0.66374165 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
10 0.65964675 1439 high scalability-2013-04-12-Stuff The Internet Says On Scalability For April 12, 2013
11 0.65841919 847 high scalability-2010-06-23-Product: dbShards - Share Nothing. Shard Everything.
12 0.65000194 1527 high scalability-2013-10-04-Stuff The Internet Says On Scalability For October 4th, 2013
13 0.64822841 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
14 0.64808571 950 high scalability-2010-11-30-NoCAP – Part III – GigaSpaces clustering explained..
15 0.64370459 672 high scalability-2009-08-06-An Unorthodox Approach to Database Design : The Coming of the Shard
16 0.64316839 381 high scalability-2008-09-08-Guerrilla Capacity Planning and the Law of Universal Scalability
17 0.63727701 1199 high scalability-2012-02-27-Zen and the Art of Scaling - A Koan and Epigram Approach
18 0.63392198 487 high scalability-2009-01-08-Paper: Sharding with Oracle Database
19 0.63253963 731 high scalability-2009-10-28-Need for change in your IT infrastructure
20 0.63181436 1024 high scalability-2011-04-15-Stuff The Internet Says On Scalability For April 15, 2011
topicId topicWeight
[(1, 0.092), (2, 0.327), (10, 0.042), (27, 0.165), (61, 0.166), (79, 0.091)]
simIndex simValue blogId blogTitle
1 0.9761948 1483 high scalability-2013-06-27-Paper: XORing Elephants: Novel Erasure Codes for Big Data
Introduction: Erasure codes are one of those seemingly magical mathematical creations that with the developments described in the paper XORing Elephants: Novel Erasure Codes for Big Data , are set to replace triple replication as the data storage protection mechanism of choice. The result says Robin Harris (StorageMojo) in an excellent article, Facebook’s advanced erasure codes : "WebCos will be able to store massive amounts of data more efficiently than ever before. Bad news: so will anyone else." Robin says with cheap disks triple replication made sense and was economical. With ever bigger BigData the overhead has become costly. But erasure codes have always suffered from unacceptably long time to repair times. This paper describes new Locally Repairable Codes (LRCs) that are efficiently repairable in disk I/O and bandwidth requirements: These systems are now designed to survive the loss of up to four storage elements – disks, servers, nodes or even entire data centers – without losing
2 0.96547049 28 high scalability-2007-07-25-Product: NetApp MetroCluster Software
Introduction: NetApp MetroCluster Software Cost-effective is an integrated high-availability storage cluster and site failover capability. NetApp MetroCluster is an integrated high-availability and disaster recovery solution that can reduce system complexity and simplify management while ensuring greater return on investment. MetroCluster uses clustered server technology to replicate data synchronously between sites located miles apart, eliminating data loss in case of a disruption. Simple and powerful recovery process minimizes downtime, with little or no user action required. At one company I worked at they used the NetApp snap mirror feature to replicate data across long distances to multiple datacenters. They had a very fast backbone and it worked well. The issue with NetApp is always one of cost, but if you can afford it, it's a good option.
3 0.96152788 1097 high scalability-2011-08-12-Stuff The Internet Says On Scalability For August 12, 2011
Introduction: Submitted for your scaling pleasure, you may not scale often, but when you scale, please drink us: Quotably quotable quotes: @mardix : There is no single point of truth in #NoSQL . #Consistency is no longer global, it's relative to the one accessing it. #Scalability @kekline : RT @CurtMonash: "...from industry figures, Basho/Riak is our third-biggest competitor." How often do you encounter them? "Never have" #nosql @dave_jacobs : Love being in a city where I can overhear a convo about Heroku scalability while doing deadlifts. #ahsanfrancisco @satheeshilu : Doctor at #hospital in india says #ge #healthcare software is slow to handle 100K X-rays an year.Scalability is critical 4 Indian #software @sufw : How can it be possible that Tagged has 80m users and I have *never* heard of it!?! @EventCloudPro : One of my vacation realizations? Whole #bigdata thing has turned into a lotta #bighype - many distinct issues & nothing to do w/ #bigdata No
same-blog 4 0.9433248 717 high scalability-2009-10-07-How to Avoid the Top 5 Scale-Out Pitfalls
Introduction: Scale-Out is incrementally adding servers as needed to scale rather than buying larger servers. Here's the MySQL idea of what a scale-out architecture looks like: This MySQL article lists 5 problems to avoid when scaling out: Don't Think Synchronously . Introduce asynchronous communication, parallelization, and strategies to deal with approximate or slightly outdated data. Don't Think Vertically. Scaling by bigger machines won't work. Plan on horizontal scaling and asynchronous architectures form the start which make it easy to add capacity on demand. Don't Mix Transactions with Business Intelligence. Transactions and analytics are inherently different. Separate out different types of data onto different databases. Avoid Mixing Hot and Cold Data. Static and fast changing data are inherently different. Separate out different types of data onto different databases. Don't Forget the Power of Memory. Make data accessible in RA
Introduction: When we last visited WhatsApp they’d just been acquired by Facebook for $19 billion. We learned about their early architecture, which centered around a maniacal focus on optimizing Erlang into handling 2 million connections a server, working on All The Phones, and making users happy through simplicity. Two years later traffic has grown 10x. How did WhatsApp make that jump to the next level of scalability? Rick Reed tells us in a talk he gave at the Erlang Factory: That's 'Billion' with a 'B': Scaling to the next level at WhatsApp ( slides ), which revealed some eye popping WhatsApp stats: What has hundreds of nodes, thousands of cores, hundreds of terabytes of RAM, and hopes to serve the billions of smartphones that will soon be a reality around the globe? The Erlang/FreeBSD-based server infrastructure at WhatsApp. We've faced many challenges in meeting the ever-growing demand for our messaging services, but as we continue to push the envelope on size (>800
6 0.92249638 1141 high scalability-2011-11-11-Stuff The Internet Says On Scalability For November 11, 2011
7 0.91641402 883 high scalability-2010-08-20-Hot Scalability Links For Aug 20, 2010
8 0.90955162 544 high scalability-2009-03-18-QCon London 2009: Upgrading Twitter without service disruptions
9 0.90254062 714 high scalability-2009-10-02-HighScalability has Moved to Squarespace.com!
10 0.90210837 306 high scalability-2008-04-21-The Search for the Source of Data - How SimpleDB Differs from a RDBMS
11 0.9020645 628 high scalability-2009-06-13-Neo4j - a Graph Database that Kicks Buttox
12 0.89875782 1017 high scalability-2011-04-06-Netflix: Run Consistency Checkers All the time to Fixup Transactions
13 0.89768392 672 high scalability-2009-08-06-An Unorthodox Approach to Database Design : The Coming of the Shard
14 0.89720929 595 high scalability-2009-05-08-Publish-subscribe model does not scale?
15 0.89580166 1074 high scalability-2011-07-06-11 Common Web Use Cases Solved in Redis
16 0.89520806 795 high scalability-2010-03-16-1 Billion Reasons Why Adobe Chose HBase
17 0.89513779 933 high scalability-2010-11-01-Hot Trend: Move Behavior to Data for a New Interactive Application Architecture
18 0.89500517 724 high scalability-2009-10-19-Drupal's Scalability Makeover - You give up some control and you get back scalability
19 0.89493293 943 high scalability-2010-11-16-Facebook's New Real-time Messaging System: HBase to Store 135+ Billion Messages a Month
20 0.89483893 1538 high scalability-2013-10-28-Design Decisions for Scaling Your High Traffic Feeds