high_scalability high_scalability-2010 high_scalability-2010-898 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Jesper Söderlund not only put together a few interesting scalability patterns , he also came up with a few interesting scalability lessons : Lesson #1 . Put Smarty compile and template caches on an active-active DRBD cluster with high load and your servers will DIE! Lesson #2 . Don't use out-of-the-box configurations. Lesson #3 . Single points of contention will eventually become a bottleneck. Lesson #4 . Plan in advance. Lesson #5 . Offload your databases as much as possible. Lesson #6 . File systems matter and can run out of space / inodes. For more details and explanations see the original post.
sentIndex sentText sentNum sentScore
1 Jesper Söderlund not only put together a few interesting scalability patterns , he also came up with a few interesting scalability lessons : Lesson #1 . [sent-1, score-0.826]
2 Put Smarty compile and template caches on an active-active DRBD cluster with high load and your servers will DIE! [sent-2, score-0.742]
3 Single points of contention will eventually become a bottleneck. [sent-6, score-0.457]
4 File systems matter and can run out of space / inodes. [sent-12, score-0.298]
5 For more details and explanations see the original post. [sent-13, score-0.481]
wordName wordTfidf (topN-words)
[('lesson', 0.494), ('soderlund', 0.29), ('smarty', 0.277), ('jesper', 0.277), ('drbd', 0.24), ('compile', 0.216), ('explanations', 0.213), ('offload', 0.197), ('template', 0.177), ('die', 0.166), ('put', 0.151), ('contention', 0.15), ('caches', 0.138), ('came', 0.128), ('original', 0.126), ('patterns', 0.123), ('interesting', 0.12), ('eventually', 0.118), ('plan', 0.115), ('lessons', 0.114), ('matter', 0.108), ('points', 0.105), ('scalability', 0.096), ('space', 0.093), ('details', 0.093), ('become', 0.084), ('together', 0.082), ('file', 0.076), ('cluster', 0.074), ('databases', 0.072), ('post', 0.062), ('run', 0.053), ('single', 0.051), ('load', 0.05), ('see', 0.049), ('much', 0.047), ('servers', 0.044), ('systems', 0.044), ('high', 0.043), ('also', 0.036), ('use', 0.027)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 898 high scalability-2010-09-09-6 Scalability Lessons
Introduction: Jesper Söderlund not only put together a few interesting scalability patterns , he also came up with a few interesting scalability lessons : Lesson #1 . Put Smarty compile and template caches on an active-active DRBD cluster with high load and your servers will DIE! Lesson #2 . Don't use out-of-the-box configurations. Lesson #3 . Single points of contention will eventually become a bottleneck. Lesson #4 . Plan in advance. Lesson #5 . Offload your databases as much as possible. Lesson #6 . File systems matter and can run out of space / inodes. For more details and explanations see the original post.
2 0.24680181 884 high scalability-2010-08-23-6 Ways to Kill Your Servers - Learning How to Scale the Hard Way
Introduction: This is a guest post by Steffen Konerow, author of the High Performance Blog . Learning how to scale isn’t easy without any prior experience. Nowadays you have plenty of websites like highscalability.com to get some inspiration, but unfortunately there is no solution that fits all websites and needs. You still have to think on your own to find a concept that works for your requirements. So did I. A few years ago, my bosses came to me and said “We’ve got a new project for you. It’s the relaunch of a website that has already 1 million users a month. You have to build the website and make sure we’ll be able to grow afterwards”. I was already an experienced coder, but not in these dimensions, so I had to start learning how to scale – the hard way. The software behind the website was a PHP content management system, based on Smarty and MySQL. The first task was finding a proper hosting company who had the experience and would also manage the servers for us. After some researc
3 0.21363761 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month
Introduction: Steve Huffman , co-founder of social news site Reddit , gave an excellent presentation ( slides , transcript ) on the lessons he learned while building and growing Reddit to 7.5 million users per month, 270 million page views per month, and 20+ database servers. Steve says a lot of the lessons were really obvious, so you may not find a lot of completely new ideas in the presentation. But Steve has an earnestness and genuineness about him that is so obviously grounded in experience that you can't help but think deeply about what you could be doing different. And if Steve didn't know about these lessons, I'm betting others don't either. There are seven lessons, each has their own summary section: Lesson one: Crash Often; Lesson 2: Separation of Services; Lesson 3: Open Schema; Lesson 4: Keep it Stateless; Lesson 5: Memcache; Lesson 6: Store Redundant Data; Lesson 7: Work Offline. By far the most surprising feature of their architecture is in Lesson Six, whose essential idea is:
4 0.19930054 897 high scalability-2010-09-08-4 General Core Scalability Patterns
Introduction: Jesper Söderlund put together an excellent list of four general scalability patterns and four subpatterns in his post Scalability patterns and an interesting story : Load distribution - Spread the system load across multiple processing units Load balancing / load sharing - Spreading the load across many components with equal properties for handling the request Partitioning - Spreading the load across many components by routing an individual request to a component that owns that data specific Vertical partitioning - Spreading the load across the functional boundaries of a problem space, separate functions being handled by different processing units Horizontal partitioning - Spreading a single type of data element across many instances, according to some partitioning key, e.g. hashing the player id and doing a modulus operation, etc. Quite often referred to as sharding. Queuing and batch - Achieve efficiencies of scale by
5 0.11943579 906 high scalability-2010-09-22-Applying Scalability Patterns to Infrastructure Architecture
Introduction: Too often software design patterns are overlooked by network and application delivery network architects but these patterns are often equally applicable to addressing a broad range of architectural challenges in the application delivery tier of the data center. By Lori Mac Vittie, F5 Networks The “ High Scalability ” blog is fast becoming one of my favorite reads. Last week did not disappoint with a post highlighting a set of scalability design patterns that was, apparently, inspired by yet another High Scalability post on “ 6 Ways to Kill Your Servers: Learning to Scale the Hard Way. ” Credit:Michael Chow/azcentral.com This particular post caught my attention primarily because although I’ve touched on many of these patterns in the past, I’ve never thought to call them what they are: scalability patterns. That’s probably a side-effect of forgetting that building an architecture of any kind is at its core computer science and thus
6 0.099572584 1633 high scalability-2014-04-16-Six Lessons Learned the Hard Way About Scaling a Million User System
7 0.09265817 196 high scalability-2007-12-30-MySQL clustering strategies and comparisions
8 0.088151693 338 high scalability-2008-06-02-Total Cost of Ownership for different web development frameworks
9 0.08294034 400 high scalability-2008-10-01-The Pattern Bible for Distributed Computing
10 0.082144082 340 high scalability-2008-06-06-Economies of Non-Scale
11 0.081480846 852 high scalability-2010-07-07-Strategy: Recompute Instead of Remember Big Data
12 0.078321129 271 high scalability-2008-03-08-Product: DRBD - Distributed Replicated Block Device
13 0.078210235 1420 high scalability-2013-03-08-Stuff The Internet Says On Scalability For March 8, 2013
14 0.07576517 17 high scalability-2007-07-16-Paper: Guide to Cost-effective Database Scale-Out using MySQL
15 0.075113155 854 high scalability-2010-07-09-Hot Scalability Links for July 9, 2010
16 0.07292068 594 high scalability-2009-05-08-Eight Best Practices for Building Scalable Systems
17 0.072857589 259 high scalability-2008-02-25-Any Suggestions for the Architecture Template?
18 0.072857589 260 high scalability-2008-02-25-Architecture Template Advice Needed
19 0.071933314 560 high scalability-2009-04-08-Learned lessons from the largest player (Flickr, YouTube, Google, etc)
20 0.071721867 3 high scalability-2007-07-09-LiveJournal Architecture
topicId topicWeight
[(0, 0.081), (1, 0.054), (2, -0.016), (3, -0.016), (4, 0.003), (5, 0.014), (6, -0.009), (7, -0.017), (8, -0.02), (9, 0.002), (10, -0.015), (11, -0.016), (12, 0.006), (13, 0.015), (14, 0.002), (15, -0.021), (16, 0.042), (17, -0.01), (18, -0.037), (19, 0.055), (20, 0.02), (21, -0.01), (22, -0.053), (23, -0.002), (24, -0.039), (25, -0.007), (26, 0.035), (27, 0.037), (28, -0.028), (29, 0.032), (30, 0.018), (31, 0.005), (32, 0.006), (33, 0.069), (34, -0.047), (35, 0.002), (36, -0.012), (37, -0.037), (38, 0.038), (39, -0.031), (40, 0.026), (41, -0.056), (42, -0.042), (43, 0.035), (44, 0.021), (45, 0.028), (46, 0.004), (47, 0.025), (48, 0.01), (49, 0.027)]
simIndex simValue blogId blogTitle
same-blog 1 0.96203244 898 high scalability-2010-09-09-6 Scalability Lessons
Introduction: Jesper Söderlund not only put together a few interesting scalability patterns , he also came up with a few interesting scalability lessons : Lesson #1 . Put Smarty compile and template caches on an active-active DRBD cluster with high load and your servers will DIE! Lesson #2 . Don't use out-of-the-box configurations. Lesson #3 . Single points of contention will eventually become a bottleneck. Lesson #4 . Plan in advance. Lesson #5 . Offload your databases as much as possible. Lesson #6 . File systems matter and can run out of space / inodes. For more details and explanations see the original post.
2 0.64918751 402 high scalability-2008-10-05-Paper: Scalability Design Patterns
Introduction: I have introduced pattern languages in my earlier post on The Pattern Bible for Distributed Computing . Achieving highest possible scalability is a complex combination of many factors. This PLoP 2007 paper presents a pattern language that can be used to make a system highly scalable. The Scalability Pattern Language introduced by Kanwardeep Singh Ahluwalia includes patterns to: Introduce Scalability Optimize Algorithm Add Hardware Add Parallelism Add Intra-Process Parallelism Add Inter-Porcess Parallelism Add Hybrid Parallelism Optimize Decentralization Control Shared Resources Automate Scalability
3 0.64541864 903 high scalability-2010-09-17-Hot Scalability Links For Sep 17, 2010
Introduction: Disqus - Scaling the Worlds Largest Django App. Interesting overview of a commenting system with 75 million comments and 250 million visitors. Lots of good details on how they partition their database, testing, continuous integration, feature switches, caching, delayed signals, and more. Things I learnt tracking a billion events in 24 hours : Know your host, Scaling isn't just servers, My servers need to talk to me more, Kill switches for users, What you don't know is the problem, Don't mix server roles, Know your most important users outside of your site. Tweets of Gold: georgebarnett : I read High Scalability for useful articles about large scaling. Sadly though, nothing useful ever shows up. #NoLongerBothering northscale : wow that is fast! :) RT @cgoldberg: was just running > 100k ops/sec against my 2-node #Membase cluster... zazooom #nosql turbofunctor : The root of many (horizontal) scalability problems is an application level access to a writab
4 0.61943603 400 high scalability-2008-10-01-The Pattern Bible for Distributed Computing
Introduction: Software design patterns are an emerging tool for guiding and documenting system design. Patterns usually describe software abstractions used by advanced designers and programmers in their software. Patterns can provide guidance for designing highly scalable distributed systems. Let's see how! Patterns are in essence solutions to problems. Most of them are expressed in a format called Alexandrian form which draws on constructs used by Christopher Alexander. There are variants but most look like this: The pattern name The problem the pattern is trying to solve Context Solution Examples Design rationale: This tells where the pattern came from, why it works, and why experts use it Patterns rarely stand alone. Each pattern works on a context, and transforms the system in that context to produce a new system in a new context. New problems arise in the new system and context, and the next ‘‘layer’’ of patterns can be applied. A pattern language is a structured col
5 0.60669768 220 high scalability-2008-01-22-The high scalability community
Introduction: Hi, First of all; thanks for a creating a GREAT resource on high scalability architecture. For us building high scalability solutions from the west coast of (tiny) Norway good input on the subject isn't always abundant. Which leads me to my next question; Are there any events or conferences on high scalability / SaaS in the US or internationally that any of you would recommend architects or data center managers to attend?
6 0.60359722 1047 high scalability-2011-05-25-Stuff to Watch from Surge 2010
7 0.60295928 595 high scalability-2009-05-08-Publish-subscribe model does not scale?
8 0.59924406 747 high scalability-2009-11-26-What I'm Thankful For on Thanksgiving
9 0.59773523 275 high scalability-2008-03-14-Problem: Mobbing the Least Used Resource Error
10 0.59134692 566 high scalability-2009-04-13-High Performance Web Pages – Real World Examples: Netflix Case Study
11 0.59034157 1420 high scalability-2013-03-08-Stuff The Internet Says On Scalability For March 8, 2013
12 0.58875829 117 high scalability-2007-10-08-Paper: Understanding and Building High Availability-Load Balanced Clusters
13 0.58695835 1473 high scalability-2013-06-10-The 10 Deadly Sins Against Scalability
14 0.58605367 1352 high scalability-2012-10-31-Gone Fishin': LiveJournal Architecture
15 0.58568412 886 high scalability-2010-08-24-21 Quality Screencasts on Scaling Rails
16 0.58522481 243 high scalability-2008-02-07-clusteradmin.blogspot.com - blog about building and administering clusters
17 0.58443487 1026 high scalability-2011-04-18-6 Ways Not to Scale that Will Make You Hip, Popular and Loved By VCs
18 0.57535392 3 high scalability-2007-07-09-LiveJournal Architecture
19 0.57245034 322 high scalability-2008-05-19-Conference: Infoscale 2008 in Italy (June 4-6)
20 0.56673884 869 high scalability-2010-07-30-Hot Scalability Links for July 30, 2010
topicId topicWeight
[(2, 0.266), (10, 0.052), (61, 0.083), (67, 0.344), (79, 0.084), (85, 0.019)]
simIndex simValue blogId blogTitle
1 0.90163314 1422 high scalability-2013-03-12-If Your System was a Symphony it Might Sound Like This...
Introduction: I am in no way a music expert, but when I listen to Symphony No. 4 by Charles Ives , I imagine it's what a complex software/hardware system might sound like if we could hear its inner workings. Ives uses a lot of riotously competing rhythms in this work. It can sound discordant, yet the effect is deeply layered and eventually harmonious, just like the systems we use, create, and become part of. I was pointed to this piece by someone who said there were two conductors. I'd never heard of such a thing! So I was intrigued. The first version of the performance sounds and looks great, but it unfortunately does not use two conductors. The second version uses two conductors, but is unfortunately just a snippet. It's strikingly odd to see two conductors, but I imagine different parts of our systems using different conductors too, running at different rhythms, sometimes slow, sometimes fast, sometimes there are outbursts, sometimes in vicious conflict. Yet conceptually it all stills seem
same-blog 2 0.85127032 898 high scalability-2010-09-09-6 Scalability Lessons
Introduction: Jesper Söderlund not only put together a few interesting scalability patterns , he also came up with a few interesting scalability lessons : Lesson #1 . Put Smarty compile and template caches on an active-active DRBD cluster with high load and your servers will DIE! Lesson #2 . Don't use out-of-the-box configurations. Lesson #3 . Single points of contention will eventually become a bottleneck. Lesson #4 . Plan in advance. Lesson #5 . Offload your databases as much as possible. Lesson #6 . File systems matter and can run out of space / inodes. For more details and explanations see the original post.
3 0.73588371 376 high scalability-2008-09-03-MapReduce framework Disco
Introduction: Disco is an open-source implementation of the MapReduce framework for distributed computing. It was started at Nokia Research Center as a lightweight framework for rapid scripting of distributed data processing tasks. The Disco core is written in Erlang. The MapReduce jobs in Disco are natively described as Python programs, which makes it possible to express complex algorithmic and data processing tasks often only in tens of lines of code.
4 0.70127952 1545 high scalability-2013-11-08-Stuff The Internet Says On Scalability For November 8th, 2013
Introduction: Hey, it's HighScalability time: Robot elephan t from 1950, which consisted of 9000 parts and could walk 27 mp/h Galaxy contains billions of potentially habitable planets . According to economic theory this should put a downward pressure on rents. Quotable Quotes: Brandon Downey : F*ck these guys. @IEEEorg : Every second 21.6 people get their first mobile device. Mobile is growing 5 times faster than the human population. @littleidea : a distributed system to deploy a distributed systems to deploy a distributed system, bring your own turtles @BenedictEvans : Photos shared/day: Facebook - 350m Snapchat - 350m Whatsapp - 400m Instagram: 55m. @SciencePorn : Price of 1gb of storage over time: 1981 $300000, 1987 $50000, 1990 $10000, 1994 $1000, 1997 $100, 2000 $10, 2004 $1, 2012 $0.10 @kellabyte : If I hear “network partitions are rare on even hundred(s) of node clusters” again I’m going to lose my shit. This fall
5 0.6542201 718 high scalability-2009-10-08-Riak - web-shaped data storage system
Introduction: Update: Short presentation NYC by Bryan Fink demonstrating the riak web-shaped data storage engine Riak is another new and interesting key-value store entrant. Some of the features it offers are: Document-oriented Scalable, decentralized key-value store Standard get , put , and delete operations. Distributed, fault-tolerant storage solution. Configurable levels of consistency, availability, and partition tolerance Support for Erlang, Ruby, PHP, Javascript, Java, Python, HTTP open source and NoSQL Pluggable backends Eventing system Monitoring Inter-cluster replication Links between records that can be traversed. Map/Reduce. Functions are executed on the data node. One interesting difference is that a list keys are required to specify which values are operated on as apposed to running calculations on all values. Related Articles Hacker News Thread . More juicy details on how Riak compares to Cassandra, mongodb, couchdb, etc.
6 0.64912194 86 high scalability-2007-09-09-Clustering Solution
7 0.64838308 776 high scalability-2010-02-12-Hot Scalability Links for February 12, 2010
8 0.64280015 1425 high scalability-2013-03-18-Beyond Threads and Callbacks - Application Architecture Pros and Cons
9 0.64273673 417 high scalability-2008-10-15-Outside.in Scales Up with Engine Yard and moving from PHP to Ruby on Rails
10 0.64231044 910 high scalability-2010-09-30-Facebook and Site Failures Caused by Complex, Weakly Interacting, Layered Systems
11 0.64192599 1001 high scalability-2011-03-09-Google and Netflix Strategy: Use Partial Responses to Reduce Request Sizes
13 0.640504 1413 high scalability-2013-02-27-42 Monster Problems that Attack as Loads Increase
14 0.64039546 844 high scalability-2010-06-18-Paper: The Declarative Imperative: Experiences and Conjectures in Distributed Logic
15 0.64030021 1418 high scalability-2013-03-06-Low Level Scalability Solutions - The Aggregation Collection
16 0.64000499 50 high scalability-2007-07-31-BerkeleyDB & other distributed high performance key-value databases
17 0.63909304 359 high scalability-2008-07-29-Ehcache - A Java Distributed Cache
18 0.63906133 1568 high scalability-2013-12-23-What Happens While Your Brain Sleeps is Surprisingly Like How Computers Stay Sane
19 0.63871312 1456 high scalability-2013-05-13-The Secret to 10 Million Concurrent Connections -The Kernel is the Problem, Not the Solution
20 0.63826716 230 high scalability-2008-01-29-Speed up (Oracle) database code with result caching