high_scalability high_scalability-2009 high_scalability-2009-568 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: There were many talks recently about twitter scalability and their specific choice of language such as Scala to address their existing Ruby based scalability. In this post i tried to provide a more methodical approach for handling twitter scalability challenges that is centered around the right choice of architecture patterns rather then the language itself. The architecture pattern are given in a generic fashion that is not specific to twitter itself and can serve anyone who is looking to build a scalable real time web application in the near future.
sentIndex sentText sentNum sentScore
1 There were many talks recently about twitter scalability and their specific choice of language such as Scala to address their existing Ruby based scalability. [sent-1, score-1.803]
2 In this post i tried to provide a more methodical approach for handling twitter scalability challenges that is centered around the right choice of architecture patterns rather then the language itself. [sent-2, score-2.766]
3 The architecture pattern are given in a generic fashion that is not specific to twitter itself and can serve anyone who is looking to build a scalable real time web application in the near future. [sent-3, score-2.256]
wordName wordTfidf (topN-words)
[('methodical', 0.346), ('twitter', 0.342), ('centered', 0.275), ('choice', 0.25), ('specific', 0.243), ('language', 0.233), ('fashion', 0.218), ('generic', 0.205), ('scala', 0.193), ('tried', 0.169), ('near', 0.164), ('pattern', 0.159), ('ruby', 0.152), ('patterns', 0.147), ('talks', 0.143), ('recently', 0.14), ('serve', 0.136), ('anyone', 0.131), ('architecture', 0.126), ('challenges', 0.124), ('handling', 0.122), ('address', 0.115), ('scalability', 0.115), ('existing', 0.114), ('given', 0.106), ('rather', 0.103), ('future', 0.1), ('provide', 0.091), ('looking', 0.087), ('approach', 0.087), ('right', 0.082), ('around', 0.08), ('real', 0.075), ('post', 0.074), ('scalable', 0.067), ('build', 0.064), ('based', 0.061), ('application', 0.051), ('many', 0.047), ('web', 0.045), ('time', 0.037)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 568 high scalability-2009-04-14-Designing a Scalable Twitter
Introduction: There were many talks recently about twitter scalability and their specific choice of language such as Scala to address their existing Ruby based scalability. In this post i tried to provide a more methodical approach for handling twitter scalability challenges that is centered around the right choice of architecture patterns rather then the language itself. The architecture pattern are given in a generic fashion that is not specific to twitter itself and can serve anyone who is looking to build a scalable real time web application in the near future.
2 0.24629626 556 high scalability-2009-04-05-At Some Point the Cost of Servers Outweighs the Cost of Programmers
Introduction: This is the intriguing quote by Bill Venners in an interview with Twitter's Alex Payne on Twitter's heretical switch from a pure Ruby stack to a Ruby on Rails stack on the front-end and JVM/Scala on the back-end: So performance was also one of the problems with JRuby, which I [Bill Venners] think helps explain better why they'd [Twitter] prefer Scala over Ruby or JRuby for some things. I have often heard Rubyists say that although Ruby is slower than Java, for many things it is plenty fast enough, and they are right. The logic goes further, saying that servers are cheap, and programmers expensive, so it makes sense to tradeoff some runtime performance for programmer productivity. And I think that's very often true too, but not always. If you have enough traffic, at some point the cost of servers outweighs the cost of programmers . I'm not sure whether Twitter is past that point, but they get a lot of traffic. And frankly this isn't an intrinsic tradeoff. Other dynamic languages
3 0.23316926 837 high scalability-2010-06-07-Six Ways Twitter May Reach its Big Hairy Audacious Goal of One Billion Users
Introduction: Twitter has a big hairy audacious goal of reaching one billion users by 2013. Three forces stand against Twitter. The world will end in 2012 . But let's be optimistic and assume we'll make it. Next is Facebook. Currently Facebook is the user leader with over 400 million users . Will Facebook stumble or will they rocket to one billion users before Twitter? And lastly, there's Twitter's "low" starting point and "slow" growth rate. Twitter currently has 106 million registered users and adds about 300,000 new users a day. That doesn't add up to a billion in three years. Twitter needs to triple the number of registered users they add per day. How will Twitter reach its goal of over one billion users served? From recent infrastructure announcements and information gleaned at Chirp ( videos ) and other talks, it has become a little clearer how they hope to reach their billion user goal: 1) Make a Big Hairy Audacious Goal 2) Hire Lots of Quality People 3) Hug Developers and Users 4) D
4 0.22339016 639 high scalability-2009-06-27-Scaling Twitter: Making Twitter 10000 Percent Faster
Introduction: Update 6: Some interesting changes from Twitter's Evan Weaver : everything in RAM now, database is a backup; peaks at 300 tweets/second; every tweet followed by average 126 people; vector cache of tweet IDs; row cache; fragment cache; page cache; keep separate caches; GC makes Ruby optimization resistant so went with Scala; Thrift and HTTP are used internally; 100s internal requests for every external request; rewrote MQ but kept interface the same; 3 queues are used to load balance requests; extensive A/B testing for backwards capability; switched to C memcached client for speed; optimize critical path; faster to get the cached results from the network memory than recompute them locally. Update 5: Twitter on Scala . A Conversation with Steve Jenson, Alex Payne, and Robey Pointer by Bill Venners. A fascinating discussion of why Twitter moved to the Java JVM for their server infrastructure (long lived processes) and why they moved to Scala to program against it (high level langu
5 0.17845884 400 high scalability-2008-10-01-The Pattern Bible for Distributed Computing
Introduction: Software design patterns are an emerging tool for guiding and documenting system design. Patterns usually describe software abstractions used by advanced designers and programmers in their software. Patterns can provide guidance for designing highly scalable distributed systems. Let's see how! Patterns are in essence solutions to problems. Most of them are expressed in a format called Alexandrian form which draws on constructs used by Christopher Alexander. There are variants but most look like this: The pattern name The problem the pattern is trying to solve Context Solution Examples Design rationale: This tells where the pattern came from, why it works, and why experts use it Patterns rarely stand alone. Each pattern works on a context, and transforms the system in that context to produce a new system in a new context. New problems arise in the new system and context, and the next ‘‘layer’’ of patterns can be applied. A pattern language is a structured col
6 0.14690745 323 high scalability-2008-05-19-Twitter as a scalability case study
7 0.14116026 855 high scalability-2010-07-11-So, Why is Twitter Really Not Using Cassandra to Store Tweets?
8 0.13224784 292 high scalability-2008-03-30-Scaling Out MySQL
9 0.1314415 780 high scalability-2010-02-19-Twitter’s Plan to Analyze 100 Billion Tweets
10 0.12469458 906 high scalability-2010-09-22-Applying Scalability Patterns to Infrastructure Architecture
11 0.11949147 1110 high scalability-2011-09-06-Big Data Application Platform
12 0.11919566 402 high scalability-2008-10-05-Paper: Scalability Design Patterns
13 0.11176378 461 high scalability-2008-12-05-Sprinkle - Provisioning Tool to Build Remote Servers
14 0.11148383 544 high scalability-2009-03-18-QCon London 2009: Upgrading Twitter without service disruptions
15 0.11120237 557 high scalability-2009-04-06-A picture is realy worth a thousand word, and also a window in time...
16 0.1018296 305 high scalability-2008-04-21-Google App Engine - what about existing applications?
17 0.098981768 783 high scalability-2010-02-24-Hot Scalability Links for February 24, 2010
18 0.096812449 1251 high scalability-2012-05-24-Build your own twitter like real time analytics - a step by step guide
19 0.091472544 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
20 0.09138304 1491 high scalability-2013-07-15-Ask HS: What's Wrong with Twitter, Why Isn't One Machine Enough?
topicId topicWeight
[(0, 0.128), (1, 0.032), (2, -0.003), (3, 0.021), (4, 0.05), (5, -0.03), (6, -0.088), (7, -0.01), (8, -0.002), (9, 0.075), (10, 0.021), (11, 0.105), (12, 0.019), (13, -0.004), (14, -0.071), (15, -0.051), (16, 0.084), (17, -0.084), (18, -0.031), (19, -0.021), (20, -0.023), (21, -0.041), (22, -0.03), (23, -0.004), (24, 0.019), (25, 0.0), (26, 0.029), (27, -0.11), (28, -0.008), (29, 0.114), (30, 0.073), (31, -0.085), (32, -0.114), (33, 0.076), (34, -0.219), (35, -0.007), (36, -0.044), (37, 0.009), (38, -0.114), (39, -0.011), (40, 0.104), (41, -0.037), (42, -0.034), (43, -0.024), (44, -0.003), (45, -0.002), (46, -0.029), (47, -0.04), (48, 0.058), (49, 0.059)]
simIndex simValue blogId blogTitle
same-blog 1 0.98287719 568 high scalability-2009-04-14-Designing a Scalable Twitter
Introduction: There were many talks recently about twitter scalability and their specific choice of language such as Scala to address their existing Ruby based scalability. In this post i tried to provide a more methodical approach for handling twitter scalability challenges that is centered around the right choice of architecture patterns rather then the language itself. The architecture pattern are given in a generic fashion that is not specific to twitter itself and can serve anyone who is looking to build a scalable real time web application in the near future.
2 0.91040951 323 high scalability-2008-05-19-Twitter as a scalability case study
Introduction: A lot has been said already about Twitter's scalability issues. Many have given Twitter as an anti-pattern of how not to deal with scalability and have suggested different solutions for scaling it. As Twitter is famously a Ruby-on-Rails deployment, this case has also been used as a weapon in the language/platform wars between the RoR and Java camps, and to a lesser degree, also with the LAMP (PHP) camp
3 0.82367492 544 high scalability-2009-03-18-QCon London 2009: Upgrading Twitter without service disruptions
Introduction: Evan Weaver from Twitter presented a talk on Twitter software upgrades, titled Improving running components as part of the Systems that never stop track at QCon London 2009 conference last Friday. The talk focused on several upgrades performed since last May, while Twitter was experiencing serious performance problems.
4 0.7672978 1491 high scalability-2013-07-15-Ask HS: What's Wrong with Twitter, Why Isn't One Machine Enough?
Introduction: Can anyone convincingly explain why properties sporting traffic statistics that may seem in-line with with the capabilities of a single big-iron machine need so many machines in their architecture? This is a common reaction to architecture profiles on High Scalability: I could do all that on a few machines so they must be doing something really stupid. Lo and behold this same reaction also occurred to the article The Architecture Twitter Uses to Deal with 150M Active Users . On Hacker News papsosouid voiced what a lot of people may have been thinking: I really question the current trend of creating big, complex, fragile architectures to "be able to scale". These numbers are a great example of why, the entire thing could run on a single server, in a very straight forward setup. When you are creating a cluster for scalability, and it has less CPU, RAM and IO than a single server, what are you gaining? They are only doing 6k writes a second for crying out loud. This is a s
5 0.75848269 556 high scalability-2009-04-05-At Some Point the Cost of Servers Outweighs the Cost of Programmers
Introduction: This is the intriguing quote by Bill Venners in an interview with Twitter's Alex Payne on Twitter's heretical switch from a pure Ruby stack to a Ruby on Rails stack on the front-end and JVM/Scala on the back-end: So performance was also one of the problems with JRuby, which I [Bill Venners] think helps explain better why they'd [Twitter] prefer Scala over Ruby or JRuby for some things. I have often heard Rubyists say that although Ruby is slower than Java, for many things it is plenty fast enough, and they are right. The logic goes further, saying that servers are cheap, and programmers expensive, so it makes sense to tradeoff some runtime performance for programmer productivity. And I think that's very often true too, but not always. If you have enough traffic, at some point the cost of servers outweighs the cost of programmers . I'm not sure whether Twitter is past that point, but they get a lot of traffic. And frankly this isn't an intrinsic tradeoff. Other dynamic languages
6 0.75060213 837 high scalability-2010-06-07-Six Ways Twitter May Reach its Big Hairy Audacious Goal of One Billion Users
7 0.70798719 574 high scalability-2009-04-20-Some things about Memcached from a Twitter software developer
8 0.69143152 402 high scalability-2008-10-05-Paper: Scalability Design Patterns
9 0.68486601 639 high scalability-2009-06-27-Scaling Twitter: Making Twitter 10000 Percent Faster
11 0.64794052 970 high scalability-2011-01-06-BankSimple Mini-Architecture - Using a Next Generation Toolchain
12 0.64465564 1251 high scalability-2012-05-24-Build your own twitter like real time analytics - a step by step guide
13 0.63783014 855 high scalability-2010-07-11-So, Why is Twitter Really Not Using Cassandra to Store Tweets?
14 0.60295564 1159 high scalability-2011-12-19-How Twitter Stores 250 Million Tweets a Day Using MySQL
15 0.57292116 780 high scalability-2010-02-19-Twitter’s Plan to Analyze 100 Billion Tweets
16 0.54076499 783 high scalability-2010-02-24-Hot Scalability Links for February 24, 2010
17 0.53434294 400 high scalability-2008-10-01-The Pattern Bible for Distributed Computing
18 0.53353935 1148 high scalability-2011-11-29-DataSift Architecture: Realtime Datamining at 120,000 Tweets Per Second
19 0.51849592 363 high scalability-2008-08-12-Strategy: Limit The New, Not The Old
20 0.51407814 1004 high scalability-2011-03-14-Twitter by the Numbers - 460,000 New Accounts and 140 Million Tweets Per Day
topicId topicWeight
[(1, 0.164), (2, 0.314), (49, 0.051), (54, 0.117), (61, 0.159), (79, 0.053)]
simIndex simValue blogId blogTitle
same-blog 1 0.97148353 568 high scalability-2009-04-14-Designing a Scalable Twitter
Introduction: There were many talks recently about twitter scalability and their specific choice of language such as Scala to address their existing Ruby based scalability. In this post i tried to provide a more methodical approach for handling twitter scalability challenges that is centered around the right choice of architecture patterns rather then the language itself. The architecture pattern are given in a generic fashion that is not specific to twitter itself and can serve anyone who is looking to build a scalable real time web application in the near future.
2 0.93500674 877 high scalability-2010-08-12-Designing Web Applications for Scalability
Introduction: I can’t even count the number of times that I’ve heard this phrase: “don’t worry about scaling your web application, worry about visitor (or customer) acquisition.” My response to this is always that you don’t need to choose one or the other, you can do both! In this post, I’m going to go over some of the strategies I’ve used to architect web applications for scalability, right from the start of the design process, in such a way that I’m prepared to scale when I need to, but not forced into doing so before its necessary. Easing the transition from small scale to large scale can be made much easier by choosing the right technologies and implementing the right coding patterns up front. You can read the full store here .
3 0.93313295 685 high scalability-2009-08-20-Dependency Injection and AOP frameworks for .NET
Introduction: We're looking to implement a framework to do Dependency Injection and AOP for a new solution we're working on. It will likely get hit pretty hard, so we'd like to chose a framework that's proven to scale well, and operates well under pressure. Right now, we're looking closely at Spring.NET, Castle Project's Windsor framework, and Unity. Does anyone have any feedback on implementing any of these in large, high traffic environments?
4 0.93075228 724 high scalability-2009-10-19-Drupal's Scalability Makeover - You give up some control and you get back scalability
Introduction: Drupal 7 is having a scalability makeover. Karoly Negyesi, Drupal Core Developer and Public Development Team Lead, explains the process in this video: Drupal 7 APIs, scalability mindset . Karoly states the general theme of the changes as: You give up some control and you get back scalability . An interesting comment on the politics of scalability? Makeover may not be quite the right word though. A makeover implies a cosmetic change, looking better by changing the surface. Drupal's changes will go deeper than that, right to Drupal's core. It's a genuine and authentic change that will hopefully allow one of the Internet's most venerable Content Management Systems (CMSs) to compete with a constant stream of younger and sexier models. Drupal is based on an older LAMP stack approach where PHP modules are scooped up and merged together each time a request is made to Drupal. Drupal's most intriguing idea is how it is built, expands, and changes by weaving together a single system out
5 0.9298684 569 high scalability-2009-04-14-Scalability resources
Introduction: I found this resources: High Scalable Architecture: - YouTube Architecture - Facebook Chat Architecture - Amazon Architecture Blogs: - Scalability Guidelines for building scalable software system (part 1) - Scalability Guidelines for building scalable software system (part 2) - Scalability Guidelines for building scalable software system (part 3) - Scalability Worst Practices - how to minimize load time for fast user experiences - Scalability principles - Challanges for Developing Enterprise Application on the Cloud - high-performance web page real-world examples netflix case study - Intro to Caching,Caching algorithms and caching frameworks part 1 - Amdahl’s low - How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale - Top 25 Most Dangerous Programming Mistakes
6 0.92948091 64 high scalability-2007-08-10-How do we make a large real-time search engine?
7 0.9293499 86 high scalability-2007-09-09-Clustering Solution
8 0.92889887 240 high scalability-2008-02-05-Handling of Session for a site running from more than 1 data center
9 0.92864442 886 high scalability-2010-08-24-21 Quality Screencasts on Scaling Rails
10 0.92774332 933 high scalability-2010-11-01-Hot Trend: Move Behavior to Data for a New Interactive Application Architecture
11 0.92705899 235 high scalability-2008-02-02-The case against ORM Frameworks in High Scalability Architectures
12 0.92644066 961 high scalability-2010-12-21-SQL + NoSQL = Yes !
13 0.92605662 379 high scalability-2008-09-04-Database question for upcoming project
14 0.92572695 1135 high scalability-2011-10-31-15 Ways to Make Your Application Feel More Responsive under Google App Engine
15 0.92506629 950 high scalability-2010-11-30-NoCAP – Part III – GigaSpaces clustering explained..
16 0.92502517 1538 high scalability-2013-10-28-Design Decisions for Scaling Your High Traffic Feeds
17 0.92317599 1401 high scalability-2013-02-06-Super Bowl Advertisers Ready for the Traffic? Nope..It's Lights Out.
18 0.92315745 1362 high scalability-2012-11-26-BigData using Erlang, C and Lisp to Fight the Tsunami of Mobile Data
20 0.92275047 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month