high_scalability high_scalability-2011 high_scalability-2011-970 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: I know people are always interested in what others are using to build their systems. Alex Payne , CTO of the new startup BankSimple , gives us a quick hit on their toolchain choices in this Quora thread . BankSimple positions itself as a customer-focused alternative to online banking. You may remember Alex from the early days of Twitter . Alex was always helpful to me on Twitter's programmer support list, so I really wish them well. Alex is also a bit of an outside the box thinker, which is reflected in some of their choices: The JVM acts as a convergence platform for these languages: Scala - ideal for writing performance-sensitive components that need the safety and expressiveness of the language's advanced type system. Clojure - rapidly prototype in a more dynamic language while still offering the benefits of functional programming. JRuby - makes available a bunch of great libraries and frameworks for doing frontend web development, like Rails and Pa
sentIndex sentText sentNum sentScore
1 I know people are always interested in what others are using to build their systems. [sent-1, score-0.077]
2 Alex Payne , CTO of the new startup BankSimple , gives us a quick hit on their toolchain choices in this Quora thread . [sent-2, score-0.392]
3 BankSimple positions itself as a customer-focused alternative to online banking. [sent-3, score-0.111]
4 You may remember Alex from the early days of Twitter . [sent-4, score-0.079]
5 Alex was always helpful to me on Twitter's programmer support list, so I really wish them well. [sent-5, score-0.335]
6 Clojure - rapidly prototype in a more dynamic language while still offering the benefits of functional programming. [sent-7, score-0.519]
7 JRuby - makes available a bunch of great libraries and frameworks for doing frontend web development, like Rails and Padrino. [sent-8, score-0.439]
8 After all, using different languages for different purposes makes as much sense as using different databases for different purposes. [sent-15, score-0.864]
9 Related Articles Node and Scaling in the Small vs Scaling in the Large by Alex Payne Alex Payne interviewed at OSCON 2010 . [sent-16, score-0.17]
wordName wordTfidf (topN-words)
[('alex', 0.402), ('payne', 0.313), ('mootools', 0.181), ('largeby', 0.181), ('oscon', 0.17), ('interviewed', 0.17), ('bucks', 0.163), ('toolchain', 0.163), ('choices', 0.159), ('expressiveness', 0.157), ('thinker', 0.148), ('languages', 0.146), ('reflected', 0.144), ('experimenting', 0.138), ('different', 0.133), ('convergence', 0.131), ('clojure', 0.129), ('quora', 0.119), ('language', 0.115), ('positions', 0.111), ('purposes', 0.11), ('acts', 0.109), ('postgres', 0.107), ('safety', 0.107), ('frontend', 0.106), ('prototype', 0.097), ('wish', 0.096), ('ideal', 0.096), ('scala', 0.095), ('helpful', 0.09), ('cto', 0.089), ('bunch', 0.086), ('libraries', 0.086), ('rails', 0.085), ('frameworks', 0.085), ('jvm', 0.085), ('offering', 0.081), ('functional', 0.079), ('remember', 0.079), ('alternative', 0.078), ('always', 0.077), ('makes', 0.076), ('rapidly', 0.074), ('stop', 0.074), ('advanced', 0.073), ('benefits', 0.073), ('programmer', 0.072), ('javascript', 0.072), ('quick', 0.07), ('outside', 0.07)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 970 high scalability-2011-01-06-BankSimple Mini-Architecture - Using a Next Generation Toolchain
Introduction: I know people are always interested in what others are using to build their systems. Alex Payne , CTO of the new startup BankSimple , gives us a quick hit on their toolchain choices in this Quora thread . BankSimple positions itself as a customer-focused alternative to online banking. You may remember Alex from the early days of Twitter . Alex was always helpful to me on Twitter's programmer support list, so I really wish them well. Alex is also a bit of an outside the box thinker, which is reflected in some of their choices: The JVM acts as a convergence platform for these languages: Scala - ideal for writing performance-sensitive components that need the safety and expressiveness of the language's advanced type system. Clojure - rapidly prototype in a more dynamic language while still offering the benefits of functional programming. JRuby - makes available a bunch of great libraries and frameworks for doing frontend web development, like Rails and Pa
2 0.19591323 556 high scalability-2009-04-05-At Some Point the Cost of Servers Outweighs the Cost of Programmers
Introduction: This is the intriguing quote by Bill Venners in an interview with Twitter's Alex Payne on Twitter's heretical switch from a pure Ruby stack to a Ruby on Rails stack on the front-end and JVM/Scala on the back-end: So performance was also one of the problems with JRuby, which I [Bill Venners] think helps explain better why they'd [Twitter] prefer Scala over Ruby or JRuby for some things. I have often heard Rubyists say that although Ruby is slower than Java, for many things it is plenty fast enough, and they are right. The logic goes further, saying that servers are cheap, and programmers expensive, so it makes sense to tradeoff some runtime performance for programmer productivity. And I think that's very often true too, but not always. If you have enough traffic, at some point the cost of servers outweighs the cost of programmers . I'm not sure whether Twitter is past that point, but they get a lot of traffic. And frankly this isn't an intrinsic tradeoff. Other dynamic languages
3 0.12589781 639 high scalability-2009-06-27-Scaling Twitter: Making Twitter 10000 Percent Faster
Introduction: Update 6: Some interesting changes from Twitter's Evan Weaver : everything in RAM now, database is a backup; peaks at 300 tweets/second; every tweet followed by average 126 people; vector cache of tweet IDs; row cache; fragment cache; page cache; keep separate caches; GC makes Ruby optimization resistant so went with Scala; Thrift and HTTP are used internally; 100s internal requests for every external request; rewrote MQ but kept interface the same; 3 queues are used to load balance requests; extensive A/B testing for backwards capability; switched to C memcached client for speed; optimize critical path; faster to get the cached results from the network memory than recompute them locally. Update 5: Twitter on Scala . A Conversation with Steve Jenson, Alex Payne, and Robey Pointer by Bill Venners. A fascinating discussion of why Twitter moved to the Java JVM for their server infrastructure (long lived processes) and why they moved to Scala to program against it (high level langu
4 0.11796419 665 high scalability-2009-07-29-Strategy: Let Google and Yahoo Host Your Ajax Library - For Free
Introduction: Update: Offloading ALL JS Files To Google . Now you can let Google serve all your javascript files. This article tells you how to do it (upload to Google Code Project) and why it's a big win (cheap, fast, caching, parallel downloads, save bandwidth). Don't have a CDN? Why not let Google and Yahoo be your CDN? At least for Ajax libraries. No charge. Google runs a content distribution network and loading architecture for the most popular open source JavaScript libraries , which include: jQuery, prototype, script.aculo.us, MooTools, and dojo. The idea is web pages directly include your library of choice from Google's global, fast, and highly available network. Some have found much better performance and others experienced slower performance. My guess is the performance may be slower if your data center is close to you, but far away users will be much happier. Some negatives: not all libraries are included, you'll load more than you need because all functionality is included. Yahoo
5 0.11769405 659 high scalability-2009-07-20-A Scalability Lament
Introduction: In Scalability issues for dummies Alex Barrera talks movingly about the challenges he faces trying to scale his startup inkzee as the lone developer. Inkzee is an online news reader that automatically groups similar topics . This is a cool problem and is one you know right away is going to have some killer scalability problems as the number of feeds and the number of users increase. And these problems lead to the point of the post, to explain here what are scalability problems and how deep the repercussions are for a small company , which Alex does admirably. Some takeaways: Sites are composed of a frontend and backend. The backend isn't visible to users, but it does all the work and this is where the scalability problems show up. As more and more users use a site it becomes slow because more users reveal bottlenecks in the system that weren't visible before. There can be many many reasons for these bottlenecks. They are often very hard to find because the backend sy
6 0.11058214 306 high scalability-2008-04-21-The Search for the Source of Data - How SimpleDB Differs from a RDBMS
7 0.10922723 459 high scalability-2008-12-03-Java World Interview on Scalability and Other Java Scalability Secrets
8 0.098245651 1145 high scalability-2011-11-18-Stuff The Internet Says On Scalability For November 18, 2011
9 0.093211368 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
10 0.089414395 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
11 0.083866447 1019 high scalability-2011-04-08-Stuff The Internet Says On Scalability For April 8, 2011
12 0.079838395 837 high scalability-2010-06-07-Six Ways Twitter May Reach its Big Hairy Audacious Goal of One Billion Users
13 0.077844545 1469 high scalability-2013-06-03-GOV.UK - Not Your Father's Stack
14 0.076457135 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
15 0.075459018 993 high scalability-2011-02-22-Is Node.js Becoming a Part of the Stack? SimpleGeo Says Yes.
16 0.074837536 1439 high scalability-2013-04-12-Stuff The Internet Says On Scalability For April 12, 2013
17 0.07286039 517 high scalability-2009-02-21-Google AppEngine - A Second Look
18 0.07223887 783 high scalability-2010-02-24-Hot Scalability Links for February 24, 2010
19 0.071992241 568 high scalability-2009-04-14-Designing a Scalable Twitter
20 0.071454138 869 high scalability-2010-07-30-Hot Scalability Links for July 30, 2010
topicId topicWeight
[(0, 0.124), (1, 0.037), (2, 0.004), (3, 0.017), (4, 0.032), (5, -0.027), (6, -0.07), (7, -0.005), (8, 0.01), (9, 0.01), (10, -0.004), (11, 0.015), (12, 0.016), (13, -0.019), (14, -0.044), (15, -0.062), (16, 0.016), (17, -0.02), (18, -0.024), (19, -0.04), (20, -0.001), (21, -0.044), (22, -0.038), (23, -0.008), (24, 0.018), (25, 0.019), (26, 0.024), (27, -0.065), (28, -0.023), (29, 0.042), (30, -0.029), (31, -0.017), (32, -0.06), (33, 0.007), (34, -0.064), (35, -0.047), (36, -0.049), (37, 0.035), (38, -0.048), (39, 0.01), (40, 0.01), (41, -0.069), (42, 0.039), (43, -0.029), (44, -0.043), (45, -0.012), (46, 0.019), (47, -0.03), (48, 0.043), (49, 0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.94988453 970 high scalability-2011-01-06-BankSimple Mini-Architecture - Using a Next Generation Toolchain
Introduction: I know people are always interested in what others are using to build their systems. Alex Payne , CTO of the new startup BankSimple , gives us a quick hit on their toolchain choices in this Quora thread . BankSimple positions itself as a customer-focused alternative to online banking. You may remember Alex from the early days of Twitter . Alex was always helpful to me on Twitter's programmer support list, so I really wish them well. Alex is also a bit of an outside the box thinker, which is reflected in some of their choices: The JVM acts as a convergence platform for these languages: Scala - ideal for writing performance-sensitive components that need the safety and expressiveness of the language's advanced type system. Clojure - rapidly prototype in a more dynamic language while still offering the benefits of functional programming. JRuby - makes available a bunch of great libraries and frameworks for doing frontend web development, like Rails and Pa
2 0.84934783 556 high scalability-2009-04-05-At Some Point the Cost of Servers Outweighs the Cost of Programmers
Introduction: This is the intriguing quote by Bill Venners in an interview with Twitter's Alex Payne on Twitter's heretical switch from a pure Ruby stack to a Ruby on Rails stack on the front-end and JVM/Scala on the back-end: So performance was also one of the problems with JRuby, which I [Bill Venners] think helps explain better why they'd [Twitter] prefer Scala over Ruby or JRuby for some things. I have often heard Rubyists say that although Ruby is slower than Java, for many things it is plenty fast enough, and they are right. The logic goes further, saying that servers are cheap, and programmers expensive, so it makes sense to tradeoff some runtime performance for programmer productivity. And I think that's very often true too, but not always. If you have enough traffic, at some point the cost of servers outweighs the cost of programmers . I'm not sure whether Twitter is past that point, but they get a lot of traffic. And frankly this isn't an intrinsic tradeoff. Other dynamic languages
3 0.73033166 323 high scalability-2008-05-19-Twitter as a scalability case study
Introduction: A lot has been said already about Twitter's scalability issues. Many have given Twitter as an anti-pattern of how not to deal with scalability and have suggested different solutions for scaling it. As Twitter is famously a Ruby-on-Rails deployment, this case has also been used as a weapon in the language/platform wars between the RoR and Java camps, and to a lesser degree, also with the LAMP (PHP) camp
4 0.7147693 568 high scalability-2009-04-14-Designing a Scalable Twitter
Introduction: There were many talks recently about twitter scalability and their specific choice of language such as Scala to address their existing Ruby based scalability. In this post i tried to provide a more methodical approach for handling twitter scalability challenges that is centered around the right choice of architecture patterns rather then the language itself. The architecture pattern are given in a generic fashion that is not specific to twitter itself and can serve anyone who is looking to build a scalable real time web application in the near future.
5 0.68202871 1469 high scalability-2013-06-03-GOV.UK - Not Your Father's Stack
Introduction: I'm not sure what I was expecting the stack GOV.UK used at launch to look like. Maybe some messenger owls and lots of cobwebs? But not so at all. So much not so I thought any organization looking at their own stack for ideas could learn something from the considered choices of others. The diversity of technologies used was surprising. They use "at least five different programming languages, three separate database types, two versions of an operating system." Some may think of this as a weakness, but they think it a strength: The reason we operate such a diverse ecosystem is that we are focused on solving real problems. Our first task is to understand the problem or need we are solving and then to choose the best tool for the job. If we restrict ourselves to moulding the need to the tools we already have, then we risk not solving the initial problem in the best way possible for the user. By restricting software diversity or enforcing rigid organisational standards on a project,
6 0.67829376 1625 high scalability-2014-04-03-Leslie Lamport to Programmers: You're Doing it Wrong
7 0.65634805 639 high scalability-2009-06-27-Scaling Twitter: Making Twitter 10000 Percent Faster
8 0.64150649 307 high scalability-2008-04-21-Using Google AppEngine for a Little Micro-Scalability
9 0.63349956 1145 high scalability-2011-11-18-Stuff The Internet Says On Scalability For November 18, 2011
10 0.62539709 993 high scalability-2011-02-22-Is Node.js Becoming a Part of the Stack? SimpleGeo Says Yes.
11 0.6146155 574 high scalability-2009-04-20-Some things about Memcached from a Twitter software developer
12 0.61309248 783 high scalability-2010-02-24-Hot Scalability Links for February 24, 2010
13 0.60943437 855 high scalability-2010-07-11-So, Why is Twitter Really Not Using Cassandra to Store Tweets?
14 0.60931331 1491 high scalability-2013-07-15-Ask HS: What's Wrong with Twitter, Why Isn't One Machine Enough?
15 0.60861754 837 high scalability-2010-06-07-Six Ways Twitter May Reach its Big Hairy Audacious Goal of One Billion Users
16 0.607584 1515 high scalability-2013-09-11-Ten Lessons from GitHub’s First Year in 2008
18 0.60369426 1024 high scalability-2011-04-15-Stuff The Internet Says On Scalability For April 15, 2011
19 0.60342211 544 high scalability-2009-03-18-QCon London 2009: Upgrading Twitter without service disruptions
20 0.59250766 1423 high scalability-2013-03-13-Iron.io Moved From Ruby to Go: 28 Servers Cut and Colossal Clusterf**ks Prevented
topicId topicWeight
[(1, 0.147), (2, 0.237), (61, 0.022), (79, 0.047), (94, 0.443)]
simIndex simValue blogId blogTitle
1 0.99695903 605 high scalability-2009-05-22-Distributed content system with bandwidth balancing
Introduction: I am looking for a way to distribute files over servers in different physical locations. My main concern is that I have bandwidth limitations on each location, and wish to spread the bandwidth load evenly. Atm. I just have 1:1 copies of the files on all servers, and have the application pick a random server to serve the file as a temp fix... It's a small video streaming service. I want to spoonfeed the stream to the client with a max bandwidth output, and support seek. At present I use php to limit the network stream, and read the file at a given offset sendt as a get parameter from the player for seek. It's psuedo streaming, but it works. I have been looking at MogileFS, which would solve the storage part. With MogileFS I can make use of my current php solution as it supports lighttpd and apache (with mod_rewrite or similar). However I don't see how I can apply MogileFS to check for bandwidth % usage? Any reccomendations for how I can solve this?
2 0.99549651 115 high scalability-2007-10-07-Using ThreadLocal to pass context information around in web applications
Introduction: Hi, In java web servers, each http request is handled by a thread in thread pool. So for a Servlet handling the request, a thread is assigned. It is tempting (and very convinient) to keep context information in the threadlocal variable. I recently had a requirement where we need to assign logged in user id and timestamp to request sent to web services. Because we already had the code in place, it was extremely difficult to change the method signatures to pass user id everywhere. The solution I thought is class ReferenceIdGenerator { public static setReferenceId(String login) { threadLocal.set(login + System.currentMillis()); } public static String getReferenceId() { return threadLocal.get(); } private static ThreadLocal threadLocal = new ThreadLocal(); } class MySevlet { void service(.....) { HttpSession session = request.getSession(false); String userId = session.get("userId"); ReferenceIdGenerator.setRefernceId(userId
3 0.97979105 1601 high scalability-2014-02-25-Peter Norvig's 9 Master Steps to Improving a Program
Introduction: Inspired by a xkcd comic , Peter Norvig , Director of Research at Google and all around interesting and nice guy, has created an above par code kata involving a regex program that demonstrates the core inner loop of many successful systems profiled on HighScalability. The original code is at xkcd 1313: Regex Golf , which comes up with an algorithm to find a short regex that matches the winners and not the losers from two arbitrary lists. The Python code is readable, the process is TDDish, and the problem, which sounds simple, but soon explodes into regex weirdness, as does most regex code. If you find regular expressions confusing you'll definitely benefit from Peter's deliberate strategy for finding a regex. The post demonstrating the iterated improvement of the program is at xkcd 1313: Regex Golf (Part 2: Infinite Problems) . As with most first solutions it wasn't optimal. To improve the program Peter recommends the following steps: Profiling : Figure out wher
4 0.97620428 1305 high scalability-2012-08-16-Paper: A Provably Correct Scalable Concurrent Skip List
Introduction: In MemSQL Architecture we learned one of the core strategies MemSQL uses to achieve their need for speed is lock-free skip lists. Skip lists are used to efficiently handle range queries. Making the skip-lists lock-free helps eliminate contention and make writes fast. If this all sounds a little pie-in-the-sky then here's a very good paper on the subject that might help make it clearer: A Provably Correct Scalable Concurrent Skip List . From the abstract: We propose a new concurrent skip list algorithm distinguished by a combination of simplicity and scalability. The algorithm employs optimistic synchronization, searching without acquiring locks, followed by short lock-based validation before adding or removing nodes. It also logically removes an item before physically unlinking it. Unlike some other concurrent skip list algorithms, this algorithm preserves the skiplist properties at all times, which facilitates reasoning about its correctness. Experimental evidence shows that
5 0.97606641 559 high scalability-2009-04-07-Six Lessons Learned Deploying a Large-scale Infrastructure in Amazon EC2
Introduction: Lessons learned from OpenX's large-scale deployment to Amazon EC2: Expect failures; what's more, embrace them Fully automate your infrastructure deployments Design your infrastructure so that it scales horizontally Establish clear measurable goals Be prepared to quickly identify and eliminate bottlenecks Play wack-a-mole for a while, until things get stable
same-blog 8 0.91386735 970 high scalability-2011-01-06-BankSimple Mini-Architecture - Using a Next Generation Toolchain
9 0.9133783 834 high scalability-2010-06-01-Web Speed Can Push You Off of Google Search Rankings! What Can You Do?
10 0.9072867 241 high scalability-2008-02-05-SLA monitoring
11 0.89329886 827 high scalability-2010-05-14-Hot Scalability Links for May 14, 2010
12 0.89327061 1025 high scalability-2011-04-16-The NewSQL Market Breakdown
13 0.88817436 1084 high scalability-2011-07-22-Stuff The Internet Says On Scalability For July 22, 2011
14 0.88320845 1412 high scalability-2013-02-25-SongPop Scales to 1 Million Active Users on GAE, Showing PaaS is not Passé
15 0.8643496 1223 high scalability-2012-04-06-Stuff The Internet Says On Scalability For April 6, 2012
16 0.86235827 1023 high scalability-2011-04-14-Strategy: Cache Application Start State to Reduce Spin-up Times
17 0.85127795 78 high scalability-2007-09-01-2 tier switch selection for colocation
18 0.84949815 1174 high scalability-2012-01-13-Stuff The Internet Says On Scalability For January 13, 2012
19 0.84262276 266 high scalability-2008-03-04-Manage Downtime Risk by Connecting Multiple Data Centers into a Secure Virtual LAN
20 0.84253156 863 high scalability-2010-07-22-How can we spark the movement of research out of the Ivory Tower and into production?