high_scalability high_scalability-2013 high_scalability-2013-1451 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Hey, it's HighScalability time: ( Giant Hurricane on Saturn , here's one in New Orleans ) 1,966,080 cores : Time Warp synchronization protocol using up to 7.8M MPI tasks on 1,966,080 cores of the {Sequoia} Blue Gene/Q supercomputer system. 33 trillion events processed in 65 seconds yielding a peak event-rate in excess of 504 billion events/second using 120 racks of Sequoia. Quotable Quotes: Thad Starner : the longer accessing a device exceeds 2s, the more its actually usage would decrease exponentially. Thus, he made a claim that wrist watch interface always sitting on one's wrist ready to use should be more successful than mobile phones which have to pulled out of the pocket. @joedevon : We came for scalability but we stayed for agility #NoSQL @jahmailay : "Our user base is exploding. I really wish we spent more time on scalability instead of features customers don't use." - Everybody, always. @bsletten : I don’t think it is a
sentIndex sentText sentNum sentScore
1 Thus, he made a claim that wrist watch interface always sitting on one's wrist ready to use should be more successful than mobile phones which have to pulled out of the pocket. [sent-5, score-0.294]
2 Prediction: We'll laugh at all this filter control talk once we have all of Google's datacenters and knowledge graph software implanted in our heads. [sent-17, score-0.07]
3 Here's a short review by me on CyberStorm by Matthew Mather. [sent-22, score-0.073]
4 " CyberStorm is a chilling blow by blow of what could happen in a real cyber attack. [sent-24, score-0.18]
5 As a programmer it's the implied idea of a kind of Crises OS built on a mesh of smartphones that I found most fascinating. [sent-25, score-0.134]
6 Not much seems to be done in this area and even the how-to of writing such applications is rarely discussed. [sent-26, score-0.074]
7 Now that's not a scalable business model for artists : "Galaxie 500's 'Tugboat' was played 7,800 times on Pandora in the first quarter of 2012, for which its three songwriters were paid a collective total of 21 cents, or seven cents each. [sent-28, score-0.082]
8 Example of optimizing a clearly stated graph computation with not 30X, but rather 20% overhead, though it took a lot of work. [sent-30, score-0.07]
9 " Security corner: great plain-English explanation of a rainbow table ; even better series of posts by John Graham-Cumming on one way functions . [sent-42, score-0.133]
10 Some lessons: CLOBs in a database are rarely ever a good thing. [sent-58, score-0.074]
11 Having fewer or even no tests is probably better than having really slow tests and definitely better than having flaky tests. [sent-60, score-0.207]
12 : If you don't program a way for the disk source to feel the back pressure from the slow mobile connection, it will read the data full-speed and flood the server's memory by buffering everything. [sent-67, score-0.078]
13 Using Vector Interfaces to Deliver Millions of IOPS from a Networked Key-value Storage Server : To address this increasing software-I/O gap, we propose using vector interfaces in high-performance networked systems. [sent-73, score-0.419]
14 Vector interfaces organize requests and computation in a distributed system into collections of similar but independent units of work, thereby providing opportunities to amortize and eliminate the redundant work common in many high-performance systems. [sent-74, score-0.123]
15 By integrating vector interfaces into storage and RPC components, we demonstrate that a single key-value storage server can provide 1. [sent-75, score-0.333]
16 6 million requests per second with a median latency below one millisecond, over fourteen times greater than the same software absent the use of vector interfaces. [sent-76, score-0.344]
17 In Search of an Understandable Consensus Algorithm : Raft is a consensus algorithm for managing a replicated log. [sent-77, score-0.084]
18 It produces a result equivalent to Paxos, and it is as efficient as Paxos, but its structure is different from Paxos; this makes Raft more understandable than Paxos and also provides a better foundation for building practical systems. [sent-78, score-0.163]
19 SSMalloc: A Low-latency, Locality-conscious Memory : This paper presents a new memory allocator that provide low-latency and locality-conscious memory management with stable performance scalability even with a large number of application threads. [sent-79, score-0.156]
20 The key design decisions underlying SSMalloc include: 1) providing low and predictable latency for memory management operations through carefully minimized critical path; 2) minimizing mmap system calls that might be contended in kernel; 3) adopting lock-free and mostly wait-free algorithms. [sent-80, score-0.141]
wordName wordTfidf (topN-words)
[('vector', 0.21), ('paxos', 0.165), ('ssmalloc', 0.147), ('wrist', 0.147), ('smartphones', 0.134), ('raft', 0.133), ('interfaces', 0.123), ('satellites', 0.119), ('matthew', 0.101), ('understandable', 0.093), ('blow', 0.09), ('networked', 0.086), ('consensus', 0.084), ('cents', 0.082), ('memory', 0.078), ('rarely', 0.074), ('review', 0.073), ('azure', 0.071), ('better', 0.07), ('graph', 0.07), ('eval', 0.067), ('subtitle', 0.067), ('fronting', 0.067), ('inspecting', 0.067), ('flaky', 0.067), ('fourteen', 0.067), ('cohort', 0.067), ('absent', 0.067), ('sigh', 0.067), ('curate', 0.067), ('doit', 0.067), ('conviva', 0.067), ('cybersecurity', 0.067), ('honesty', 0.067), ('onmaking', 0.067), ('orbit', 0.067), ('srinivasan', 0.067), ('shards', 0.066), ('clobs', 0.063), ('lover', 0.063), ('ashort', 0.063), ('useredis', 0.063), ('ouch', 0.063), ('rainbow', 0.063), ('pandora', 0.063), ('characterization', 0.063), ('hurricane', 0.063), ('singularity', 0.063), ('contended', 0.063), ('ofscaling', 0.063)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1451 high scalability-2013-05-03-Stuff The Internet Says On Scalability For May 3, 2013
Introduction: Hey, it's HighScalability time: ( Giant Hurricane on Saturn , here's one in New Orleans ) 1,966,080 cores : Time Warp synchronization protocol using up to 7.8M MPI tasks on 1,966,080 cores of the {Sequoia} Blue Gene/Q supercomputer system. 33 trillion events processed in 65 seconds yielding a peak event-rate in excess of 504 billion events/second using 120 racks of Sequoia. Quotable Quotes: Thad Starner : the longer accessing a device exceeds 2s, the more its actually usage would decrease exponentially. Thus, he made a claim that wrist watch interface always sitting on one's wrist ready to use should be more successful than mobile phones which have to pulled out of the pocket. @joedevon : We came for scalability but we stayed for agility #NoSQL @jahmailay : "Our user base is exploding. I really wish we spent more time on scalability instead of features customers don't use." - Everybody, always. @bsletten : I don’t think it is a
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
4 0.14134064 357 high scalability-2008-07-26-Google's Paxos Made Live – An Engineering Perspective
Introduction: This is an unusually well written and useful paper . It talks in detail about experiences implementing a complex project, something we don't see very often. They shockingly even admit that creating a working implementation of Paxos was more difficult than just translating the pseudo code. Imagine that, programmers aren't merely typists! I particularly like the explanation of the Paxos algorithm and why anyone would care about it, working with disk corruption, using leases to support simultaneous reads, using epoch numbers to indicate a new master election, using snapshots to prevent unbounded logs, using MultiOp to implement database transactions, how they tested the system, and their openness with the various problems they had. A lot to learn here. From the paper: We describe our experience building a fault-tolerant data-base using the Paxos consensus algorithm. Despite the existing literature in the field, building such a database proved to be non-trivial. We describe selected alg
5 0.13842261 768 high scalability-2010-02-01-What Will Kill the Cloud?
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. If datacenters are the new castles, then what will be the new gunpowder? As soon as gunpowder came on the scene, castles, which are defensive structures, quickly became the future's cold, drafty hotels. Gunpowder fueled cannon balls make short work of castle walls. There's a long history of "gunpowder" type inventions in the tech industry. PCs took out the timeshare model. The cloud is taking out the PC model. There must be something that will take out the cloud. Right now it's hard to believe the cloud will one day be no more. They seem so much the future, but something will transcend the cloud. We even have a law that says so: Bell's Law of Computer Classes which holds that roughly every decade a new, lower priced computer class forms based on a new programming platform, network, and interface resulting in new usage and the establishment of
6 0.13175128 1498 high scalability-2013-08-07-RAFT - In Search of an Understandable Consensus Algorithm
7 0.13144436 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
8 0.12953444 1345 high scalability-2012-10-22-Spanner - It's About Programmers Building Apps Using SQL Semantics at NoSQL Scale
9 0.12846544 1630 high scalability-2014-04-11-Stuff The Internet Says On Scalability For April 11th, 2014
10 0.12644976 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
11 0.12557034 661 high scalability-2009-07-25-Latency is Everywhere and it Costs You Sales - How to Crush it
12 0.12544636 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
13 0.122924 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
14 0.12257914 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
15 0.1217839 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
16 0.11980842 1637 high scalability-2014-04-25-Stuff The Internet Says On Scalability For April 25th, 2014
17 0.11897767 1576 high scalability-2014-01-10-Stuff The Internet Says On Scalability For January 10th, 2014
18 0.11780894 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
19 0.11744554 1036 high scalability-2011-05-06-Stuff The Internet Says On Scalability For May 6th, 2011
20 0.11693563 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
topicId topicWeight
[(0, 0.254), (1, 0.13), (2, 0.006), (3, 0.08), (4, 0.006), (5, 0.005), (6, -0.026), (7, 0.071), (8, -0.039), (9, 0.026), (10, 0.004), (11, -0.043), (12, -0.018), (13, -0.002), (14, -0.02), (15, 0.032), (16, 0.036), (17, 0.045), (18, -0.028), (19, -0.035), (20, -0.015), (21, -0.007), (22, -0.06), (23, 0.017), (24, -0.053), (25, 0.023), (26, -0.018), (27, 0.002), (28, -0.005), (29, -0.007), (30, -0.013), (31, 0.012), (32, -0.018), (33, 0.001), (34, 0.019), (35, -0.04), (36, 0.01), (37, 0.04), (38, -0.008), (39, 0.001), (40, -0.026), (41, 0.01), (42, 0.036), (43, 0.014), (44, 0.019), (45, 0.03), (46, -0.007), (47, 0.061), (48, -0.008), (49, 0.012)]
simIndex simValue blogId blogTitle
same-blog 1 0.95110577 1451 high scalability-2013-05-03-Stuff The Internet Says On Scalability For May 3, 2013
Introduction: Hey, it's HighScalability time: ( Giant Hurricane on Saturn , here's one in New Orleans ) 1,966,080 cores : Time Warp synchronization protocol using up to 7.8M MPI tasks on 1,966,080 cores of the {Sequoia} Blue Gene/Q supercomputer system. 33 trillion events processed in 65 seconds yielding a peak event-rate in excess of 504 billion events/second using 120 racks of Sequoia. Quotable Quotes: Thad Starner : the longer accessing a device exceeds 2s, the more its actually usage would decrease exponentially. Thus, he made a claim that wrist watch interface always sitting on one's wrist ready to use should be more successful than mobile phones which have to pulled out of the pocket. @joedevon : We came for scalability but we stayed for agility #NoSQL @jahmailay : "Our user base is exploding. I really wish we spent more time on scalability instead of features customers don't use." - Everybody, always. @bsletten : I don’t think it is a
2 0.86763752 1339 high scalability-2012-10-12-Stuff The Internet Says On Scalability For October 12, 2012
Introduction: It's HighScalability Time: Quotable Quotes: @rodos : With so much manure in the room of #BigData the must be a pony in here somewhere! @adron : #ricon2012 OH "it isn't cloud, nosql or whatever, it's distributed systems... That is the change!" Smartest thing stated at a conf in ages. @justinesherry : First lesson watching @eric_brewer's keynote at #ricon2012 -- distributed systems make you go bald. (Both audience and speaker!) @adron : OH "What's the hello world of distributed systems?" "Twitter." Mikael Ronstrom : A long career on distributed systems has learnt me that it is extremely important to do proper partitioning of data sets to achieve scalability on network level. But it is still extremely important to make each node in the distributed system as large as possible. Greg Linden : Surprising: "It can be three to four times the cost for the retailer to pick items and pack them from a store versus having a really e
3 0.84529865 1572 high scalability-2014-01-03-Stuff The Internet Says On Scalability For January 3rd, 2014
Introduction: Hey, it's HighScalability time, can you handle the truth? Should software architectures include parasites ? They increase diversity and complexity in the food web. 10 Million : classic hockey stick growth pattern for GitHub repositories Quotable Quotes: Seymour Cray : A supercomputer is a device for turning compute-bound problems into IO-bound problems. Robert Sapolsky : And why is self-organization so beautiful to my atheistic self? Because if complex, adaptive systems don’t require a blue print, they don’t require a blue print maker. If they don’t require lightning bolts, they don’t require Someone hurtling lightning bolts. @swardley : Asked for a history of PaaS? From memory, public launch - Zimki ('06), BungeeLabs ('06), Heroku ('07), GAE ('08), CloudFoundry ('11) ... @neil_conway : If you're designing scalable systems, you should understand backpressure and build mechanisms to support it. Scott Aaronson ...the
4 0.83928919 1247 high scalability-2012-05-18-Stuff The Internet Says On Scalability For May 18, 2012
Introduction: It's HighScalability Time: 42 Billion : Netflix API Requests /Month Quotable quotes: @commonlisp : Ideas from the talk: In Haskell laziness + thunks + garbage collection (GC) impede multicore scalability. Parallel GC is crucial. @Bulldozer0 : Global state is the enemy of scalability; not only in software, but in governance. If you've ever, as I have, felt the terror of a misplaced "rm -rf /", you'll love this story: Did Pixar accidentally delete Toy Story 2 during production? It did, the reconstruction was heroic, and parts of Toy Story 2 were lost forever. For YouPorn - Targeting 200 Million Views A Day And Beyond we didn't have the source video, but that's been fixed. Here's the video of the talk . Nice job. The DevOps Reading List: 10 Books & Blogs You Should Be Reading . Excellent list of resources with thoughtful descriptions. 10 essential performance tips for MySQL . Baron Schwartz with great advice that applies to mor
5 0.83764124 1523 high scalability-2013-09-27-Stuff The Internet Says On Scalability For September 27, 2013
Introduction: Hey, it's HighScalability time: ( The WINLAB at Rutgers, with software defined radios tied into GENI. ) 384 cores & 32TB of RAM : Oracle's SPARC M6 Quotable Quotes: @jennyinc : 2003: "I replaced you with a set of very small shell scripts." 2013: "I replaced your scripts with a six-figure enterprise DevOps platform." @tomdale : OH: “Redis is so fast, why don’t we replace RAM with Redis?” @petrillic : OH "Promises/futures are the one-night stands of architectural constructs" nice #strangeloop @TwitterEng : "Java and Scala let Twitter readily share and modify its enormous codebase across a team of hundreds of developers." Lots of juicy numbers revealed at Structure:Europe : Netflix streams 114,000 years of video every month; Custom build Netflix boxes for its content-delivery network that contain between 100 and 150 terabytes of stor
6 0.83403492 1455 high scalability-2013-05-10-Stuff The Internet Says On Scalability For May 10, 2013
7 0.83028644 1506 high scalability-2013-08-23-Stuff The Internet Says On Scalability For August 23, 2013
8 0.82877278 1603 high scalability-2014-02-28-Stuff The Internet Says On Scalability For February 28th, 2014
9 0.82706857 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
10 0.82425255 1071 high scalability-2011-07-01-Stuff The Internet Says On Scalability For July 1, 2011
11 0.82272786 1621 high scalability-2014-03-28-Stuff The Internet Says On Scalability For March 28th, 2014
12 0.82186276 1255 high scalability-2012-06-01-Stuff The Internet Says On Scalability For June 1, 2012
13 0.82162905 1330 high scalability-2012-09-28-Stuff The Internet Says On Scalability For September 28, 2012
14 0.81945544 1637 high scalability-2014-04-25-Stuff The Internet Says On Scalability For April 25th, 2014
15 0.81868684 1468 high scalability-2013-05-31-Stuff The Internet Says On Scalability For May 31, 2013
16 0.81512642 826 high scalability-2010-05-12-The Rise of the Virtual Cellular Machines
17 0.81461483 1509 high scalability-2013-08-30-Stuff The Internet Says On Scalability For August 30, 2013
18 0.81430662 1490 high scalability-2013-07-12-Stuff The Internet Says On Scalability For July 12, 2013
19 0.81424487 1460 high scalability-2013-05-17-Stuff The Internet Says On Scalability For May 17, 2013
20 0.81331199 1537 high scalability-2013-10-25-Stuff The Internet Says On Scalability For October 25th, 2013
topicId topicWeight
[(1, 0.11), (2, 0.163), (3, 0.02), (5, 0.012), (10, 0.048), (30, 0.029), (39, 0.012), (40, 0.026), (47, 0.012), (51, 0.013), (52, 0.142), (56, 0.017), (61, 0.095), (77, 0.036), (79, 0.104), (85, 0.047), (94, 0.038)]
simIndex simValue blogId blogTitle
1 0.93199092 47 high scalability-2007-07-30-Product: Yslow to speed up your web pages
Introduction: Update : Speed up Apache - how I went from F to A in YSlow . Good example of using YSlow to speed up a website with solid code examples. Every layer in the multi-layer cake that is your website contributes to how long a page takes to display. YSlow , from Yahoo, is a cool tool for discovering how the ingredients of your site's top layer contribute to performance. YSlow analyzes web pages and tells you why they're slow based on the rules for high performance web sites. YSlow is a Firefox add-on integrated with the popular Firebug web development tool. YSlow gives you: Performance report card HTTP/HTML summary List of components in the page Tools including JSLint
2 0.93125147 872 high scalability-2010-08-05-Pairing NoSQL and Relational Data Storage: MySQL with MongoDB
Introduction: I’ve largely steered clear of publicly commenting on the “NoSQL vs. Relational” conflict. Keeping in mind that this argument is more about currently available solutions and the features their developers have chosen to build in, I’d like to dig into this and provide a decidedly neutral viewpoint. In fact, by erring on the side of caution, I’ve inadvertently given myself plenty of time to consider the pros and cons of both data storage approaches, and although my mind was initially swaying toward the NoSQL camp, I can say with a fair amount of certainty, that I’ve found a good compromise. You can read the full store here .
Introduction: Scoble the Ubiquitous has a fascinating post on how Mogulus, a live video channel startup, uses S3/EC2 and doesn't own a single server. The trends that have been happening for a while now are going mainstream. To do great things you no longer need to start by creating a huge war chest. You can forage off the land, like any good mobile, light weight fighting unit. For a strategy hit he mentions the same needed change in perspective as Beau Lebens talked about when making FeedBlendr : One tip he gave us is that when using Amazon’s services you have to design your systems with the assumption that they will never be up and running. What he means by that is services are “volatile” and can go up and down without notice. So, he’s designed his systems to survive that. He told me that it meant his engineering teams had to be quite disciplined in designing their architecture.
Introduction: This is a guest post by Ivan de Prado and Pere Ferrera , founders of Datasalt , the company behind Pangool and Splout SQL Big Data open-source projects. The amount of payments performed using credit cards is huge. It is clear that there is inherent value in the data that can be derived from analyzing all the transactions. Client fidelity, demographics, heat maps of activity, shop recommendations, and many other statistics are useful to both clients and shops for improving their relationship with the market. At Datasalt we have developed a system in collaboration with the BBVA bank that is able to analyze years of data and serve insights and statistics to different low-latency web and mobile applications. The main challenge we faced besides processing Big Data input is that the output was also Big Data, and even bigger than the input . And this output needed to be served quickly, under high load. The solution we developed has an infrastructure cost of just a few tho
Introduction: Tech Crunch chomped down on some initial scaling problems with Yahoo's new live video streaming service Yahoo Live . After a bit of chewing on Yahoo's old bones, TC spat out: If Yahoo cant scale something like this (no matter how much they claim it’s an experiment, it’s still a live service), it shows how far the once brightest star of the online world has fallen . This kind of thinking kills innovation. When there's no room for a few hiccups or a little failure you have to cover your ass so completely nothing new will ever see the light of day. I thought we were supposed to be agile . We are supposed to release early and often. Not every 'i' has to be dotted and not every last router has to be installed before we take the first step of a grand new journey. Get it out there. Let users help you make it better. Listen to customers, make changes, push the new code out, listen some more, and fix problems as they come up. Following this process we'll make somethi
same-blog 6 0.89605993 1451 high scalability-2013-05-03-Stuff The Internet Says On Scalability For May 3, 2013
7 0.887541 1117 high scalability-2011-09-16-Stuff The Internet Says On Scalability For September 16, 2011
8 0.87883723 1432 high scalability-2013-04-01-Khan Academy Checkbook Scaling to 6 Million Users a Month on GAE
9 0.87822497 882 high scalability-2010-08-18-Misco: A MapReduce Framework for Mobile Systems - Start of the Ambient Cloud?
10 0.87698621 1573 high scalability-2014-01-06-How HipChat Stores and Indexes Billions of Messages Using ElasticSearch and Redis
11 0.87239331 1406 high scalability-2013-02-14-When all the Program's a Graph - Prismatic's Plumbing Library
13 0.86455375 1072 high scalability-2011-07-01-TripAdvisor Strategy: No Architects, Engineers Work Across the Entire Stack
14 0.85815221 429 high scalability-2008-10-25-Product: Puppet the Automated Administration System
15 0.85746878 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
16 0.85416347 851 high scalability-2010-07-02-Hot Scalability Links for July 2, 2010
17 0.85364276 1148 high scalability-2011-11-29-DataSift Architecture: Realtime Datamining at 120,000 Tweets Per Second
18 0.8535471 1109 high scalability-2011-09-02-Stuff The Internet Says On Scalability For September 2, 2011
19 0.85306817 1302 high scalability-2012-08-10-Stuff The Internet Says On Scalability For August 10, 2012
20 0.85242122 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014