high_scalability high_scalability-2014 high_scalability-2014-1612 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Hey, it's HighScalability time: LifeExplorer Cells in 3D Quotable Quotes: The Master Switch : History shows a typical progression of information technologies: from somebody’s hobby to somebody’s industry; from jury-rigged contraption to slick production marvel; from a freely accessible channel to one strictly controlled by a single corporation or cartel—from open to closed system. @adrianco : #qconlondon @russmiles on PaaS "As old as I am, a leaky abstraction would be awful..." @Obdurodon : "Scaling is hard. Let's make excuses." @TomRoyce : @jeffjarvis the rot is deep... The New Jersey pols just used Tesla to shake down the car dealers. @CompSciFact : "The cheapest, fastest and most reliable components of a computer system are those that aren't there." -- Gordon Bell @glyph : “Eventually consistent” is just another way to say “not consistent right now”. @nutshell : LinkedIn is shutting down access to their APIs
sentIndex sentText sentNum sentScore
1 @muratdemirbas : "Simple clear purpose&principles; give rise to complex&intelligent; behavior. [sent-16, score-0.083]
2 Even Facebook's infrastructure struggles when a new feature becomes an unexpected hit. [sent-30, score-0.08]
3 That's the situation described in an engaging story: Looking back on “Look Back” videos . [sent-31, score-0.22]
4 Look Back's are one minute videos generated from a user's pics and posts. [sent-32, score-0.134]
5 For the release they planned on 187 Gbps more bandwidth and 25 petabytes of disk. [sent-33, score-0.122]
6 Less storage was actually needed because the video could be regenerated so a high replication factor wasn't needed. [sent-37, score-0.074]
7 The videos were an unexpected hit with a 40% reshare instead of the projected 10% reshare. [sent-39, score-0.148]
8 For example in the figure, the Blue algorithm was better with a million training dataset. [sent-51, score-0.093]
9 If one had stopped at that scale, one would think of optimizing that algorithm for better performance. [sent-52, score-0.093]
10 But as the scale increased the purple algorithm started showing promise – in fact the blue one starts deteriorating at larger scale; In general, Google prefers algorithms that get better with data. [sent-53, score-0.335]
11 Not all algorithms are like that, but Google likes to after the ones with this type of performance characteristic. [sent-54, score-0.076]
12 People don't want an artist to have multiple songs within a certain window and math doesn't care. [sent-60, score-0.12]
13 Reduce bandwidth by reducing the number of players in a game; reducing update frequency which also increases latency; improving the serialization isn't usually a big win; compression; patch based updates; replicate fewer objects using rules based, static, and geometric strategies. [sent-72, score-0.122]
14 If you can fit all your data inside a single UDP packet then you've done a lot to win the latency and bandwidth game. [sent-77, score-0.299]
15 In Oodle Packet Compression for UDP you learn how to create a static model compression where "you pre-train some model based on a capture of typical network data. [sent-78, score-0.112]
16 The Truth About MapReduce Performance on SSDs : SSDs have up to 70 percent higher performance, for 2. [sent-83, score-0.142]
17 Customers can consider paying a premium cost to obtain up to 70 percent higher performance. [sent-86, score-0.217]
18 Network operators commonly discuss on mailing lists how the big four access shops all maintain edges which are boiling hot unless you pay them, or buy from an intermediary paying them. [sent-94, score-0.241]
19 However, we emphasize the process of embedding secret information into an innocent-looking carrier is not some recent invention—it has been known and used for ages by humankind. [sent-97, score-0.253]
20 This process is called steganography and its origins can be traced back to ancient times. [sent-98, score-0.246]
wordName wordTfidf (topN-words)
[('hbo', 0.163), ('steganography', 0.163), ('peering', 0.157), ('bandwidth', 0.122), ('songs', 0.12), ('carrier', 0.117), ('compression', 0.112), ('packet', 0.108), ('udp', 0.101), ('history', 0.094), ('edges', 0.094), ('somebody', 0.093), ('algorithm', 0.093), ('blue', 0.092), ('rise', 0.083), ('back', 0.083), ('unexpected', 0.08), ('google', 0.078), ('ssds', 0.076), ('algorithms', 0.076), ('percent', 0.075), ('paying', 0.075), ('regenerated', 0.074), ('glimpse', 0.074), ('marvel', 0.074), ('glyph', 0.074), ('deteriorating', 0.074), ('muratdemirbas', 0.074), ('cooperated', 0.074), ('acton', 0.074), ('koum', 0.074), ('enjoy', 0.073), ('unless', 0.072), ('providers', 0.07), ('computer', 0.07), ('imaginable', 0.069), ('nutshell', 0.069), ('qconlondon', 0.069), ('emphasize', 0.069), ('infinity', 0.069), ('leaky', 0.069), ('congested', 0.069), ('win', 0.069), ('described', 0.069), ('videos', 0.068), ('secret', 0.067), ('higher', 0.067), ('internap', 0.066), ('hobby', 0.066), ('pics', 0.066)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1612 high scalability-2014-03-14-Stuff The Internet Says On Scalability For March 14th, 2014
Introduction: Hey, it's HighScalability time: LifeExplorer Cells in 3D Quotable Quotes: The Master Switch : History shows a typical progression of information technologies: from somebody’s hobby to somebody’s industry; from jury-rigged contraption to slick production marvel; from a freely accessible channel to one strictly controlled by a single corporation or cartel—from open to closed system. @adrianco : #qconlondon @russmiles on PaaS "As old as I am, a leaky abstraction would be awful..." @Obdurodon : "Scaling is hard. Let's make excuses." @TomRoyce : @jeffjarvis the rot is deep... The New Jersey pols just used Tesla to shake down the car dealers. @CompSciFact : "The cheapest, fastest and most reliable components of a computer system are those that aren't there." -- Gordon Bell @glyph : “Eventually consistent” is just another way to say “not consistent right now”. @nutshell : LinkedIn is shutting down access to their APIs
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
4 0.1477555 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
Introduction: This is one of my favorite posts for a couple of reasons. I think it gives a lot of useful information in an interesting space. And Kyle Vogt was just a real pleasure to talk to. He was very helpful and forthcoming, which makes the whole experience better for everyone. The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do tha
5 0.14770411 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
Introduction: The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do that? To find out I talked to Kyle Vogt, Justin.tv Founder and VP of Engineering. Justin.tv certainly has the numbers. Their 30 million unique monthly visitors even outshine YouTube in the video upload game, reportedly uploading nearly 30 hours per minute of video compared to Y
6 0.13336754 904 high scalability-2010-09-21-Playfish's Social Gaming Architecture - 50 Million Monthly Users and Growing
7 0.1333513 661 high scalability-2009-07-25-Latency is Everywhere and it Costs You Sales - How to Crush it
8 0.13329028 761 high scalability-2010-01-17-Applications Become Black Boxes Using Markets to Scale and Control Costs
9 0.12864015 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
10 0.12491204 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
11 0.12313282 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
12 0.12083113 1389 high scalability-2013-01-18-Stuff The Internet Says On Scalability For January 18, 2013
13 0.1186562 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
14 0.11810528 517 high scalability-2009-02-21-Google AppEngine - A Second Look
16 0.11725 1564 high scalability-2013-12-13-Stuff The Internet Says On Scalability For December 13th, 2013
17 0.11674192 1291 high scalability-2012-07-25-Vertical Scaling Ascendant - How are SSDs Changing Architectures?
18 0.11627832 768 high scalability-2010-02-01-What Will Kill the Cloud?
19 0.11619286 1552 high scalability-2013-11-22-Stuff The Internet Says On Scalability For November 22th, 2013
20 0.11596789 1634 high scalability-2014-04-18-Stuff The Internet Says On Scalability For April 18th, 2014
topicId topicWeight
[(0, 0.257), (1, 0.126), (2, 0.04), (3, 0.049), (4, 0.006), (5, -0.056), (6, -0.004), (7, 0.115), (8, 0.009), (9, 0.002), (10, 0.007), (11, -0.065), (12, -0.012), (13, 0.021), (14, 0.034), (15, 0.049), (16, -0.013), (17, 0.035), (18, 0.002), (19, -0.032), (20, 0.012), (21, 0.008), (22, 0.031), (23, -0.021), (24, -0.017), (25, 0.054), (26, -0.001), (27, -0.026), (28, 0.026), (29, -0.001), (30, -0.018), (31, -0.01), (32, -0.015), (33, 0.02), (34, -0.04), (35, 0.001), (36, 0.043), (37, -0.018), (38, 0.023), (39, -0.027), (40, -0.008), (41, 0.018), (42, -0.015), (43, -0.002), (44, 0.011), (45, 0.019), (46, 0.008), (47, -0.017), (48, -0.027), (49, -0.017)]
simIndex simValue blogId blogTitle
same-blog 1 0.96905744 1612 high scalability-2014-03-14-Stuff The Internet Says On Scalability For March 14th, 2014
Introduction: Hey, it's HighScalability time: LifeExplorer Cells in 3D Quotable Quotes: The Master Switch : History shows a typical progression of information technologies: from somebody’s hobby to somebody’s industry; from jury-rigged contraption to slick production marvel; from a freely accessible channel to one strictly controlled by a single corporation or cartel—from open to closed system. @adrianco : #qconlondon @russmiles on PaaS "As old as I am, a leaky abstraction would be awful..." @Obdurodon : "Scaling is hard. Let's make excuses." @TomRoyce : @jeffjarvis the rot is deep... The New Jersey pols just used Tesla to shake down the car dealers. @CompSciFact : "The cheapest, fastest and most reliable components of a computer system are those that aren't there." -- Gordon Bell @glyph : “Eventually consistent” is just another way to say “not consistent right now”. @nutshell : LinkedIn is shutting down access to their APIs
2 0.88323784 1592 high scalability-2014-02-07-Stuff The Internet Says On Scalability For February 7th, 2014
Introduction: Hey, it's HighScalability time: Google "Corkboard" Server, 1999 5 billion requests per day : Heroku serves 60,000 requests per second; 500 Petabytes : Backblaze's New Data Center; 25000 simultaneous connections : on a Percona Server How algorithms help determine that shape of our world. First we encode normative rules of an idealized world in algorithms. Second those algorithms help enforce those expectations by nudging humans in to acting accordingly. A fun example is the story of Ed Bolian's Record-Breaking Drive . Ed raced from New York to L.A. at speeds of up to 158 mph, "breaking countless laws – and the previous record, by more than two hours." His approach is one that any nerd would love. He had three radar detectors, two laser jammers, two nav systems, a CB radio, a scanner, two iPhones and two iPads running applications like Waze, lookouts in the back seat scanning for cops, and someone scouting ahead. Awesome! For the moral of the story, they were going s
3 0.87134695 1603 high scalability-2014-02-28-Stuff The Internet Says On Scalability For February 28th, 2014
Introduction: Hey, it's HighScalability time: Plus ça change, plus c'est la même chose ( full ) Quotable Quotes: @ML_Hipster : A machine learning researcher, a crypto-currency expert, and an Erlang programmer walk into a bar. Facebook buys the bar for $27 billion. OH: Network effects don't happen on toll roads. Benedict Evans : Google is a vast machine learning engine... and it spent 10-15 years building that learning engine and feeding it data. Mining Experiment: Running 600 Servers for a Year Yields 0.4 Bitcoin . Yes, this is a far superior way of doing things. Chew up the commons for marginal gain. It's like old times. Game designers, forget the sardines and go hunt some whale . Swrve found : half of free-to-play games’ in-app purchases came from 0.15 percent of players. Only 1.5 percent of players of games in the Swrve network spent any money at all. Google has a beta version of their cloud pricing calculator . The interface is a little fun
4 0.85788113 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
Introduction: Hey, it's HighScalability time (a particularly bountiful week): The Telephone Wires of Manhattan in 1887 ( full ) $19 billion: you know what it is; $46 billion : cost of Sochi Olympics; 400 gigabytes : data transmitted during the Sochi opening ceremony; 26.9 million : Stack Overflow community monthly visitors; 93 million : Candy Crush daily active users; 200-400 Gbps : The New Normal in DDoS Attacks Quotable Quotes: @brianacton : Facebook turned me down. It was a great opportunity to connect with some fantastic people. Looking forward to life's next adventure. @BenedictEvans : Flickr: $35m. Youtube: $1.65bn Whatsapp: $19bn. Mobile is big. And global. And the next computing platform. Paying attention? @taziden : On the Internet, worst cases will become common cases #fosdem #postfix Brian Hayes : Any quantum program must have a stovepipe architecture: Information flows straight through. So you think V
5 0.85402155 1460 high scalability-2013-05-17-Stuff The Internet Says On Scalability For May 17, 2013
Introduction: Hey, it's HighScalability time: ( Earth sized solar flare , some more flair ) Google I/O to world: Just try to keep up with us. You can't. But go ahead and try. Nah na na na nah... 17 billion : Google Cloud Messaging messages per day with 60ms latency; 1B page views : 500px; 121 billion : edge graph using Titan; 4 billion hours : hours watched on Netflix per quarter; 4.5 trillion : BigTable transactions per month Quotable Quotes: to3m : As with any time you make plans for the future, sometimes you get it wrong. Ars longa vita brevis, and all that. Callaghan’s law : a given row can’t be modified more than once per RTT Josh Haberman : I had an epiphany one day when I realized that the kernel is nothing but a library with an expensive calling convention. fread2281 : Insane speed calls for insane measures. Luke Gorrie : hardware really wants to run fast and you only need to avoid getting in the way -- not too hard if you writ
6 0.8491168 1607 high scalability-2014-03-07-Stuff The Internet Says On Scalability For March 7th, 2014
7 0.8335278 1464 high scalability-2013-05-24-Stuff The Internet Says On Scalability For May 24, 2013
8 0.83223653 1630 high scalability-2014-04-11-Stuff The Internet Says On Scalability For April 11th, 2014
9 0.8297466 1506 high scalability-2013-08-23-Stuff The Internet Says On Scalability For August 23, 2013
10 0.82799721 1572 high scalability-2014-01-03-Stuff The Internet Says On Scalability For January 3rd, 2014
11 0.82596081 1637 high scalability-2014-04-25-Stuff The Internet Says On Scalability For April 25th, 2014
12 0.82467973 1476 high scalability-2013-06-14-Stuff The Internet Says On Scalability For June 14, 2013
13 0.8245914 1490 high scalability-2013-07-12-Stuff The Internet Says On Scalability For July 12, 2013
14 0.82405251 1576 high scalability-2014-01-10-Stuff The Internet Says On Scalability For January 10th, 2014
15 0.82029796 1091 high scalability-2011-08-02-How Will DIDO Wireless Networking Change Everything?
16 0.82018828 1472 high scalability-2013-06-07-Stuff The Internet Says On Scalability For June 7, 2013
17 0.81702811 1552 high scalability-2013-11-22-Stuff The Internet Says On Scalability For November 22th, 2013
18 0.81598628 1626 high scalability-2014-04-04-Stuff The Internet Says On Scalability For April 4th, 2014
19 0.81434679 1499 high scalability-2013-08-09-Stuff The Internet Says On Scalability For August 9, 2013
20 0.81097573 1214 high scalability-2012-03-23-Stuff The Internet Says On Scalability For March 23, 2012
topicId topicWeight
[(1, 0.099), (2, 0.182), (10, 0.057), (17, 0.022), (25, 0.011), (26, 0.017), (30, 0.036), (40, 0.035), (48, 0.074), (61, 0.057), (77, 0.024), (79, 0.151), (85, 0.056), (93, 0.013), (94, 0.075)]
simIndex simValue blogId blogTitle
same-blog 1 0.95773882 1612 high scalability-2014-03-14-Stuff The Internet Says On Scalability For March 14th, 2014
Introduction: Hey, it's HighScalability time: LifeExplorer Cells in 3D Quotable Quotes: The Master Switch : History shows a typical progression of information technologies: from somebody’s hobby to somebody’s industry; from jury-rigged contraption to slick production marvel; from a freely accessible channel to one strictly controlled by a single corporation or cartel—from open to closed system. @adrianco : #qconlondon @russmiles on PaaS "As old as I am, a leaky abstraction would be awful..." @Obdurodon : "Scaling is hard. Let's make excuses." @TomRoyce : @jeffjarvis the rot is deep... The New Jersey pols just used Tesla to shake down the car dealers. @CompSciFact : "The cheapest, fastest and most reliable components of a computer system are those that aren't there." -- Gordon Bell @glyph : “Eventually consistent” is just another way to say “not consistent right now”. @nutshell : LinkedIn is shutting down access to their APIs
2 0.93806291 289 high scalability-2008-03-27-Amazon Announces Static IP Addresses and Multiple Datacenter Operation
Introduction: Amazon is fixing two of their major problems: no static IP addresses and single datacenter operation. By adding these two new features developers can finally build a no apology system on Amazon. Before you always had to throw in an apology or two. No, we don't have low failover times because of the silly DNS games and unexceptionable DNS update and propagation times and no, we don't operate in more than one datacenter. No more. Now Amazon is adding Elastic IP Addresses and Availability Zones . Elastic IP addresses are far better than normal IP addresses because they are both in tight with Jessica Alba and they are: Static IP addresses designed for dynamic cloud computing. An Elastic IP address is associated with your account, not a particular instance, and you control that address until you choose to explicitly release it. Unlike traditional static IP addresses, however, Elastic IP addresses allow you to mask instance or availability zone failures by programmatica
3 0.93654221 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
Introduction: One consequence of IT standardization and commodification has been Google’s datacenter is the computer view of the world. In that view all compute resources (memory, CPU, storage) are fungible. They are interchangeable and location independent, individual computers lose identity and become just a part of a service. Thwarting that nirvana has been the abysmal performance of commodity datacenter networks which have caused the preference of architectures that favor the collocation of state and behaviour on the same box. MapReduce famously ships code over to storage nodes for just this reason. Change the network and you change the fundamental assumption driving collocation based software architectures. You are then free to store data anywhere and move compute anywhere you wish. The datacenter becomes the computer. On the host side with an x8 slot running at PCI-Express 3.0 speeds able to push 8GB/sec (that’s bytes) of bandwidth in both directions, we have
Introduction: “ Data is everywhere, never be at a single location. Not scalable, not maintainable. ” –Alex Szalay While Galileo played life and death doctrinal games over the mysteries revealed by the telescope, another revolution went unnoticed, the microscope gave up mystery after mystery and nobody yet understood how subversive would be what it revealed. For the first time these new tools of perceptual augmentation allowed humans to peek behind the veil of appearance. A new new eye driving human invention and discovery for hundreds of years. Data is another material that hides, revealing itself only when we look at different scales and investigate its underlying patterns. If the universe is truly made of information , then we are looking into truly primal stuff. A new eye is needed for Data and an ambitious project called Data-scope aims to be the lens. A detailed paper on the Data-Scope tells more about what it is: The Data-Scope is a new scientific instrum
Introduction: This is a guest post by Ivan de Prado and Pere Ferrera , founders of Datasalt , the company behind Pangool and Splout SQL Big Data open-source projects. The amount of payments performed using credit cards is huge. It is clear that there is inherent value in the data that can be derived from analyzing all the transactions. Client fidelity, demographics, heat maps of activity, shop recommendations, and many other statistics are useful to both clients and shops for improving their relationship with the market. At Datasalt we have developed a system in collaboration with the BBVA bank that is able to analyze years of data and serve insights and statistics to different low-latency web and mobile applications. The main challenge we faced besides processing Big Data input is that the output was also Big Data, and even bigger than the input . And this output needed to be served quickly, under high load. The solution we developed has an infrastructure cost of just a few tho
6 0.92846352 716 high scalability-2009-10-06-Building a Unique Data Warehouse
7 0.92837048 1389 high scalability-2013-01-18-Stuff The Internet Says On Scalability For January 18, 2013
8 0.92677206 1545 high scalability-2013-11-08-Stuff The Internet Says On Scalability For November 8th, 2013
9 0.92614448 789 high scalability-2010-03-05-Strategy: Planning for a Power Outage Google Style
10 0.92600793 517 high scalability-2009-02-21-Google AppEngine - A Second Look
12 0.92580163 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
13 0.92535084 1647 high scalability-2014-05-14-Google Says Cloud Prices Will Follow Moore’s Law: Are We All Renters Now?
14 0.92509037 1112 high scalability-2011-09-07-What Google App Engine Price Changes Say About the Future of Web Architecture
15 0.92467749 873 high scalability-2010-08-06-Hot Scalability Links for Aug 6, 2010
16 0.92460191 1630 high scalability-2014-04-11-Stuff The Internet Says On Scalability For April 11th, 2014
17 0.92412782 1649 high scalability-2014-05-16-Stuff The Internet Says On Scalability For May 16th, 2014
18 0.92331523 1460 high scalability-2013-05-17-Stuff The Internet Says On Scalability For May 17, 2013
19 0.92285234 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
20 0.92175996 890 high scalability-2010-09-01-Paper: The Case for Determinism in Database Systems