high_scalability high_scalability-2013 high_scalability-2013-1392 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Ivan Pepelnjak, in his short and information packed REDUNDANT DATA CENTER INTERNET CONNECTIVIT Y video, shows why networking as played at the highest levels is something you want to leave to professionals, like a large animal country vetenarian delivering a stuck foal at 2AM on a dark and stormy night. There are always a lot questions about the black art of building redundant datacenter networks and there's a shortage of accessible explanations. What I liked about Ivan's video is how effortlessly he explains the issues and tradeoffs you can expect in designing your own solution, as well as giving creative solutions to those problems. A lot of years of experience are boiled down to a 17 minute video. Ivan begins by showing what a canonical fully redundant datacenter would look like: It's like an ark where everything goes two by two. You have two datacenters, each datacenter has redundant core switches, redundant servers, redundant disk arrays, redundant links between d
sentIndex sentText sentNum sentScore
1 There are always a lot questions about the black art of building redundant datacenter networks and there's a shortage of accessible explanations. [sent-2, score-0.783]
2 Ivan begins by showing what a canonical fully redundant datacenter would look like: It's like an ark where everything goes two by two. [sent-5, score-0.848]
3 Everything is redundant so what could possibly go wrong? [sent-9, score-0.402]
4 Faults in two sensitive areas and your expensive and complicated redundant datacenter setup is reduced to one lone datacenter operating on it's own. [sent-11, score-1.216]
5 The reason is you quickly learn that creating a system that can handle double faults is impossibly complex and expensive. [sent-13, score-0.214]
6 When you do handle double faults the logic becomes so complex it tranforms into a bigger fault domain than the original problem. [sent-14, score-0.306]
7 Ivan proposes a plausible double fault scenario where you lose a router and a link so that one datacenter is completely isolated from the Internet. [sent-15, score-0.851]
8 Maybe the datacenter can operate independently and resync at some later time? [sent-21, score-0.381]
9 Maybe one datacenter needs to die until the connections are back up? [sent-22, score-0.459]
10 Maybe some servers need to migrate from one datacenter to the other? [sent-23, score-0.494]
11 On a link failure, if your Internet links are 10G and your datacenter links are 10G, then they might be able to substitute for one another temporarily. [sent-25, score-0.823]
12 Maybe you want to migrate servers or take one datacenter out of operation. [sent-27, score-0.494]
13 If the DCI links are congested and you want to migrate servers, live migration will just make the congestion problem worse. [sent-28, score-0.404]
14 If you don't think the connections will come back up then it may make sense to shutdown a datacenter and have a temporary loss of service rather than migrate servers over an already congested link. [sent-29, score-0.732]
15 If the Internet links are down for one datacenter it makes sense to think of routing user traffic to that datacenter through the other datacenter and through the DCI links. [sent-30, score-1.531]
16 One problem is the firewalls in the different datacenters are stateful, so they won't have the session state for the other datacenter. [sent-32, score-0.318]
17 Ivan says creating stretched VLANs to link the firewalls into a cluster is a bad idea . [sent-33, score-0.33]
18 Another problem is the remaining firewall will now have to perform NAT translations for both firewalls, which means you need redundant configurations. [sent-34, score-0.546]
19 Ivan proposes creating an external WAN backbone that can be used to exchange data between the four edge routers. [sent-41, score-0.308]
20 Following it gave me a headache, but made sense as he was explaining it :-) Well worth watching in detail if you are wrestling with the problem building datacenter networks. [sent-45, score-0.555]
wordName wordTfidf (topN-words)
[('redundant', 0.402), ('datacenter', 0.381), ('ivan', 0.334), ('links', 0.186), ('dcis', 0.185), ('datacenters', 0.167), ('internet', 0.154), ('firewalls', 0.151), ('routers', 0.133), ('connectivity', 0.131), ('nat', 0.125), ('migrate', 0.113), ('scenario', 0.112), ('dci', 0.112), ('double', 0.112), ('stateful', 0.106), ('congested', 0.105), ('faults', 0.102), ('maybe', 0.1), ('edge', 0.097), ('arrives', 0.094), ('fault', 0.092), ('firewall', 0.091), ('proposes', 0.084), ('traffic', 0.082), ('isps', 0.082), ('connections', 0.078), ('wan', 0.071), ('backbone', 0.071), ('link', 0.07), ('goes', 0.065), ('routing', 0.065), ('explaining', 0.063), ('says', 0.056), ('uplinks', 0.056), ('wrestling', 0.056), ('scenarios', 0.056), ('exchange', 0.056), ('sense', 0.055), ('video', 0.053), ('resync', 0.053), ('animal', 0.053), ('stretched', 0.053), ('stormy', 0.053), ('translations', 0.053), ('alarge', 0.053), ('reduced', 0.052), ('explains', 0.052), ('center', 0.051), ('prefix', 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000002 1392 high scalability-2013-01-23-Building Redundant Datacenter Networks is Not For Sissies - Use an Outside WAN Backbone
Introduction: Ivan Pepelnjak, in his short and information packed REDUNDANT DATA CENTER INTERNET CONNECTIVIT Y video, shows why networking as played at the highest levels is something you want to leave to professionals, like a large animal country vetenarian delivering a stuck foal at 2AM on a dark and stormy night. There are always a lot questions about the black art of building redundant datacenter networks and there's a shortage of accessible explanations. What I liked about Ivan's video is how effortlessly he explains the issues and tradeoffs you can expect in designing your own solution, as well as giving creative solutions to those problems. A lot of years of experience are boiled down to a 17 minute video. Ivan begins by showing what a canonical fully redundant datacenter would look like: It's like an ark where everything goes two by two. You have two datacenters, each datacenter has redundant core switches, redundant servers, redundant disk arrays, redundant links between d
2 0.17728496 687 high scalability-2009-08-24-How Google Serves Data from Multiple Datacenters
Introduction: Update: Streamy Explains CAP and HBase's Approach to CAP . We plan to employ inter-cluster replication, with each cluster located in a single DC. Remote replication will introduce some eventual consistency into the system, but each cluster will continue to be strongly consistent. Ryan Barrett, Google App Engine datastore lead, gave this talk Transactions Across Datacenters (and Other Weekend Projects) at the Google I/O 2009 conference. While the talk doesn't necessarily break new technical ground, Ryan does an excellent job explaining and evaluating the different options you have when architecting a system to work across multiple datacenters. This is called multihoming , operating from multiple datacenters simultaneously. As multihoming is one of the most challenging tasks in all computing, Ryan's clear and thoughtful style comfortably leads you through the various options. On the trip you learn: The different multi-homing options are: Backups, Master-Slave, Multi-M
3 0.16860354 284 high scalability-2008-03-19-RAD Lab is Creating a Datacenter Operating System
Introduction: The RAD Lab (Reliable Adaptive Distributed Systems Laboratory) wants to leapfrog the Big Switch and create The Next Big Switch, skipping the cloud/utility evolutionary stage altogether. This hyper-evolutionary niche buster develops technology so advanced the cloud disperses and you can go back to building your own personal datacenters again. Where Google took years to create their datacenters, using a prefab Datacenter Operating System you might create your own in a long holiday weekend. Not St. Patrick's of course. Their vision: Enable one person to invent and run the next revolutionary IT service, operationally expressing a new business idea as a multi-million-user service over the course of a long weekend. By doing so we hope to enable an Internet "Fortune 1 million". How? By wizardry in the form of a “datacenter operating system” created from a pinch of "statistical machine learning (SML)" and a tincture of "recent insights from networking and distributed systems." Bu
4 0.13825899 266 high scalability-2008-03-04-Manage Downtime Risk by Connecting Multiple Data Centers into a Secure Virtual LAN
Introduction: Update: VcubeV - an OpenVPN-based solution designed to build and operate a multisourced infrastructure. True high availability requires a presence in multiple data centers. The recent downtime of even a high quality operation like Amazon makes this need all the more clear. Typically only the big boys can afford the complexity of operating in two or more data centers. Cloud computing along with utility billing starts to change that equation, leveling the playing field. Even smaller outfits will be in a position to manage risk by spreading machines amongst EC2, 3tera, Slicehost, Mosso and other providers. The question then becomes: given we aren't Angels, how do we walk amongst the clouds? One fascinating answer is exquisitely explained by Dmitriy Samovskiy in his Linux Journal article titled Building a Multisourced Infrastructure Using OpenVPN . Dmitriy's idea is to create a secure UDP tunnel between different data centers over public internet links so your applicatio
5 0.13739671 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
Introduction: One consequence of IT standardization and commodification has been Google’s datacenter is the computer view of the world. In that view all compute resources (memory, CPU, storage) are fungible. They are interchangeable and location independent, individual computers lose identity and become just a part of a service. Thwarting that nirvana has been the abysmal performance of commodity datacenter networks which have caused the preference of architectures that favor the collocation of state and behaviour on the same box. MapReduce famously ships code over to storage nodes for just this reason. Change the network and you change the fundamental assumption driving collocation based software architectures. You are then free to store data anywhere and move compute anywhere you wish. The datacenter becomes the computer. On the host side with an x8 slot running at PCI-Express 3.0 speeds able to push 8GB/sec (that’s bytes) of bandwidth in both directions, we have
6 0.12752479 1140 high scalability-2011-11-10-Kill the Telcos Save the Internet - The Unsocial Network
7 0.12513877 1116 high scalability-2011-09-15-Paper: It's Time for Low Latency - Inventing the 1 Microsecond Datacenter
8 0.12432747 879 high scalability-2010-08-12-Think of Latency as a Pseudo-permanent Network Partition
9 0.12247925 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
10 0.12238779 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
11 0.11979086 759 high scalability-2010-01-11-Strategy: Don't Use Polling for Real-time Feeds
12 0.11473741 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
13 0.11246081 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
14 0.10970925 698 high scalability-2009-09-10-Building Scalable Databases: Denormalization, the NoSQL Movement and Digg
15 0.10832617 645 high scalability-2009-06-30-Hot New Trend: Linking Clouds Through Cheap IP VPNs Instead of Private Lines
16 0.10645505 289 high scalability-2008-03-27-Amazon Announces Static IP Addresses and Multiple Datacenter Operation
17 0.10603406 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
19 0.10555802 42 high scalability-2007-07-30-Product: GridLayer. Utility computing for online application
topicId topicWeight
[(0, 0.141), (1, 0.081), (2, 0.015), (3, 0.009), (4, -0.067), (5, -0.071), (6, 0.009), (7, -0.001), (8, -0.032), (9, -0.019), (10, -0.026), (11, 0.004), (12, -0.066), (13, -0.005), (14, 0.074), (15, 0.091), (16, 0.05), (17, 0.053), (18, -0.06), (19, -0.023), (20, 0.035), (21, 0.075), (22, -0.022), (23, -0.036), (24, -0.024), (25, 0.097), (26, -0.002), (27, 0.034), (28, -0.026), (29, -0.04), (30, -0.019), (31, -0.059), (32, -0.044), (33, 0.007), (34, 0.055), (35, 0.041), (36, -0.006), (37, 0.03), (38, -0.033), (39, 0.083), (40, 0.058), (41, 0.042), (42, -0.043), (43, 0.025), (44, 0.034), (45, 0.019), (46, 0.018), (47, 0.003), (48, -0.05), (49, -0.027)]
simIndex simValue blogId blogTitle
same-blog 1 0.96974581 1392 high scalability-2013-01-23-Building Redundant Datacenter Networks is Not For Sissies - Use an Outside WAN Backbone
Introduction: Ivan Pepelnjak, in his short and information packed REDUNDANT DATA CENTER INTERNET CONNECTIVIT Y video, shows why networking as played at the highest levels is something you want to leave to professionals, like a large animal country vetenarian delivering a stuck foal at 2AM on a dark and stormy night. There are always a lot questions about the black art of building redundant datacenter networks and there's a shortage of accessible explanations. What I liked about Ivan's video is how effortlessly he explains the issues and tradeoffs you can expect in designing your own solution, as well as giving creative solutions to those problems. A lot of years of experience are boiled down to a 17 minute video. Ivan begins by showing what a canonical fully redundant datacenter would look like: It's like an ark where everything goes two by two. You have two datacenters, each datacenter has redundant core switches, redundant servers, redundant disk arrays, redundant links between d
2 0.78221238 1140 high scalability-2011-11-10-Kill the Telcos Save the Internet - The Unsocial Network
Introduction: Someone is killing the Internet. Since you probably use the Internet everyday you might find this surprising. It almost sounds silly, and the reason is technical, but our crack team of networking experts has examined the patient and made the diagnosis. What did they find? Diagnostic team : the Packet Pushers gang ( Greg Ferro , Jan Zorz , Ivan Pepelnjak ) in the podcast How We Are Killing the Internet . Diagnosis : invasive tunnelation. ( tubes anyone? ) Prognosis : even Dr. House might not be able to help. Cure : go back to what the Internet was; kill the tunnels; route IPv4 and IPv6; have public addresses on everything; disrupt the telcos. This is a classic story in a strange setting--the network--but the themes are universal: centralization vs. decentralization (that's where the telcos obviously come in), good vs. evil, order vs. disorder, tyranny vs. freedom, change vs. stasis, simplicity vs. complexity. And it's all being carried out on battlefield few get
3 0.75673324 645 high scalability-2009-06-30-Hot New Trend: Linking Clouds Through Cheap IP VPNs Instead of Private Lines
Introduction: You might think major Internet companies have a latency, availability, and bandwidth advantage because they can afford expensive dedicated point-to-point private line networks between their data centers. And you would be right. It's a great advantage. Or it at least it was a great advantage. Cost is the great equalizer and companies are now scrambling for ways to cut costs. Many of the most recognizable Internet companies are moving to IP VPNs (Virtual Private Networks) as a much cheaper alternative to private lines. This is a strategy you can effectively use too. This trend has historical precedent in the data center. In the same way leading edge companies moved early to virtualize their data centers, leading edge companies are now virtualizing their networks using IP VPNs to build inexpensive private networks over a shared public network. In kindergarten we learned sharing was polite, it turns out sharing can also save a lot of money in both the data center and on the network. The
4 0.72901285 1651 high scalability-2014-05-20-It's Networking. In Space! Or How E.T. Will Phone Home.
Introduction: What will the version of the Internet that follows us to the stars look like? Yes, people are really thinking seriously about this sort of thing. Specifically the InterPlanetary Networking Special Interest Group (IPNSIG). Ansible-like faster-than-light communication it isn't. There's no magical warp drive. Nor is a network of telepaths acting as a 'verse spanning telegraph system. It's more mundane than that. And in many ways more interesting as it's sort of like the old Internet on steroids, the one that was based on on UUCP and dial-up connections, but over vastly longer distances and with much longer delays : The Interplanetary Internet (based on IPN, also called InterPlaNet) is a conceived computer network in space, consisting of a set of network nodes which can communicate with each other.[1][2] Communication would be greatly delayed by the great interplanetary distances, so the IPN needs a new set of protocols and technology that are tolerant to large delays and
5 0.68125373 1051 high scalability-2011-06-01-Why is your network so slow? Your switch should tell you.
Introduction: Who hasn't cursed their network for being slow while waiting for that annoying little hour glass of pain to release all its grains of sand? But what's really going on? Is your network really slow? PacketPushers Show 45 – Arista – EOS Network Software Architecture has a good explanation of what may be really at fault (paraphrased): Network operators get calls from application guys saying the network is slow, but the problem is usually dropped packets due to congestion. It's not usually latency, it's usually packet loss. Packet loss causes TCP to back off and retransmit, which causes applications to appear slow. Packet loss can be caused by a flakey transceiver, but the problem is usually network congestion. Somewhere on the network there's fan-in, a bottleneck develops, queues build up to a certain point, and when a queue overflows it drops packets. Often the first sign of this happening is application slowness. Queues get deeper and deeper because the network is getting more
6 0.67282277 879 high scalability-2010-08-12-Think of Latency as a Pseudo-permanent Network Partition
7 0.67224044 266 high scalability-2008-03-04-Manage Downtime Risk by Connecting Multiple Data Centers into a Secure Virtual LAN
8 0.6707685 1256 high scalability-2012-06-04-OpenFlow-SDN is Not a Silver Bullet for Network Scalability
9 0.66708118 1116 high scalability-2011-09-15-Paper: It's Time for Low Latency - Inventing the 1 Microsecond Datacenter
10 0.65026814 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
11 0.63715589 270 high scalability-2008-03-08-DNS-Record TTL on worst case scenarios
12 0.63444966 284 high scalability-2008-03-19-RAD Lab is Creating a Datacenter Operating System
13 0.63116455 371 high scalability-2008-08-24-A Scalable, Commodity Data Center Network Architecture
14 0.62880599 446 high scalability-2008-11-18-Scalability Perspectives #2: Van Jacobson – Content-Centric Networking
15 0.62100142 1584 high scalability-2014-01-22-How would you build the next Internet? Loons, Drones, Copters, Satellites, or Something Else?
16 0.61631453 267 high scalability-2008-03-05-Oprah is the Real Social Network
17 0.61522114 23 high scalability-2007-07-24-Major Websites Down: Or Why You Want to Run in Two or More Data Centers.
18 0.61349475 1091 high scalability-2011-08-02-How Will DIDO Wireless Networking Change Everything?
19 0.61123264 1338 high scalability-2012-10-11-RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store
20 0.6100229 289 high scalability-2008-03-27-Amazon Announces Static IP Addresses and Multiple Datacenter Operation
topicId topicWeight
[(1, 0.029), (2, 0.154), (10, 0.066), (17, 0.228), (40, 0.018), (42, 0.02), (61, 0.067), (77, 0.019), (79, 0.231), (85, 0.018), (94, 0.063)]
simIndex simValue blogId blogTitle
1 0.91482419 1393 high scalability-2013-01-24-NoSQL Parody: say No! No! and No!
Introduction: While certainly not in the same class as Hilarious Video: Relational Database vs NoSQL Fanbois or NSFW: Hilarious Fault-Tolerance Cartoon , this parody does have some really good moments:
2 0.88690251 1225 high scalability-2012-04-09-Why My Slime Mold is Better than Your Hadoop Cluster
Introduction: Update : Organism without a brain creates external memories for navigation shows slime mold is even cooler than originally thought, storing a record of where it's been using slime: The authors conclude, the slime isn't just the mold's calling card. Instead, it's a way of marking the environment so that the organism can sense where it's been, and not expend effort on searches that won't pay off. Although the situation isn't an exact parallel, the authors make a comparison to the pheromone trails used by ants. In After Life: The Strange Science Of Decay there’s a truly incredible sequence of gorgeously shot video showing how creeping slime mold solves mazes and performs other other amazing feats of computation. Take a look at what simple one celled organisms can do: The whole video is really well done and shockingly revelatory. It’s the story of decay, how atoms created during the Big Bang and through countless supernova explosions are continually rearranged an
same-blog 3 0.87362587 1392 high scalability-2013-01-23-Building Redundant Datacenter Networks is Not For Sissies - Use an Outside WAN Backbone
Introduction: Ivan Pepelnjak, in his short and information packed REDUNDANT DATA CENTER INTERNET CONNECTIVIT Y video, shows why networking as played at the highest levels is something you want to leave to professionals, like a large animal country vetenarian delivering a stuck foal at 2AM on a dark and stormy night. There are always a lot questions about the black art of building redundant datacenter networks and there's a shortage of accessible explanations. What I liked about Ivan's video is how effortlessly he explains the issues and tradeoffs you can expect in designing your own solution, as well as giving creative solutions to those problems. A lot of years of experience are boiled down to a 17 minute video. Ivan begins by showing what a canonical fully redundant datacenter would look like: It's like an ark where everything goes two by two. You have two datacenters, each datacenter has redundant core switches, redundant servers, redundant disk arrays, redundant links between d
Introduction: If you were going to design a next generation Internet at the physical layer that routes around the current Internet, what would it look like? What should it do? How should it work? Who should own it? How should it be paid for? How would you access it? It has long been said the Internet routes around obstacles. Snowden has revealed some major obstacles. The beauty of the current current app and web system is the physical network doesn't matter. We can just replace it with something else. Something that doesn't flow through choke points like backhaul networks , under sea cables , and cell towers . What might that something else look like? Google's Loon Project Project Loon was so named because the idea was thought to be loony. Maybe not. The idea is to float high-altitude balloons 20 miles in the air to create an aerial wireless network with up to 3G-like speeds. Signals travel through the balloon network from balloon to balloon, then to a ground-based station conne
5 0.85217875 506 high scalability-2009-02-03-10 More Rules for Even Faster Websites
Introduction: Update: How-To Minimize Load Time for Fast User Experiences . Shows how to analyze the bottlenecks preventing websites and blogs from loading quickly and how to resolve them. 80-90% of the end-user response time is spent on the frontend, so it makes sense to concentrate efforts there before heroically rewriting the backend. Take a shower before buying a Porsche, if you know what I mean. Steve Souders, author of High Performance Websites and Yslow , has ten more best practices to speed up your website : Split the initial payload Load scripts without blocking Don’t scatter scripts Split dominant content domains Make static content cookie-free Reduce cookie weight Minify CSS Optimize images Use iframes sparingly To www or not to www Sadly, according to String Theory, there are only 26.7 rules left, so get them while they're still in our dimension. Here are slides on the first few rules. Love the speeding dog slide. That's exactly what my dog looks like trav
6 0.83824027 1467 high scalability-2013-05-30-Google Finds NUMA Up to 20% Slower for Gmail and Websearch
7 0.77634484 956 high scalability-2010-12-08-How To Get Experience Working With Large Datasets
8 0.76878858 765 high scalability-2010-01-25-Let's Welcome our Neo-Feudal Overlords
9 0.76464057 1420 high scalability-2013-03-08-Stuff The Internet Says On Scalability For March 8, 2013
10 0.76031458 1048 high scalability-2011-05-27-Stuff The Internet Says On Scalability For May 27, 2011
11 0.75885332 1403 high scalability-2013-02-08-Stuff The Internet Says On Scalability For February 8, 2013
12 0.7548036 871 high scalability-2010-08-04-Dremel: Interactive Analysis of Web-Scale Datasets - Data as a Programming Paradigm
13 0.75385278 1494 high scalability-2013-07-19-Stuff The Internet Says On Scalability For July 19, 2013
14 0.74852872 631 high scalability-2009-06-15-Large-scale Graph Computing at Google
15 0.74772316 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
16 0.74596697 581 high scalability-2009-04-26-Map-Reduce for Machine Learning on Multicore
17 0.74564433 483 high scalability-2009-01-04-Paper: MapReduce: Simplified Data Processing on Large Clusters
18 0.74503475 867 high scalability-2010-07-27-YeSQL: An Overview of the Various Query Semantics in the Post Only-SQL World
19 0.74444157 543 high scalability-2009-03-17-Sun to Announce Open Cloud APIs at CommunityOne
20 0.74354827 1181 high scalability-2012-01-25-Google Goes MoreSQL with Tenzing - SQL Over MapReduce