high_scalability high_scalability-2007 high_scalability-2007-114 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Wackamole is an application that helps with making a cluster highly available. It manages a bunch of virtual IPs, that should be available to the outside world at all times. Wackamole ensures that a single machine within a cluster is listening on each virtual IP address that Wackamole manages. If it discovers that particular machines within the cluster are not alive, it will almost immediately ensure that other machines acquire these public IPs. At no time will more than one machine listen on any virtual IP. Wackamole also works toward achieving a balanced distribution of number IPs on the machine within the cluster it manages. There is no other software like Wackamole. Wackamole is quite unique in that it operates in a completely peer-to-peer mode within the cluster. Other products that provide the same high-availability guarantees use a "VIP" method. Wackamole is an application that runs as root in a cluster to make it highly available. It uses the membership notifications prov
sentIndex sentText sentNum sentScore
1 Wackamole is an application that helps with making a cluster highly available. [sent-1, score-0.237]
2 It manages a bunch of virtual IPs, that should be available to the outside world at all times. [sent-2, score-0.299]
3 Wackamole ensures that a single machine within a cluster is listening on each virtual IP address that Wackamole manages. [sent-3, score-0.694]
4 If it discovers that particular machines within the cluster are not alive, it will almost immediately ensure that other machines acquire these public IPs. [sent-4, score-0.944]
5 At no time will more than one machine listen on any virtual IP. [sent-5, score-0.295]
6 Wackamole also works toward achieving a balanced distribution of number IPs on the machine within the cluster it manages. [sent-6, score-0.711]
7 Wackamole is quite unique in that it operates in a completely peer-to-peer mode within the cluster. [sent-8, score-0.291]
8 Other products that provide the same high-availability guarantees use a "VIP" method. [sent-9, score-0.074]
9 Wackamole is an application that runs as root in a cluster to make it highly available. [sent-10, score-0.346]
10 It uses the membership notifications provided by the Spread toolkit to generate a consistent state that is agreed upon among all of the connected Wackamole instances. [sent-11, score-0.73]
11 Wackamole is released under the CNDS Open Source License. [sent-12, score-0.065]
12 Note: This post has been adapted from the linked to web site. [sent-13, score-0.184]
wordName wordTfidf (topN-words)
[('wackamole', 0.775), ('ips', 0.179), ('cluster', 0.169), ('within', 0.141), ('discovers', 0.132), ('clustersby', 0.126), ('vip', 0.117), ('virtual', 0.114), ('agreed', 0.103), ('machine', 0.1), ('acquire', 0.1), ('adapted', 0.1), ('toolkit', 0.1), ('toward', 0.097), ('membership', 0.092), ('alive', 0.091), ('schlossnagle', 0.09), ('theo', 0.089), ('listening', 0.087), ('linked', 0.084), ('ensures', 0.083), ('listen', 0.081), ('achieving', 0.081), ('operates', 0.081), ('notifications', 0.076), ('machines', 0.075), ('guarantees', 0.074), ('balanced', 0.072), ('mode', 0.069), ('highly', 0.068), ('bunch', 0.067), ('upon', 0.066), ('released', 0.065), ('manages', 0.064), ('root', 0.063), ('among', 0.062), ('generate', 0.061), ('connected', 0.06), ('ip', 0.06), ('immediately', 0.06), ('provided', 0.06), ('spread', 0.057), ('outside', 0.054), ('public', 0.053), ('distribution', 0.051), ('consistent', 0.05), ('particular', 0.048), ('almost', 0.046), ('runs', 0.046), ('ensure', 0.045)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 114 high scalability-2007-10-07-Product: Wackamole
Introduction: Wackamole is an application that helps with making a cluster highly available. It manages a bunch of virtual IPs, that should be available to the outside world at all times. Wackamole ensures that a single machine within a cluster is listening on each virtual IP address that Wackamole manages. If it discovers that particular machines within the cluster are not alive, it will almost immediately ensure that other machines acquire these public IPs. At no time will more than one machine listen on any virtual IP. Wackamole also works toward achieving a balanced distribution of number IPs on the machine within the cluster it manages. There is no other software like Wackamole. Wackamole is quite unique in that it operates in a completely peer-to-peer mode within the cluster. Other products that provide the same high-availability guarantees use a "VIP" method. Wackamole is an application that runs as root in a cluster to make it highly available. It uses the membership notifications prov
2 0.092327669 289 high scalability-2008-03-27-Amazon Announces Static IP Addresses and Multiple Datacenter Operation
Introduction: Amazon is fixing two of their major problems: no static IP addresses and single datacenter operation. By adding these two new features developers can finally build a no apology system on Amazon. Before you always had to throw in an apology or two. No, we don't have low failover times because of the silly DNS games and unexceptionable DNS update and propagation times and no, we don't operate in more than one datacenter. No more. Now Amazon is adding Elastic IP Addresses and Availability Zones . Elastic IP addresses are far better than normal IP addresses because they are both in tight with Jessica Alba and they are: Static IP addresses designed for dynamic cloud computing. An Elastic IP address is associated with your account, not a particular instance, and you control that address until you choose to explicitly release it. Unlike traditional static IP addresses, however, Elastic IP addresses allow you to mask instance or availability zone failures by programmatica
3 0.087819099 168 high scalability-2007-11-30-Strategy: Efficiently Geo-referencing IPs
Introduction: A lot of apps need to map IP addresses to locations. Jeremy Cole in On efficiently geo-referencing IPs with MaxMind GeoIP and MySQL GIS succinctly explains the many uses for such a feature: Geo-referencing IPs is, in a nutshell, converting an IP address, perhaps from an incoming web visitor, a log file, a data file, or some other place, into the name of some entity owning that IP address. There are a lot of reasons you may want to geo-reference IP addresses to country, city, etc., such as in simple ad targeting systems, geographic load balancing, web analytics, and many more applications. This is difficult to do efficiently, at least it gives me a bit of brain freeze. In the same post Jeremy nicely explains where to get the geo-rereferncing data, how to load data, and the performance of different approaches for IP address searching. It's a great practical introduction to the subject.
4 0.086723037 122 high scalability-2007-10-14-Product: The Spread Toolkit
Introduction: Complex applications coordinating work across a lot of machines often need a highly performing fault tolerant message layer. Though a blast to write, it's probably a better use of your time to use an off the shelf solution. And that's where Spread comes in. Flickr, for example, uses Spread to create real-time event feeds from their web server logs. What exactly is Spread? From the Spread website: Spread is an open source toolkit that provides a high performance messaging service that is resilient to faults across local and wide area networks. Spread functions as a unified message bus for distributed applications, and provides highly tuned application-level multicast, group communication, and point to point support. Spread services range from reliable messaging to fully ordered messages with delivery guarantees. Spread can be used in many distributed applications that require high reliability, high performance, and robust communication among various subsets of members. The
5 0.07445924 254 high scalability-2008-02-19-Hadoop Getting Closer to 1.0 Release
Introduction: Update: Yahoo! Launches World's Largest Hadoop Production Application . A 10,000 core Hadoop cluster produces data used in every Yahoo! Web search query. Raw disk is at 5 Petabytes. Their previous 1 petabyte database couldn't handle the load and couldn't grow larger. Greg Linden thinks the Google cluster has way over 133,000 machines. From an InfoQ interview with project lead Doug Cutting, it appears Hadoop , an open source distributed computing platform, is making good progress towards their 1.0 release. They've successfully reached a 1000 node cluster size, improved file system integrity, and jacked performance by 20x in the last year. How they are making progress could be a good model for anyone: The speedup has been an aggregation of our work in the past few years, and has been accomplished mostly by trial-and-error. We get things running smoothly on a cluster of a given size, then double the size of the cluster and see what breaks. We aim for performan
6 0.060277052 13 high scalability-2007-07-15-Lustre cluster file system
7 0.060119644 1075 high scalability-2011-07-07-Myth: Google Uses Server Farms So You Should Too - Resurrection of the Big-Ass Machines
8 0.059005942 87 high scalability-2007-09-10-Blog: Esoteric Curio by Theo Schlossnagle
9 0.057450708 1042 high scalability-2011-05-17-Facebook: An Example Canonical Architecture for Scaling Billions of Messages
10 0.057416268 790 high scalability-2010-03-09-Applications as Virtual States
11 0.054604806 841 high scalability-2010-06-14-How scalable could be a cPanel Hosting service?
12 0.054489091 117 high scalability-2007-10-08-Paper: Understanding and Building High Availability-Load Balanced Clusters
13 0.054154187 890 high scalability-2010-09-01-Paper: The Case for Determinism in Database Systems
14 0.052109785 507 high scalability-2009-02-03-Paper: Optimistic Replication
15 0.051842812 1445 high scalability-2013-04-24-Strategy: Using Lots of RAM Often Cheaper than Using a Hadoop Cluster
16 0.05002727 111 high scalability-2007-10-04-Number of load balanced servers
17 0.048683111 1279 high scalability-2012-07-09-Data Replication in NoSQL Databases
18 0.047805984 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
20 0.047787625 1307 high scalability-2012-08-20-The Performance of Distributed Data-Structures Running on a "Cache-Coherent" In-Memory Data Grid
topicId topicWeight
[(0, 0.079), (1, 0.017), (2, 0.011), (3, 0.009), (4, -0.015), (5, 0.001), (6, 0.045), (7, -0.022), (8, -0.012), (9, 0.019), (10, 0.001), (11, 0.02), (12, 0.006), (13, -0.019), (14, 0.018), (15, 0.012), (16, 0.016), (17, -0.01), (18, -0.009), (19, 0.004), (20, 0.012), (21, 0.029), (22, -0.03), (23, -0.016), (24, -0.018), (25, 0.036), (26, -0.009), (27, -0.015), (28, -0.032), (29, -0.009), (30, 0.007), (31, -0.003), (32, -0.009), (33, -0.002), (34, 0.012), (35, 0.02), (36, -0.03), (37, -0.044), (38, -0.008), (39, 0.015), (40, -0.008), (41, 0.028), (42, -0.014), (43, 0.002), (44, -0.015), (45, 0.08), (46, 0.012), (47, -0.004), (48, 0.005), (49, 0.007)]
simIndex simValue blogId blogTitle
same-blog 1 0.96009463 114 high scalability-2007-10-07-Product: Wackamole
Introduction: Wackamole is an application that helps with making a cluster highly available. It manages a bunch of virtual IPs, that should be available to the outside world at all times. Wackamole ensures that a single machine within a cluster is listening on each virtual IP address that Wackamole manages. If it discovers that particular machines within the cluster are not alive, it will almost immediately ensure that other machines acquire these public IPs. At no time will more than one machine listen on any virtual IP. Wackamole also works toward achieving a balanced distribution of number IPs on the machine within the cluster it manages. There is no other software like Wackamole. Wackamole is quite unique in that it operates in a completely peer-to-peer mode within the cluster. Other products that provide the same high-availability guarantees use a "VIP" method. Wackamole is an application that runs as root in a cluster to make it highly available. It uses the membership notifications prov
2 0.71274781 13 high scalability-2007-07-15-Lustre cluster file system
Introduction: Lustre速 is a scalable, secure, robust, highly-available cluster file system. It is designed, developed and maintained by Cluster File Systems, Inc. The central goal is the development of a next-generation cluster file system which can serve clusters with 10,000's of nodes, provide petabytes of storage, and move 100's of GB/sec with state-of-the-art security and management infrastructure. Lustre runs on many of the largest Linux clusters in the world, and is included by CFS's partners as a core component of their cluster offering (examples include HP StorageWorks SFS, and the Cray XT3 and XD1 supercomputers). Today's users have also demonstrated that Lustre scales down as well as it scales up, and runs in production on clusters as small as 4 and as large as 25,000 nodes. The latest version of Lustre is always available from Cluster File Systems, Inc. Public Open Source releases of Lustre are available under the GNU General Public License. These releases are found here, and are used
3 0.6582045 228 high scalability-2008-01-28-Product: ISPMan Centralized ISP Management System
Introduction: From FRESH Ports and their website: ISPman is an ISP management software written in perl, using an LDAP backend to manage virtual hosts for an ISP. It can be used to manage, DNS, virtual hosts for apache config, postfix configuration, cyrus mail boxes, proftpd etc. ISPMan was written as a management tool for the network at 4unet where between 30 to 50 domains are hosted and the number is crazily growing. Managing these domains and their users was a little time consuming, and needed an Administrator who knows linux and these daemons fluently. Now the help-desk can easily manage the domains and users. LDAP data can be easily replicated site wide, and mail box server can be scaled from 1 to n as required. An LDAP entry called maildrop tells the SMTP server (postfix) where to deliver the mail. The SMTP servers can be loadbalanced with one of many load balancing techniques. The program is written with scalability and High availability in mind. This may not be the right s
4 0.65323538 272 high scalability-2008-03-08-Product: FAI - Fully Automatic Installation
Introduction: From their website: FAI is an automated installation tool to install or deploy Debian GNU/Linux and other distributions on a bunch of different hosts or a Cluster. It's more flexible than other tools like kickstart for Red Hat, autoyast and alice for SuSE or Jumpstart for SUN Solaris. FAI can also be used for configuration management of a running system. You can take one or more virgin PCs, turn on the power and after a few minutes Linux is installed, configured and running on all your machines, without any interaction necessary. FAI it's a scalable method for installing and updating all your computers unattended with little effort involved. It's a centralized management system for your Linux deployment. FAI's target group are system administrators who have to install Linux onto one or even hundreds of computers. It's not only a tool for doing a Cluster installation but a general purpose installation tool. It can be used for installing a Beowulf cluster, a rendering farm,
Introduction: At a Cloud Computing Meetup , Siddharth "Sid" Anand of Netflix, backed by a merry band of Netflixians, gave an interesting talk: Keeping Movies Running Amid Thunderstorms . While the talk gave a good overview of their move to the cloud, issues with capacity planning, thundering herds , latency problems, and simian armageddon , I found myself most taken with how they handle software deployment in the cloud . I've worked on half a dozen or more build and deployment systems, some small, some quite large, but never for a large organization like Netflix in the cloud. The cloud has this amazing capability that has never existed before that enables a novel approach to fault-tolerant software deployments: the ability to spin up huge numbers of instances to completely run a new release while running the old release at the same time . The process goes something like: A canary machine is launched first with the new software load running real traffic to sanity test the load in a p
6 0.61402583 227 high scalability-2008-01-28-Howto setup GFS-GNBD
7 0.612427 820 high scalability-2010-05-03-100 Node Hazelcast cluster on Amazon EC2
8 0.60946035 263 high scalability-2008-02-27-Product: System Imager - Automate Deployment and Installs
9 0.60173815 25 high scalability-2007-07-25-Paper: Designing Disaster Tolerant High Availability Clusters
10 0.59063017 1142 high scalability-2011-11-14-Using Gossip Protocols for Failure Detection, Monitoring, Messaging and Other Good Things
11 0.5903253 254 high scalability-2008-02-19-Hadoop Getting Closer to 1.0 Release
13 0.58271593 807 high scalability-2010-04-09-Vagrant - Build and Deploy Virtualized Development Environments Using Ruby
14 0.58139139 1104 high scalability-2011-08-25-Colmux - Finding Memory Leaks, High I-O Wait Times, and Hotness on 3000 Node Clusters
15 0.57887083 1256 high scalability-2012-06-04-OpenFlow-SDN is Not a Silver Bullet for Network Scalability
16 0.57565838 773 high scalability-2010-02-06-GEO-aware traffic load balancing and caching at CNBC.com
17 0.55148232 1260 high scalability-2012-06-07-Case Study on Scaling PaaS infrastructure
18 0.55017382 237 high scalability-2008-02-03-Product: Collectl - Performance Data Collector
19 0.54600078 42 high scalability-2007-07-30-Product: GridLayer. Utility computing for online application
20 0.54575348 1077 high scalability-2011-07-11-ATMCash Exploits Virtualization for Security - Immutability and Reversion
topicId topicWeight
[(1, 0.174), (2, 0.099), (30, 0.025), (34, 0.196), (61, 0.046), (69, 0.017), (79, 0.174), (85, 0.071), (94, 0.053)]
simIndex simValue blogId blogTitle
same-blog 1 0.89151561 114 high scalability-2007-10-07-Product: Wackamole
Introduction: Wackamole is an application that helps with making a cluster highly available. It manages a bunch of virtual IPs, that should be available to the outside world at all times. Wackamole ensures that a single machine within a cluster is listening on each virtual IP address that Wackamole manages. If it discovers that particular machines within the cluster are not alive, it will almost immediately ensure that other machines acquire these public IPs. At no time will more than one machine listen on any virtual IP. Wackamole also works toward achieving a balanced distribution of number IPs on the machine within the cluster it manages. There is no other software like Wackamole. Wackamole is quite unique in that it operates in a completely peer-to-peer mode within the cluster. Other products that provide the same high-availability guarantees use a "VIP" method. Wackamole is an application that runs as root in a cluster to make it highly available. It uses the membership notifications prov
2 0.84186113 777 high scalability-2010-02-15-Scaling Ambition at StackOverflow
Introduction: Joel Spolsky and Jeff Atwood are raising VC money for StackOverflow. This is interesting for three reasons: 1) Joel has always seemed like a keep it small and grow organically type of guy, so this is a big step in a different direction. 2) It means they think there's a very big market in the Q&A; space and they mean to capture as much as the market as possible. 3) Most importantly for this blog, Joel gives some good advice on when to stay fresh and local and when it's time to jump for the brass ring, scale up your ambition, and go for VC money. Please see Joel's blog post for the details, but here's when to go VC: There’s a land grab going on. There is a provable concept that’s repeatable. The business itself could benefit from the publicity. The investor will add substantial value to the business. The business can potentially have a big exit or become a large, publically traded company. The founders are not in it for their own personal aggrandizement. Joel t
3 0.83218521 1594 high scalability-2014-02-12-Paper: Network Stack Specialization for Performance
Introduction: In the scalability is specialization department here is an interesting paper presented at HotNets '13 on high performance networking: Network Stack Specialization for Performance . The idea is generalizing a service so it fits in the kernel comes at a high performance cost. So move TCP into user space. The result is a web server with ~3.5x the throughput of Nginx "while experiencing low CPU utilization, linear scaling on multicore systems, and saturating current NIC hardware." Here's a good description of the paper published on Layer 9 : Traditionally, servers and OSes have been built to be general purpose. However now we have a high degree of specialization. In fact, in a big web service, you might have thousands of machines dedicated to one function. Therefore, there's scope for specialization. This paper looks at a specific opportunity in that space. Network stacks today are good for high throughput with large transfers, but not small files (which are common in web browsi
4 0.79137194 830 high scalability-2010-05-25-Strategy: Rule of 3 Admins to Save Your Sanity
Introduction: The idea came up in this Hacker News thread , commenting on a 37signals interview, that having three system administrators is the minimum optimal number of admins. Everyone wants to lower their costs by having each admin administer a lot of machines. The problem is when you have fewer than three admins you can never get a break from the constant corrosive pressure of always being on call. When every moment of your life you are dreading the next emergency, it eats at you. Having three admins solves that problem. With three admins you can: Go on a real vacation. The two remaining admins can switch off being on call. Not be on call all the time. A larger shop will naturally have more admins so it's not as big an issue, but at smaller shops trying to minimize head count, carrying three admins (or people in those roles) might be something to consider.
5 0.78933483 706 high scalability-2009-09-16-The VeriScale Architecture - Elasticity and efficiency for private clouds
Introduction: The modern datacenter is evolving into the network centric datacenter model, which is applied to both public and private cloud computing. In this model, networking, platform, storage, and software infrastructure are provided as services that scale up or down on demand. The network centric model allows the datacenter to be viewed as a collection of automatically deployed and managed application services that utilize underlying virtualized services. Providing sufficient elasticity and scalability for the rapidly growing needs of the datacenter requires these collections of automatically-managed services to scale efficiently and with essentially no limits, letting services adapt easily to changing requirements and workloads. Sun’s VeriScale architecture provides the architectural platform that can deliver these capabilities. Sun Microsystems has been developing open and modular infrastructure architectures for more than a decade. The features of these architectures, such as elasticity, ar
6 0.78079778 442 high scalability-2008-11-13-Plenty of Fish Says Scaling for Free Doesn't Pay
7 0.78061581 1092 high scalability-2011-08-04-Jim Starkey is Creating a Brave New World by Rethinking Databases for the Cloud
8 0.78051525 380 high scalability-2008-09-05-Product: Tungsten Replicator
9 0.77798676 1575 high scalability-2014-01-08-Under Snowden's Light Software Architecture Choices Become Murky
10 0.77693796 387 high scalability-2008-09-22-Paper: On Delivering Embarrassingly Distributed Cloud Services
11 0.77651948 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
12 0.77170545 304 high scalability-2008-04-19-How to build a real-time analytics system?
13 0.77121764 1056 high scalability-2011-06-09-Retrospect on recent AWS outage and Resilient Cloud-Based Architecture
14 0.76415312 450 high scalability-2008-11-24-Scalability Perspectives #3: Marc Andreessen – Internet Platforms
15 0.76329809 1036 high scalability-2011-05-06-Stuff The Internet Says On Scalability For May 6th, 2011
16 0.76284999 888 high scalability-2010-08-27-OpenStack - The Answer to: How do We Compete with Amazon?
17 0.7628414 650 high scalability-2009-07-02-Product: Hbase
18 0.76234239 509 high scalability-2009-02-05-Product: HAProxy - The Reliable, High Performance TCP-HTTP Load Balancer
19 0.76139909 1014 high scalability-2011-03-31-8 Lessons We Can Learn from the MySpace Incident - Balance, Vision, Fearlessness