high_scalability high_scalability-2007 high_scalability-2007-25 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: A very detailed (339 pages) paper on how to use HP products to create a highly available cluster. It's somewhat dated and obviously concentrates on HP products, but it is still good information. Table of contents: 1. Disaster Tolerance and Recovery in a Serviceguard Cluster 2. Building an Extended Distance Cluster Using ServiceGuard 3. Designing a Metropolitan Cluster 4. Designing a Continental Cluster 5. Building Disaster-Tolerant Serviceguard Solutions Using Metrocluster with Continuous Access XP 6. Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with EMC SRDF 7. Cascading Failover in a Continental Cluster Evaluating the Need for Disaster Tolerance What is a Disaster Tolerant Architecture? Types of Disaster Tolerant Clusters Extended Distance Clusters Metropolitan Cluster Continental Cluster Continental Cluster With Cascading Failover Disaster Tolerant Architecture Guidelines Protecting Nodes through Geographic Dispersion Protecting Data th
sentIndex sentText sentNum sentScore
1 It's somewhat dated and obviously concentrates on HP products, but it is still good information. [sent-2, score-0.031]
wordName wordTfidf (topN-words)
[('continental', 0.579), ('serviceguard', 0.307), ('continentalclusters', 0.273), ('recovery', 0.257), ('emc', 0.222), ('metrocluster', 0.216), ('cluster', 0.216), ('disaster', 0.199), ('tolerant', 0.195), ('package', 0.14), ('clustersetting', 0.136), ('symmetrix', 0.136), ('packages', 0.113), ('srdf', 0.102), ('rac', 0.098), ('continuous', 0.081), ('xp', 0.075), ('clusteradding', 0.068), ('clustercontinental', 0.068), ('configurationssetting', 0.068), ('configurationswitching', 0.068), ('disasterfailback', 0.068), ('metropolitan', 0.068), ('proceduresdata', 0.068), ('scenariosmaintaining', 0.068), ('srdfsetting', 0.068), ('distance', 0.062), ('cascading', 0.059), ('primary', 0.058), ('oracle', 0.051), ('notifications', 0.05), ('designing', 0.048), ('surviving', 0.046), ('failover', 0.04), ('access', 0.04), ('integrating', 0.04), ('device', 0.039), ('center', 0.038), ('building', 0.037), ('extended', 0.037), ('replication', 0.036), ('solutions', 0.036), ('hp', 0.035), ('remain', 0.035), ('data', 0.032), ('dated', 0.031), ('eventshow', 0.031), ('fence', 0.031), ('initialization', 0.031), ('networkingdata', 0.031)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 25 high scalability-2007-07-25-Paper: Designing Disaster Tolerant High Availability Clusters
Introduction: A very detailed (339 pages) paper on how to use HP products to create a highly available cluster. It's somewhat dated and obviously concentrates on HP products, but it is still good information. Table of contents: 1. Disaster Tolerance and Recovery in a Serviceguard Cluster 2. Building an Extended Distance Cluster Using ServiceGuard 3. Designing a Metropolitan Cluster 4. Designing a Continental Cluster 5. Building Disaster-Tolerant Serviceguard Solutions Using Metrocluster with Continuous Access XP 6. Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with EMC SRDF 7. Cascading Failover in a Continental Cluster Evaluating the Need for Disaster Tolerance What is a Disaster Tolerant Architecture? Types of Disaster Tolerant Clusters Extended Distance Clusters Metropolitan Cluster Continental Cluster Continental Cluster With Cascading Failover Disaster Tolerant Architecture Guidelines Protecting Nodes through Geographic Dispersion Protecting Data th
2 0.21039718 28 high scalability-2007-07-25-Product: NetApp MetroCluster Software
Introduction: NetApp MetroCluster Software Cost-effective is an integrated high-availability storage cluster and site failover capability. NetApp MetroCluster is an integrated high-availability and disaster recovery solution that can reduce system complexity and simplify management while ensuring greater return on investment. MetroCluster uses clustered server technology to replicate data synchronously between sites located miles apart, eliminating data loss in case of a disruption. Simple and powerful recovery process minimizes downtime, with little or no user action required. At one company I worked at they used the NetApp snap mirror feature to replicate data across long distances to multiple datacenters. They had a very fast backbone and it worked well. The issue with NetApp is always one of cost, but if you can afford it, it's a good option.
3 0.088502817 254 high scalability-2008-02-19-Hadoop Getting Closer to 1.0 Release
Introduction: Update: Yahoo! Launches World's Largest Hadoop Production Application . A 10,000 core Hadoop cluster produces data used in every Yahoo! Web search query. Raw disk is at 5 Petabytes. Their previous 1 petabyte database couldn't handle the load and couldn't grow larger. Greg Linden thinks the Google cluster has way over 133,000 machines. From an InfoQ interview with project lead Doug Cutting, it appears Hadoop , an open source distributed computing platform, is making good progress towards their 1.0 release. They've successfully reached a 1000 node cluster size, improved file system integrity, and jacked performance by 20x in the last year. How they are making progress could be a good model for anyone: The speedup has been an aggregation of our work in the past few years, and has been accomplished mostly by trial-and-error. We get things running smoothly on a cluster of a given size, then double the size of the cluster and see what breaks. We aim for performan
4 0.072148629 813 high scalability-2010-04-19-The cost of High Availability (HA) with Oracle
Introduction: What's the cost of downtime to your business? $100,000 per hour, $1,000,000 or more? The recent Volcanic ash that has grounded European flights is estimated to be costing the airlines $200M a day. In the IT world, High Availability (HA) architectures allow for disaster recovery as well as uninterrupted business continuity during system failure. This post focuses on a customer’s backend, comprised of a business application stack supported by a dozen Oracle databases. They wish to equip this infrastructure with HA features and ensure that outages do not cost business. How do we address the challenge of pricing the complete solution, with hardware, software, services and annual support? Read more on BigDataMatters.com
5 0.070934258 822 high scalability-2010-05-04-Business continuity with real-time data integration
Introduction: Enterprises want to protect their data. As the appetite for data volumes grows, storage technology becomes a critical business asset on which business continuity relies. My recent survey in the medium-size enterprise segment shows the five dominant investment directions at the level of data management architecture: disaster recovery (DR), high availability (HA), backup, data processing performance and migration to more advanced databases. This suggests that corporations generally have sufficiently structured data collections but are concerned with business continuity and continuous availability of data. What infrastructures can provide these assurances? In this post I want to focus on yet another option, and that is the Real-Time Data Integration model. As an example I am going to discuss Oracle GoldenGate, which permits you to manage the data critical to your business in safety, ensuring business continuity without disruption even if the data is distributed among multiple, h
8 0.065047689 1338 high scalability-2012-10-11-RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store
9 0.064044021 13 high scalability-2007-07-15-Lustre cluster file system
10 0.063753687 1445 high scalability-2013-04-24-Strategy: Using Lots of RAM Often Cheaper than Using a Hadoop Cluster
11 0.059632368 271 high scalability-2008-03-08-Product: DRBD - Distributed Replicated Block Device
12 0.058805004 1042 high scalability-2011-05-17-Facebook: An Example Canonical Architecture for Scaling Billions of Messages
13 0.056804497 1279 high scalability-2012-07-09-Data Replication in NoSQL Databases
14 0.056112867 1565 high scalability-2013-12-16-22 Recommendations for Building Effective High Traffic Web Software
15 0.054994196 1266 high scalability-2012-06-18-Google on Latency Tolerant Systems: Making a Predictable Whole Out of Unpredictable Parts
16 0.054824267 1057 high scalability-2011-06-10-Stuff The Internet Says On Scalability For June 10, 2011
17 0.053817835 820 high scalability-2010-05-03-100 Node Hazelcast cluster on Amazon EC2
18 0.053371493 27 high scalability-2007-07-25-Product: 3 PAR REMOTE COPY
19 0.053227291 1174 high scalability-2012-01-13-Stuff The Internet Says On Scalability For January 13, 2012
20 0.049310476 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
topicId topicWeight
[(0, 0.061), (1, 0.021), (2, -0.008), (3, 0.015), (4, -0.004), (5, 0.022), (6, 0.068), (7, -0.045), (8, 0.012), (9, 0.007), (10, -0.01), (11, 0.016), (12, 0.008), (13, -0.031), (14, 0.011), (15, 0.042), (16, -0.0), (17, 0.0), (18, -0.003), (19, 0.03), (20, 0.015), (21, 0.03), (22, -0.019), (23, 0.006), (24, -0.052), (25, 0.034), (26, -0.012), (27, -0.005), (28, -0.031), (29, -0.0), (30, -0.002), (31, 0.016), (32, 0.009), (33, -0.01), (34, -0.021), (35, 0.029), (36, -0.013), (37, -0.041), (38, 0.002), (39, 0.023), (40, 0.038), (41, -0.027), (42, -0.025), (43, 0.046), (44, -0.014), (45, -0.001), (46, 0.0), (47, 0.0), (48, -0.046), (49, -0.019)]
simIndex simValue blogId blogTitle
same-blog 1 0.94774014 25 high scalability-2007-07-25-Paper: Designing Disaster Tolerant High Availability Clusters
Introduction: A very detailed (339 pages) paper on how to use HP products to create a highly available cluster. It's somewhat dated and obviously concentrates on HP products, but it is still good information. Table of contents: 1. Disaster Tolerance and Recovery in a Serviceguard Cluster 2. Building an Extended Distance Cluster Using ServiceGuard 3. Designing a Metropolitan Cluster 4. Designing a Continental Cluster 5. Building Disaster-Tolerant Serviceguard Solutions Using Metrocluster with Continuous Access XP 6. Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with EMC SRDF 7. Cascading Failover in a Continental Cluster Evaluating the Need for Disaster Tolerance What is a Disaster Tolerant Architecture? Types of Disaster Tolerant Clusters Extended Distance Clusters Metropolitan Cluster Continental Cluster Continental Cluster With Cascading Failover Disaster Tolerant Architecture Guidelines Protecting Nodes through Geographic Dispersion Protecting Data th
2 0.64144802 271 high scalability-2008-03-08-Product: DRBD - Distributed Replicated Block Device
Introduction: From their website: DRBD is a block device which is designed to build high availability clusters. This is done by mirroring a whole block device via (a dedicated) network. You could see it as a network raid-1. DRBD takes over the data, writes it to the local disk and sends it to the other host. On the other host, it takes it to the disk there. The other components needed are a cluster membership service, which is supposed to be heartbeat, and some kind of application that works on top of a block device. Examples: A filesystem & fsck. A journaling FS. A database with recovery capabilities. Each device (DRBD provides more than one of these devices) has a state, which can be 'primary' or 'secondary'. On the node with the primary device the application is supposed to run and to access the device (/dev/drbdX). Every write is sent to the local 'lower level block device' and to the node with the device in 'secondary' state. The secondary device simply writes the data to its lowe
3 0.63144165 28 high scalability-2007-07-25-Product: NetApp MetroCluster Software
Introduction: NetApp MetroCluster Software Cost-effective is an integrated high-availability storage cluster and site failover capability. NetApp MetroCluster is an integrated high-availability and disaster recovery solution that can reduce system complexity and simplify management while ensuring greater return on investment. MetroCluster uses clustered server technology to replicate data synchronously between sites located miles apart, eliminating data loss in case of a disruption. Simple and powerful recovery process minimizes downtime, with little or no user action required. At one company I worked at they used the NetApp snap mirror feature to replicate data across long distances to multiple datacenters. They had a very fast backbone and it worked well. The issue with NetApp is always one of cost, but if you can afford it, it's a good option.
4 0.62462682 1338 high scalability-2012-10-11-RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store
Introduction: RAMCube is a datacenter oriented design for RAM-based key-value store that supports thousands or tens of thousands of servers to offer up to hundreds of terabytes of RAM storage. Here's the PDF Paper describing the system and here's a video of the presentation given at HotCloud . The big idea is: RAMCube exploits the proximity of a BCube network to construct a symmetric MultiRing structure, restricting all failure detection and recovery traffic within a one-hop neighborhood, which addresses problems including false failure detection and recovery traffic congestion. In addition, RAMCube leverages BCube’s multiple paths between any pairs of servers to handle switch failures. A few notes: 75% of Facebook data is stored in memcache. RAM is 1000 time faster than disk RAM is used in caches, but this increases application complexity as applications are responsible for cache consistency. Under a high work load a 1% cache miss rate can lead to a 10x performance penalty. So st
5 0.6090458 13 high scalability-2007-07-15-Lustre cluster file system
Introduction: Lustre速 is a scalable, secure, robust, highly-available cluster file system. It is designed, developed and maintained by Cluster File Systems, Inc. The central goal is the development of a next-generation cluster file system which can serve clusters with 10,000's of nodes, provide petabytes of storage, and move 100's of GB/sec with state-of-the-art security and management infrastructure. Lustre runs on many of the largest Linux clusters in the world, and is included by CFS's partners as a core component of their cluster offering (examples include HP StorageWorks SFS, and the Cray XT3 and XD1 supercomputers). Today's users have also demonstrated that Lustre scales down as well as it scales up, and runs in production on clusters as small as 4 and as large as 25,000 nodes. The latest version of Lustre is always available from Cluster File Systems, Inc. Public Open Source releases of Lustre are available under the GNU General Public License. These releases are found here, and are used
6 0.5939036 114 high scalability-2007-10-07-Product: Wackamole
8 0.58230865 101 high scalability-2007-09-27-Product: Ganglia Monitoring System
9 0.57676649 820 high scalability-2010-05-03-100 Node Hazelcast cluster on Amazon EC2
10 0.56822318 1386 high scalability-2013-01-14-MongoDB and GridFS for Inter and Intra Datacenter Data Replication
11 0.56699497 27 high scalability-2007-07-25-Product: 3 PAR REMOTE COPY
12 0.55851471 1142 high scalability-2011-11-14-Using Gossip Protocols for Failure Detection, Monitoring, Messaging and Other Good Things
13 0.54576737 1020 high scalability-2011-04-12-Caching and Processing 2TB Mozilla Crash Reports in memory with Hazelcast
14 0.5449996 68 high scalability-2007-08-20-TypePad Architecture
15 0.54019743 1041 high scalability-2011-05-15-Building a Database remote availability site
16 0.53360707 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day
17 0.52866745 368 high scalability-2008-08-17-Wuala - P2P Online Storage Cloud
18 0.52749032 809 high scalability-2010-04-13-Strategy: Saving Your Butt With Deferred Deletes
19 0.52543294 1042 high scalability-2011-05-17-Facebook: An Example Canonical Architecture for Scaling Billions of Messages
20 0.52357239 1221 high scalability-2012-04-03-Hazelcast 2.0: Big Data In-Memory
topicId topicWeight
[(1, 0.109), (2, 0.185), (10, 0.016), (11, 0.396), (27, 0.068), (61, 0.032), (85, 0.027), (94, 0.014)]
simIndex simValue blogId blogTitle
same-blog 1 0.82322234 25 high scalability-2007-07-25-Paper: Designing Disaster Tolerant High Availability Clusters
Introduction: A very detailed (339 pages) paper on how to use HP products to create a highly available cluster. It's somewhat dated and obviously concentrates on HP products, but it is still good information. Table of contents: 1. Disaster Tolerance and Recovery in a Serviceguard Cluster 2. Building an Extended Distance Cluster Using ServiceGuard 3. Designing a Metropolitan Cluster 4. Designing a Continental Cluster 5. Building Disaster-Tolerant Serviceguard Solutions Using Metrocluster with Continuous Access XP 6. Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with EMC SRDF 7. Cascading Failover in a Continental Cluster Evaluating the Need for Disaster Tolerance What is a Disaster Tolerant Architecture? Types of Disaster Tolerant Clusters Extended Distance Clusters Metropolitan Cluster Continental Cluster Continental Cluster With Cascading Failover Disaster Tolerant Architecture Guidelines Protecting Nodes through Geographic Dispersion Protecting Data th
2 0.74621874 668 high scalability-2009-08-01-15 Scalability and Performance Best Practices
Introduction: These are from Laura Thomson of OmniTi : Profile early, profile often. Pick a profiling tool and learn it in and out. Dev-ops cooperation is essential. The most critical difference in organizations that handles crises well. Test on production data. Code behavior (especially performance) is often data driven. Track and trend. Understanding your historical performance characteristics is essential for spotting emerging problems. Assumptions will burn you. Systems are complex and often break in unexpected ways. Decouple. Isolate performance failures. Cache. Caching is the core of most optimizations. Federate. Data federation is taking a single data set and spreading it across multiple database/application servers. Replicate. Replication is making synchronized copies of data available in more than one place. Avoid straining hard-to-scale resources. Some resources are inherently hard to scale: Uncacheable’ data, Data with a very high read+write rate
3 0.72311658 134 high scalability-2007-10-26-Paper: Wikipedia's Site Internals, Configuration, Code Examples and Management Issues
Introduction: Wikipedia and Wikimedia have some of the best, most complete real-world documentation on how to build highly scalable systems. This paper by Domas Mituzas covers a lot of details about how Wikipedia works, including: an overview of the different packages used (Linux, PowerDNS, LVS, Squid, lighttpd, Apache, PHP5, Lucene, Mono, Memcached), how they use their CDN, how caching works, how they profile their code, how they store their media, how they structure their database access, how they handle search, how they handle load balancing and administration. All with real code examples and examples of configuration files. This is a really useful resource. Related Articles Wikimedia Architecture Domas Mituzas' Blog
4 0.62424308 457 high scalability-2008-12-01-Sun FireTM X4540 Server as Backup Server for Zmanda's Amanda Enterprise 2.6 Software
Introduction: Sun FireTM X4540 Server as Backup Server for Zmanda's Amanda Enterprise 2.6 Software by Thomas Hanvey (Sun Microsystems) and Dmitri Joukovski and Ken Crandall (Zmanda) September, 2008 Explosive data growth, combined with demanding requirements for data availability, has placed a tremendous burden on IT operations staff at businesses of all sizes. Yet, many organizations do not have the staff or budget to purchase and manage complex and expensive backup and recovery software products. The Sun FireTM X4540 server can deliver massive storage capacity and remarkable throughput so it is well-suited as a nearline storage platform for backup and restore applications. Combining the power of the SolarisTM 10 Operating System with the data integrity and simplified administration of ZFS, the Sun Fire X4540 server can be an ideal candidate for streamlining and improving backup and restore operations. Amanda Enterprise Edition from Zmanda was designed to address these challenges,
5 0.60864407 908 high scalability-2010-09-28-6 Strategies for Scaling BBC iPlayer
Introduction: The BBC's iPlayer site averages 8 million page views a day for 1.3 million users. Technical Architect Simon Frost describes how they scaled their site in Scaling the BBC iPlayer to handle demand : Use frameworks . Frameworks support component based development which makes it convenient for team development, but can introduce delays that have to be minimized. Zend/PHP is used because it supports components and is easy to recruit for. MySQL is used for program metadata. CouchDB is used for key-value access for fast read/write of user-focused data. Prove architecture before building it . Eliminate guesswork by coming up with alternate architectures and create prototypes to determine which option works best. Balance performance with factors like ease of development. Cache a lot . Data is cached in memcached for a few seconds to minutes. Short cache invalidation periods keep the data up to date for the users, but even these short periods make a huge difference in performance.
7 0.59195739 771 high scalability-2010-02-04-Hot Scalability Links for February 4, 2010
8 0.57052076 105 high scalability-2007-10-01-Statistics Logging Scalability
9 0.56610054 1236 high scalability-2012-04-30-Masstree - Much Faster than MongoDB, VoltDB, Redis, and Competitive with Memcached
10 0.56324077 1055 high scalability-2011-06-08-Stuff to Watch from Google IO 2011
11 0.54490966 136 high scalability-2007-10-28-Scaling Early Stage Startups
12 0.54024071 699 high scalability-2009-09-10-How to handle so many socket connection
13 0.51050597 5 high scalability-2007-07-10-mixi.jp Architecture
14 0.49403605 824 high scalability-2010-05-06-Going global on EC2
15 0.49183294 442 high scalability-2008-11-13-Plenty of Fish Says Scaling for Free Doesn't Pay
16 0.47956634 72 high scalability-2007-08-22-Wikimedia architecture
17 0.47476959 883 high scalability-2010-08-20-Hot Scalability Links For Aug 20, 2010
18 0.47448522 754 high scalability-2009-12-22-Incremental deployment
19 0.47166109 1141 high scalability-2011-11-11-Stuff The Internet Says On Scalability For November 11, 2011
20 0.47121289 303 high scalability-2008-04-18-Scaling Mania at MySQL Conference 2008