high_scalability high_scalability-2007 high_scalability-2007-101 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
sentIndex sentText sentNum sentScore
1 Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. [sent-1, score-0.699]
2 It is based on a hierarchical design targeted at federations of clusters. [sent-2, score-0.656]
3 It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. [sent-3, score-0.918]
4 It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. [sent-4, score-1.083]
5 The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. [sent-5, score-1.463]
6 It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes. [sent-6, score-1.684]
wordName wordTfidf (topN-words)
[('clusters', 0.402), ('campuses', 0.267), ('federations', 0.267), ('rrdtool', 0.251), ('ported', 0.231), ('overheads', 0.224), ('engineered', 0.19), ('compact', 0.178), ('transport', 0.17), ('representation', 0.168), ('portable', 0.162), ('leverages', 0.16), ('widely', 0.156), ('university', 0.152), ('carefully', 0.151), ('hierarchical', 0.151), ('targeted', 0.141), ('xml', 0.14), ('extensive', 0.127), ('processor', 0.119), ('robust', 0.119), ('around', 0.116), ('structures', 0.114), ('link', 0.112), ('achieve', 0.102), ('algorithms', 0.091), ('architectures', 0.09), ('operating', 0.087), ('implementation', 0.086), ('currently', 0.084), ('thousands', 0.083), ('node', 0.081), ('technologies', 0.081), ('data', 0.08), ('systems', 0.076), ('used', 0.075), ('nodes', 0.074), ('monitoring', 0.069), ('low', 0.069), ('computing', 0.063), ('uses', 0.062), ('handle', 0.058), ('design', 0.053), ('world', 0.052), ('set', 0.052), ('scalable', 0.048), ('across', 0.048), ('based', 0.044), ('storage', 0.044), ('distributed', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 101 high scalability-2007-09-27-Product: Ganglia Monitoring System
Introduction: Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
2 0.14938958 58 high scalability-2007-08-04-Product: Cacti
Introduction: Cacti is a network statistics graphing tool designed as a frontend to RRDtool's data storage and graphing functionality. It is intended to be intuitive and easy to use, as well as robust and scalable. It is generally used to graph time-series data like CPU load and bandwidth use. The frontend is written in PHP; it can handle multiple users, each with their own graph sets, so it is sometimes used by web hosting providers (especially dedicated server, virtual private server, and colocation providers) to display bandwidth statistics for their customers. It can be used to configure the data collection itself, allowing certain setups to be monitored without any manual configuration of RRDtool.
3 0.11937623 91 high scalability-2007-09-13-Design Preparations for Scaling
Introduction: Hi there, what do you think is crucial in the code designing of a scalable site? How does one prepare for webfarms and clusters (e.g. in PHP)? Thanks, Stephan
4 0.11327661 13 high scalability-2007-07-15-Lustre cluster file system
Introduction: Lustre速 is a scalable, secure, robust, highly-available cluster file system. It is designed, developed and maintained by Cluster File Systems, Inc. The central goal is the development of a next-generation cluster file system which can serve clusters with 10,000's of nodes, provide petabytes of storage, and move 100's of GB/sec with state-of-the-art security and management infrastructure. Lustre runs on many of the largest Linux clusters in the world, and is included by CFS's partners as a core component of their cluster offering (examples include HP StorageWorks SFS, and the Cray XT3 and XD1 supercomputers). Today's users have also demonstrated that Lustre scales down as well as it scales up, and runs in production on clusters as small as 4 and as large as 25,000 nodes. The latest version of Lustre is always available from Cluster File Systems, Inc. Public Open Source releases of Lustre are available under the GNU General Public License. These releases are found here, and are used
5 0.10011852 243 high scalability-2008-02-07-clusteradmin.blogspot.com - blog about building and administering clusters
Introduction: A blog about cluster administration. Written by a System Administrator working at HPC (High Performance Computing) data-center, mostly dealing with PC clusters (100s of servers), SMP machines and distributed installations. The blog concentrates on software/configuration/installation management systems, load balancers, monitoring and other cluster-related solutions.
7 0.090007268 1075 high scalability-2011-07-07-Myth: Google Uses Server Farms So You Should Too - Resurrection of the Big-Ass Machines
8 0.088463366 414 high scalability-2008-10-15-Hadoop - A Primer
9 0.080611318 1142 high scalability-2011-11-14-Using Gossip Protocols for Failure Detection, Monitoring, Messaging and Other Good Things
10 0.07950715 601 high scalability-2009-05-17-Product: Hadoop
11 0.078520998 1209 high scalability-2012-03-14-The Azure Outage: Time Is a SPOF, Leap Day Doubly So
12 0.077981673 371 high scalability-2008-08-24-A Scalable, Commodity Data Center Network Architecture
13 0.076866359 448 high scalability-2008-11-22-Google Architecture
14 0.076646179 72 high scalability-2007-08-22-Wikimedia architecture
15 0.075005807 1596 high scalability-2014-02-14-Stuff The Internet Says On Scalability For February 14th, 2014
16 0.074715592 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
17 0.073907845 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
18 0.073587812 801 high scalability-2010-03-30-Running Large Graph Algorithms - Evaluation of Current State-of-the-Art and Lessons Learned
19 0.070247293 558 high scalability-2009-04-06-How do you monitor the performance of your cluster?
20 0.068749644 1216 high scalability-2012-03-27-Big Data In the Cloud Using Cloudify
topicId topicWeight
[(0, 0.107), (1, 0.026), (2, 0.022), (3, 0.043), (4, -0.024), (5, 0.06), (6, 0.06), (7, -0.03), (8, -0.021), (9, 0.089), (10, 0.009), (11, 0.017), (12, 0.011), (13, -0.011), (14, 0.022), (15, 0.024), (16, 0.003), (17, 0.02), (18, 0.013), (19, 0.013), (20, 0.018), (21, 0.035), (22, -0.034), (23, 0.002), (24, -0.024), (25, -0.03), (26, 0.004), (27, 0.018), (28, -0.02), (29, -0.002), (30, -0.005), (31, 0.037), (32, -0.028), (33, 0.012), (34, -0.025), (35, 0.001), (36, 0.009), (37, -0.024), (38, -0.045), (39, 0.033), (40, -0.023), (41, -0.029), (42, -0.007), (43, 0.049), (44, 0.013), (45, 0.051), (46, -0.046), (47, 0.01), (48, -0.02), (49, -0.007)]
simIndex simValue blogId blogTitle
same-blog 1 0.92659491 101 high scalability-2007-09-27-Product: Ganglia Monitoring System
Introduction: Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
Introduction: Much of the focus of high performance computing (HPC) has centered on CPU performance. However, as computing requirements grow, HPC clusters are demanding higher rates of aggregate data throughput. Today's clusters feature larger numbers of nodes with increased compute speeds. The higher clock rates and operations per clock cycle create increased demand for local data on each node. In addition, InfiniBand and other high-speed, low-latency interconnects increase the data throughput available to each node. Traditional shared file systems such as NFS have not been able to scale to meet this growing demand for data throughput on HPC clusters. Scalable cluster file systems that can provide parallel data access to hundreds of nodes and petabytes of storage are needed to provide the high data throughput required by large HPC applications, including manufacturing, electronic design, and research. This paper describes an implementation of the Sun Lustre file system as a scalable storage
3 0.71930325 979 high scalability-2011-01-27-Comet - An Example of the New Key-Code Databases
Introduction: Comet is an active distributed key-value store built at the University of Washington. The paper describing Comet is Comet: An active distributed key-value store , there are also slides , and a MP3 of a presentation given at OSDI '10 . Here's a succinct overview of Comet : Today's cloud storage services, such as Amazon S3 or peer-to-peer DHTs, are highly inflexible and impose a variety of constraints on their clients: specific replication and consistency schemes, fixed data timeouts, limited logging, etc. We witnessed such inflexibility first-hand as part of our Vanish work, where we used a DHT to store encryption keys temporarily. To address this issue, we built Comet, an extensible storage service that allows clients to inject snippets of code that control their data's behavior inside the storage service. I found this paper quite interesting because it takes the initial steps of collocating code with a key-value store, which turns it into what might called a key-code
4 0.70688355 756 high scalability-2009-12-30-Terrastore - Scalable, elastic, consistent document store.
Introduction: Terrastore is a new-born document store which provides advanced scalability and elasticity features without sacrificing consistency. Here are a few highlights: Ubiquitous: based on the universally supported HTTP protocol. Distributed: nodes can run and live everywhere on your network. Elastic: you can add and remove nodes dynamically to/from your running cluster with no downtime and no changes at all to your configuration. Scalable at the data layer: documents are partitioned and distributed among your nodes, with automatic and transparent re-balancing when nodes join and leave. Scalable at the computational layer: query and update operations are distributed to the nodes which actually holds the queried/updated data, minimizing network traffic and spreading computational load. Consistent: providing per-document consistency, you're guaranteed to always get the latest value of a single document, with read committed isolation for concurrent modifications. Schemales
Introduction: When building a system on top of a set of wildly uncooperative and unruly computers you have knowledge problems: knowing when other nodes are dead; knowing when nodes become alive; getting information about other nodes so you can make local decisions, like knowing which node should handle a request based on a scheme for assigning nodes to a certain range of users; learning about new configuration data; agreeing on data values; and so on. How do you solve these problems? A common centralized approach is to use a database and all nodes query it for information. Obvious availability and performance issues for large distributed clusters. Another approach is to use Paxos , a protocol for solving consensus in a network to maintain strict consistency requirements for small groups of unreliable processes. Not practical when larger number of nodes are involved. So what's the super cool decentralized way to bring order to large clusters? Gossip protocols , which maintain relaxed consi
6 0.69537705 368 high scalability-2008-08-17-Wuala - P2P Online Storage Cloud
7 0.69481146 25 high scalability-2007-07-25-Paper: Designing Disaster Tolerant High Availability Clusters
8 0.68463707 13 high scalability-2007-07-15-Lustre cluster file system
9 0.68346119 850 high scalability-2010-06-30-Paper: GraphLab: A New Framework For Parallel Machine Learning
10 0.68302542 705 high scalability-2009-09-16-Paper: A practical scalable distributed B-tree
11 0.68131232 1483 high scalability-2013-06-27-Paper: XORing Elephants: Novel Erasure Codes for Big Data
12 0.67570359 651 high scalability-2009-07-02-Product: Project Voldemort - A Distributed Database
13 0.67376167 839 high scalability-2010-06-09-Paper: Propagation Networks: A Flexible and Expressive Substrate for Computation
14 0.67215568 882 high scalability-2010-08-18-Misco: A MapReduce Framework for Mobile Systems - Start of the Ambient Cloud?
15 0.6633004 433 high scalability-2008-10-29-CTL - Distributed Control Dispatching Framework
17 0.66251737 1338 high scalability-2012-10-11-RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store
18 0.65987128 421 high scalability-2008-10-17-A High Performance Memory Database for Web Application Caches
19 0.65039384 1138 high scalability-2011-11-07-10 Core Architecture Pattern Variations for Achieving Scalability
20 0.64501715 326 high scalability-2008-05-25-Product: Condor - Compute Intensive Workload Management
topicId topicWeight
[(1, 0.097), (2, 0.12), (35, 0.29), (40, 0.018), (77, 0.058), (79, 0.194), (94, 0.094)]
simIndex simValue blogId blogTitle
1 0.8414064 740 high scalability-2009-11-11-Hot Scalability Links for Nov 11 2009
Introduction: The Cost of Latency by James Hamilton. James summarizes latency info from Steve Souder , Greg Linden , and Marissa Mayer . Speed [is] an undervalued and under-discussed asset on the web. Dynamo - Part I: a followup and re-rebuttals . Dynamo under attack as having Design flaws and the resounding rebuttal in response. Programming Bits and Atoms . Thinking about programming and scaling as a problem in physics. Absolutely fascinating and inspiring. Scaling Servers with the Cloud: Amazon S3 . Build a static site using S3 for pennies. An oldly but still a goody idea. Are Wireless Road Trains the Cure for Traffic Congestion? The concept of road trains--up to eight vehicles zooming down the road together--has long been considered a faster, safer, and greener way of traveling long distances by car. Erlang at Facebook by Eugene Letuchy. How Facebook uses Erlang to implement Chat, AIM Presence, and Chat Jabber support. Yahoo Open Sources Traffic Server . Traffic Serv
same-blog 2 0.83620906 101 high scalability-2007-09-27-Product: Ganglia Monitoring System
Introduction: Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
3 0.71335006 440 high scalability-2008-11-11-Arhcitecture for content management
Introduction: Hi, I am looking for logical architecture of content management of portal. Say an org has got lot of business process and integrates with few applicaitons and it is portal based application. How does it look to have architecture framework for this type of fucntionality.
4 0.69135761 1647 high scalability-2014-05-14-Google Says Cloud Prices Will Follow Moore’s Law: Are We All Renters Now?
Introduction: After Google cut prices on their Google Cloud Platform Amazon quickly followed with their own price cuts . Even more interesting is what the future holds for pricing. The near future looks great. After that? We'll see. Adrian Cockcroft highlights that Google thinks prices should follow Moore’s law, which means we should expect prices to halve every 18-24 months. That's good news. Greater cost certainty means you can make much more aggressive build out plans. With the savings you can hire more people, handle more customers, and add those media rich features you thought you couldn't afford. Design is directly related to costs. Without Google competing with Amazon there's little doubt the price reduction curve would be much less favorable. As a late cloud entrant Google is now in a customer acquisition phase, so they are willing to pay for customers, which means lower prices are an acceptable cost of doing business. Profit and high margins are not the objective. Getting marke
Introduction: Many enterprises' high-availability architecture is based on the assumption that you can prevent failure from happening by putting all your critical data in a centralized database, back it up with expensive storage, and replicate it somehow between the sites. As I argued in one of my previous posts ( Why Existing Databases (RAC) are So Breakable! ) many of those assumptions are broken at their core, as storage is doomed to failure just like any other device, expensive hardware doesn’t make things any better and database replication is often not enough. One of the main lessons that we can take from the likes of Amazon and Google is that the right way to ensure continuous high availability is by designing our system to cope with failure. We need to assume that what we tend to think of as unthinkable will probably happen, as that’s the nature of failure. So rather than trying to prevent failures, we need to build a system that will tolerate them. As we can learn from a recent outage
6 0.63888752 539 high scalability-2009-03-16-Books: Web 2.0 Architectures and Cloud Application Architectures
7 0.6320433 1403 high scalability-2013-02-08-Stuff The Internet Says On Scalability For February 8, 2013
8 0.63154465 1420 high scalability-2013-03-08-Stuff The Internet Says On Scalability For March 8, 2013
9 0.63154435 680 high scalability-2009-08-13-Reconnoiter - Large-Scale Trending and Fault-Detection
10 0.63143915 448 high scalability-2008-11-22-Google Architecture
11 0.62821668 706 high scalability-2009-09-16-The VeriScale Architecture - Elasticity and efficiency for private clouds
12 0.62672311 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
13 0.62254673 650 high scalability-2009-07-02-Product: Hbase
14 0.62238199 1494 high scalability-2013-07-19-Stuff The Internet Says On Scalability For July 19, 2013
15 0.62103599 871 high scalability-2010-08-04-Dremel: Interactive Analysis of Web-Scale Datasets - Data as a Programming Paradigm
16 0.62067407 581 high scalability-2009-04-26-Map-Reduce for Machine Learning on Multicore
17 0.6180042 995 high scalability-2011-02-24-Strategy: Eliminate Unnecessary SQL
18 0.61751139 409 high scalability-2008-10-13-Challenges from large scale computing at Google
19 0.61578703 1048 high scalability-2011-05-27-Stuff The Internet Says On Scalability For May 27, 2011
20 0.6140452 1150 high scalability-2011-12-02-Stuff The Internet Says On Scalability For December 2, 2011