high_scalability high_scalability-2010 high_scalability-2010-820 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Deploying, running and monitoring application on a big cluster is a challenging task. Recently Hazelcast team deployed a demo application on Amazon EC2 platform to show how Hazelcast p2p cluster scales and screen recorded the entire process from deployment to monitoring. Hazelcast is open source (Apache License), transactional, distributed caching solution for Java. It is a little more than a cache though as it provides distributed implementation of map, multimap, queue, topic, lock and executor service. Details of running 100 node Hazelcast cluster on Amazon EC2 can be found here . Make sure to watch the screencast !
sentIndex sentText sentNum sentScore
1 Deploying, running and monitoring application on a big cluster is a challenging task. [sent-1, score-0.718]
2 Recently Hazelcast team deployed a demo application on Amazon EC2 platform to show how Hazelcast p2p cluster scales and screen recorded the entire process from deployment to monitoring. [sent-2, score-1.592]
3 Hazelcast is open source (Apache License), transactional, distributed caching solution for Java. [sent-3, score-0.359]
4 It is a little more than a cache though as it provides distributed implementation of map, multimap, queue, topic, lock and executor service. [sent-4, score-0.934]
5 Â Details of running 100 node Hazelcast cluster on Amazon EC2 can be found here . [sent-5, score-0.433]
wordName wordTfidf (topN-words)
[('hazelcast', 0.534), ('executor', 0.29), ('screencast', 0.267), ('cluster', 0.223), ('recorded', 0.211), ('demo', 0.201), ('screen', 0.201), ('license', 0.195), ('challenging', 0.16), ('topic', 0.143), ('deploying', 0.138), ('transactional', 0.137), ('lock', 0.136), ('map', 0.134), ('amazon', 0.133), ('deployed', 0.131), ('running', 0.116), ('queue', 0.111), ('scales', 0.109), ('deployment', 0.106), ('apache', 0.106), ('show', 0.1), ('implementation', 0.1), ('distributed', 0.094), ('node', 0.094), ('details', 0.093), ('though', 0.092), ('sure', 0.089), ('application', 0.085), ('platform', 0.083), ('monitoring', 0.08), ('entire', 0.079), ('provides', 0.077), ('little', 0.074), ('caching', 0.072), ('cache', 0.071), ('solution', 0.068), ('source', 0.064), ('process', 0.063), ('open', 0.061), ('big', 0.054), ('make', 0.038)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 820 high scalability-2010-05-03-100 Node Hazelcast cluster on Amazon EC2
Introduction: Deploying, running and monitoring application on a big cluster is a challenging task. Recently Hazelcast team deployed a demo application on Amazon EC2 platform to show how Hazelcast p2p cluster scales and screen recorded the entire process from deployment to monitoring. Hazelcast is open source (Apache License), transactional, distributed caching solution for Java. It is a little more than a cache though as it provides distributed implementation of map, multimap, queue, topic, lock and executor service. Details of running 100 node Hazelcast cluster on Amazon EC2 can be found here . Make sure to watch the screencast !
2 0.38197562 1020 high scalability-2011-04-12-Caching and Processing 2TB Mozilla Crash Reports in memory with Hazelcast
Introduction: Mozilla processes TB's of Firefox crash reports daily using HBase, Hadoop, Python and Thrift protocol. The project is called Socorro , a system for collecting, processing, and displaying crash reports from clients. Today the Socorro application stores about 2.6 million crash reports per day. During peak traffic, it receives about 2.5K crashes per minute. In this article we are going to demonstrate a proof of concept showing how Mozilla could integrate Hazelcast into Socorro and achieve caching and processing 2TB of crash reports with 50 node Hazelcast cluster. The video for the demo is available here . Currently, Socorro has pythonic collectors, processors, and middleware that communicate with HBase via the Thrift protocol. One of the biggest limitations of the current architecture is that it is very sensitive to latency or outages on the HBase side. If the collectors cannot store an item in HBase then they will store it on local disk and it will not be accessible to th
3 0.3498008 1221 high scalability-2012-04-03-Hazelcast 2.0: Big Data In-Memory
Introduction: As it is said in the recent article "Google: Taming the Long Latency Tail - When More Machines Equals Worse Results" , latency variability has greater impact in larger scale clusters where a typical request is composed of multiple distributed/parallel requests. The overall response time dramatically decreases if latency of each request is not consistent and low. In dynamically scalable partitioned storage systems, whether it is a NoSQL database, filesystem or in-memory data grid, changes in the cluster (adding or removing a node) can lead to big data moves in the network to re-balance the cluster. Re-balancing will be needed for both primary and backup data on those nodes. If a node crashes for example, dead node’s data has to be re-owned (become primary) by other node(s) and also its backup has to be taken immediately to be fail-safe again. Shuffling MBs of data around has a negative effect in the cluster as it consumes your valuable resources such as network, CPU and RAM. It mig
4 0.11382492 254 high scalability-2008-02-19-Hadoop Getting Closer to 1.0 Release
Introduction: Update: Yahoo! Launches World's Largest Hadoop Production Application . A 10,000 core Hadoop cluster produces data used in every Yahoo! Web search query. Raw disk is at 5 Petabytes. Their previous 1 petabyte database couldn't handle the load and couldn't grow larger. Greg Linden thinks the Google cluster has way over 133,000 machines. From an InfoQ interview with project lead Doug Cutting, it appears Hadoop , an open source distributed computing platform, is making good progress towards their 1.0 release. They've successfully reached a 1000 node cluster size, improved file system integrity, and jacked performance by 20x in the last year. How they are making progress could be a good model for anyone: The speedup has been an aggregation of our work in the past few years, and has been accomplished mostly by trial-and-error. We get things running smoothly on a cluster of a given size, then double the size of the cluster and see what breaks. We aim for performan
5 0.089665867 886 high scalability-2010-08-24-21 Quality Screencasts on Scaling Rails
Introduction: This a follow-up post to an earlier post on the Scaling Rails Screencast Series by Gregg Pollack , when there were only 13 screencasts, now there are 21. Eight more have been added on topics like load testing and database scaling. This series is of surprisingly high quality. While I didn't view every screencast, I sampled a large set and found them to have solid content and high production values. In fact, how did they make these things? The instructor moves around in a little box while the content flows around him. A very cool effect. But that wouldn't matter if the content didn't deliver, here's what's new: Episode #14 - Rack & Metal Episode #15 - Load Testing - Part 1 Episode #16 - Load Testing - Part 2 Episode #17 - Scaling Your Database - Part 1 Episode #18 - Scaling Your Database - Part 2 Episode #19 - On The Edge - Part 1 Episode #20 - On The Edge - Part 2 Episode #21 - On The Edge - Part 3 The courses while targeted at Rails are more generally applica
6 0.08333043 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
7 0.078532122 448 high scalability-2008-11-22-Google Architecture
9 0.075441882 821 high scalability-2010-05-03-MocoSpace Architecture - 3 Billion Mobile Page Views a Month
10 0.075025044 454 high scalability-2008-12-01-Deploying MySQL Database in Solaris Cluster Environments
12 0.073481716 480 high scalability-2008-12-30-Scalability Perspectives #5: Werner Vogels – The Amazon Technology Platform
13 0.072094746 1508 high scalability-2013-08-28-Sean Hull's 20 Biggest Bottlenecks that Reduce and Slow Down Scalability
14 0.071335718 13 high scalability-2007-07-15-Lustre cluster file system
15 0.070503756 1160 high scalability-2011-12-21-In Memory Data Grid Technologies
16 0.068625696 1155 high scalability-2011-12-12-Netflix: Developing, Deploying, and Supporting Software According to the Way of the Cloud
17 0.066407338 1445 high scalability-2013-04-24-Strategy: Using Lots of RAM Often Cheaper than Using a Hadoop Cluster
18 0.065569237 1042 high scalability-2011-05-17-Facebook: An Example Canonical Architecture for Scaling Billions of Messages
19 0.065001294 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
20 0.064574689 674 high scalability-2009-08-07-The Canonical Cloud Architecture
topicId topicWeight
[(0, 0.108), (1, 0.03), (2, -0.009), (3, 0.012), (4, -0.017), (5, 0.027), (6, 0.07), (7, -0.057), (8, 0.011), (9, 0.023), (10, 0.005), (11, 0.015), (12, 0.05), (13, -0.037), (14, -0.052), (15, -0.006), (16, -0.01), (17, -0.042), (18, -0.005), (19, -0.014), (20, -0.023), (21, 0.049), (22, 0.034), (23, 0.044), (24, -0.023), (25, 0.012), (26, 0.061), (27, -0.022), (28, 0.003), (29, 0.03), (30, 0.015), (31, 0.022), (32, 0.014), (33, -0.055), (34, 0.032), (35, 0.007), (36, -0.031), (37, -0.09), (38, -0.011), (39, -0.001), (40, 0.033), (41, 0.004), (42, 0.03), (43, 0.067), (44, -0.005), (45, 0.107), (46, 0.063), (47, -0.016), (48, -0.034), (49, -0.029)]
simIndex simValue blogId blogTitle
same-blog 1 0.95947838 820 high scalability-2010-05-03-100 Node Hazelcast cluster on Amazon EC2
Introduction: Deploying, running and monitoring application on a big cluster is a challenging task. Recently Hazelcast team deployed a demo application on Amazon EC2 platform to show how Hazelcast p2p cluster scales and screen recorded the entire process from deployment to monitoring. Hazelcast is open source (Apache License), transactional, distributed caching solution for Java. It is a little more than a cache though as it provides distributed implementation of map, multimap, queue, topic, lock and executor service. Details of running 100 node Hazelcast cluster on Amazon EC2 can be found here . Make sure to watch the screencast !
2 0.72906494 1020 high scalability-2011-04-12-Caching and Processing 2TB Mozilla Crash Reports in memory with Hazelcast
Introduction: Mozilla processes TB's of Firefox crash reports daily using HBase, Hadoop, Python and Thrift protocol. The project is called Socorro , a system for collecting, processing, and displaying crash reports from clients. Today the Socorro application stores about 2.6 million crash reports per day. During peak traffic, it receives about 2.5K crashes per minute. In this article we are going to demonstrate a proof of concept showing how Mozilla could integrate Hazelcast into Socorro and achieve caching and processing 2TB of crash reports with 50 node Hazelcast cluster. The video for the demo is available here . Currently, Socorro has pythonic collectors, processors, and middleware that communicate with HBase via the Thrift protocol. One of the biggest limitations of the current architecture is that it is very sensitive to latency or outages on the HBase side. If the collectors cannot store an item in HBase then they will store it on local disk and it will not be accessible to th
3 0.67975271 1221 high scalability-2012-04-03-Hazelcast 2.0: Big Data In-Memory
Introduction: As it is said in the recent article "Google: Taming the Long Latency Tail - When More Machines Equals Worse Results" , latency variability has greater impact in larger scale clusters where a typical request is composed of multiple distributed/parallel requests. The overall response time dramatically decreases if latency of each request is not consistent and low. In dynamically scalable partitioned storage systems, whether it is a NoSQL database, filesystem or in-memory data grid, changes in the cluster (adding or removing a node) can lead to big data moves in the network to re-balance the cluster. Re-balancing will be needed for both primary and backup data on those nodes. If a node crashes for example, dead node’s data has to be re-owned (become primary) by other node(s) and also its backup has to be taken immediately to be fail-safe again. Shuffling MBs of data around has a negative effect in the cluster as it consumes your valuable resources such as network, CPU and RAM. It mig
4 0.62302661 114 high scalability-2007-10-07-Product: Wackamole
Introduction: Wackamole is an application that helps with making a cluster highly available. It manages a bunch of virtual IPs, that should be available to the outside world at all times. Wackamole ensures that a single machine within a cluster is listening on each virtual IP address that Wackamole manages. If it discovers that particular machines within the cluster are not alive, it will almost immediately ensure that other machines acquire these public IPs. At no time will more than one machine listen on any virtual IP. Wackamole also works toward achieving a balanced distribution of number IPs on the machine within the cluster it manages. There is no other software like Wackamole. Wackamole is quite unique in that it operates in a completely peer-to-peer mode within the cluster. Other products that provide the same high-availability guarantees use a "VIP" method. Wackamole is an application that runs as root in a cluster to make it highly available. It uses the membership notifications prov
5 0.59067047 364 high scalability-2008-08-14-Product: Terracotta - Open Source Network-Attached Memory
Introduction: Update: Evaluating Terracotta by Piotr Woloszyn. Nice writeup that covers resilience, failover, DB persistence, Distributed caching implementation, OS/Platform restrictions, Ease of implementation, Hardware requirements, Performance, Support package, Code stability, partitioning, Transactional, Replication and consistency. Terracotta is Network Attached Memory (NAM) for Java VMs. It provides up to a terabyte of virtual heap for Java applications that spans hundreds of connected JVMs. NAM is best suited for storing what they call scratch data. Scratch data is defined as object oriented data that is critical to the execution of a series of Java operations inside the JVM, but may not be critical once a business transaction is complete. The Terracotta Architecture has three components: Client Nodes - Each client node corresponds to a client node in the cluster which runs on a standard JVM Server Cluster - java process that provides the clustering intelligence. Th
6 0.58660358 25 high scalability-2007-07-25-Paper: Designing Disaster Tolerant High Availability Clusters
8 0.55616552 13 high scalability-2007-07-15-Lustre cluster file system
9 0.54363823 1118 high scalability-2011-09-19-Big Iron Returns with BigMemory
10 0.54072714 254 high scalability-2008-02-19-Hadoop Getting Closer to 1.0 Release
11 0.53778046 696 high scalability-2009-09-07-Product: Infinispan - Open Source Data Grid
12 0.53566772 597 high scalability-2009-05-12-GemStone Unveils GemFire Enterprise 6.0
13 0.53534806 1138 high scalability-2011-11-07-10 Core Architecture Pattern Variations for Achieving Scalability
14 0.52950919 795 high scalability-2010-03-16-1 Billion Reasons Why Adobe Chose HBase
15 0.52115369 414 high scalability-2008-10-15-Hadoop - A Primer
16 0.51953733 423 high scalability-2008-10-19-Alternatives to Google App Engine
17 0.51281077 968 high scalability-2011-01-04-Map-Reduce With Ruby Using Hadoop
18 0.50662196 459 high scalability-2008-12-03-Java World Interview on Scalability and Other Java Scalability Secrets
20 0.50142497 862 high scalability-2010-07-20-Strategy: Consider When a Service Starts Billing in Your Algorithm Cost
topicId topicWeight
[(1, 0.01), (2, 0.188), (79, 0.084), (85, 0.568)]
simIndex simValue blogId blogTitle
1 0.93715692 59 high scalability-2007-08-04-Try Squid as a Reverse Proxy
Introduction: This scalability strategy is brought to you by Erik Osterman: My recommendations for anyone dealing with explosive growth on a limited budget with lots of cachable content (e.g. content capable of returning valid expiration headers) is employ a reverse proxy as mentioned in this article. In the last week, we had a site get AP'd, triggering 100K unique visitors to a single IIS server in under 5 hours. It took out the IIS server. Placing a single squid infront of the server handled the entire onslaught with a max server load of 0.10 on a modest Intel IV 3Ghz. It's trivial to implement for anyone interested...
2 0.92365694 1049 high scalability-2011-05-31-Awesome List of Advanced Distributed Systems Papers
Introduction: As part of Dr. Indranil Gupta 's CS 525 Spring 2011 Advanced Distributed Systems class, he has collected an incredible list of resources on distributed systems . His research group is also doing some interesting work. The various topics include: Before there Were Clouds, Cloud Computing, P2P Systems, Basic Distributed Computing Concepts, Sensor Networks, Overlays and DHTs, Cloud Programming, Cloud Scheduling, Key-Value Stores, Storage, Sensor Net Routing, Geo-Distribution, P2P Apps, In-network processing, Epidemics, Probabilistic Membership Protocols, Distributed Monitoring and Management, Publish-Subscribe/CDNs, Measurement Studies, Old Wine: Stale or Vintage?, In Byzantium, Cloud Pricing, Other Industrial Systems, Structure of Networks, Completing the Circle, Green Clouds, Distributed Debugging, Flash!, The Middle or the End?, Availability-Aware Systems, Design Methodologies, Handling Stress, Sources of unreliability in networks, Handling Stress, Selfish algorithms, Securi
3 0.89706993 191 high scalability-2007-12-23-Synchronizing Memcached application
Introduction: I have an application with couple of web servers that uses MemcacheD. How can i synchronize concurrent put to the cache? The value of the entry is list. Atomic append operation could have been helpful, but unfortunately memcahe doesn't support atomic append.
4 0.87873852 143 high scalability-2007-11-06-Product: ChironFS
Introduction: If you are trying to create highly available file systems, especially across data centers, then ChironFS is one potential solution. It's relatively new, so there aren't lots of experience reports, but it looks worth considering. What is ChironFS and how does it work? Adapted from the ChironFS website: The Chiron Filesystem is a Fuse based filesystem that frees you from single points of failure. It's main purpose is to guarantee filesystem availability using replication. But it isn't a RAID implementation. RAID replicates DEVICES not FILESYSTEMS. Why not just use RAID over some network block device? Because it is a block device and if one server mounts that device in RW mode, no other server will be able to mount it in RW mode. Any real network may have many servers and offer a variety of services. Keeping everything running can become a real nightmare!
same-blog 5 0.8603878 820 high scalability-2010-05-03-100 Node Hazelcast cluster on Amazon EC2
Introduction: Deploying, running and monitoring application on a big cluster is a challenging task. Recently Hazelcast team deployed a demo application on Amazon EC2 platform to show how Hazelcast p2p cluster scales and screen recorded the entire process from deployment to monitoring. Hazelcast is open source (Apache License), transactional, distributed caching solution for Java. It is a little more than a cache though as it provides distributed implementation of map, multimap, queue, topic, lock and executor service. Details of running 100 node Hazelcast cluster on Amazon EC2 can be found here . Make sure to watch the screencast !
6 0.84741867 447 high scalability-2008-11-19-High Definition Video Delivery on the Web?
7 0.8231914 102 high scalability-2007-09-27-Product: Sequoia Database Clustering Technology
8 0.80624044 1039 high scalability-2011-05-12-Paper: Mind the Gap: Reconnecting Architecture and OS Research
9 0.78369808 1032 high scalability-2011-05-02-Stack Overflow Makes Slow Pages 100x Faster by Simple SQL Tuning
10 0.77308786 1164 high scalability-2011-12-27-PlentyOfFish Update - 6 Billion Pageviews and 32 Billion Images a Month
11 0.74953985 1577 high scalability-2014-01-13-NYTimes Architecture: No Head, No Master, No Single Point of Failure
12 0.74557447 1500 high scalability-2013-08-12-100 Curse Free Lessons from Gordon Ramsay on Building Great Software
13 0.74058795 1239 high scalability-2012-05-04-Stuff The Internet Says On Scalability For May 4, 2012
14 0.71633857 492 high scalability-2009-01-16-Database Sharding for startups
15 0.69327843 118 high scalability-2007-10-09-High Load on production Webservers after Sourcecode sync
16 0.68191111 1024 high scalability-2011-04-15-Stuff The Internet Says On Scalability For April 15, 2011
17 0.67427397 1592 high scalability-2014-02-07-Stuff The Internet Says On Scalability For February 7th, 2014
18 0.67112124 638 high scalability-2009-06-26-PlentyOfFish Architecture
19 0.66947985 646 high scalability-2009-07-01-Podcast about Facebook's Cassandra Project and the New Wave of Distributed Databases
20 0.6693576 1361 high scalability-2012-11-22-Gone Fishin': PlentyOfFish Architecture