high_scalability high_scalability-2011 high_scalability-2011-1160 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: After winning a CSC Leading Edge Forum (LEF) research grant, I (Paul Colmer) wanted to publish some of the highlights of my research to share with the wider technology community. What is an In Memory Data Grid? It is not an in-memory relational database, a NOSQL database or a relational database. It is a different breed of software datastore. In summary an IMDG is an ‘off the shelf’ software product that exhibits the following characteristics: The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. All servers can be active in each site. All data is stored in the RAM of the servers. Servers can be added or removed non-disruptively, to increase the amount of RAM available. The data model is non-relational and is object-based. Distributed applications written on the .NET and Java application platforms are s
sentIndex sentText sentNum sentScore
1 It is not an in-memory relational database, a NOSQL database or a relational database. [sent-3, score-0.238]
2 In summary an IMDG is an ‘off the shelf’ software product that exhibits the following characteristics: The data model is distributed across many servers in a single location or across multiple locations. [sent-5, score-0.384]
3 The data model is non-relational and is object-based. [sent-11, score-0.213]
4 I use the term in-memory data grid appliance to describe this group of products and these were excluded from my research. [sent-16, score-0.381]
5 Net) And here are the rest of products available in the market now, that I consider IMDGs: IBM eXtreme Scale Terracotta Enterprise Suite Jboss (Redhat) Infinispan Relative newcomers to this space, and worthy of watching closely are Microsoft and Tibco. [sent-19, score-0.25]
6 The data model and application code are inextricably linked. [sent-25, score-0.213]
7 How does an In Memory Data Grid map to real business benefits? [sent-29, score-0.182]
8 Safety – businesses can improve the quality of their decision-making. [sent-31, score-0.21]
9 Productivity – improved business process efficiency reduces waster and likely to improve profitability. [sent-32, score-0.293]
10 Improved Customer Experience – provides the basis for a faster, reliable web service which is a strong differentiator in the online business sector. [sent-33, score-0.182]
11 Install the IMDG software on all the servers and choose the appropriate topology for the product. [sent-37, score-0.192]
12 For multi-site operations I always recommend a partitioned and replicated cache. [sent-38, score-0.238]
13 Develop your data model and the business logic around the model. [sent-40, score-0.395]
14 With a partitioned and replicated cache, you simply partition the cache on the servers that best suits the business needs to trying to fulfil, and the replicated part ensures there are sufficient copies across all the servers. [sent-41, score-0.734]
15 The key here is to design a topology that mitigates all business risk, so that if a server or a site is inoperable, the service keeps running seamlessly in the background. [sent-44, score-0.364]
16 There are also some tough decisions you may need to make regarding data consistency vs performance. [sent-45, score-0.191]
17 You can trade the performance to improve data consistency and vice versa. [sent-46, score-0.221]
18 Online Retailer: Providing a highly available, easily maintainable and scalable solution for 3+ million visitors per month in the online card retailer market. [sent-51, score-0.242]
19 Aviation: Three-site active / active / active flight booking system for a major European budget-airline carrier. [sent-52, score-0.67]
20 About the Author: Paul Colmer is a technology consultant working for CSC and director and active professional musician for Music4Film. [sent-55, score-0.25]
wordName wordTfidf (topN-words)
[('alachisoft', 0.203), ('colmer', 0.203), ('csc', 0.203), ('ncache', 0.203), ('grid', 0.193), ('business', 0.182), ('retailer', 0.173), ('active', 0.171), ('replicated', 0.153), ('imdg', 0.146), ('gemfire', 0.146), ('memory', 0.125), ('relational', 0.119), ('java', 0.116), ('topology', 0.113), ('vmware', 0.112), ('improve', 0.111), ('data', 0.11), ('paul', 0.107), ('model', 0.103), ('market', 0.103), ('businesses', 0.099), ('exhibits', 0.092), ('brisbane', 0.092), ('dublin', 0.092), ('fulfil', 0.092), ('imdgs', 0.092), ('booking', 0.086), ('partitioned', 0.085), ('suits', 0.082), ('decisions', 0.081), ('proven', 0.081), ('consultant', 0.079), ('profitability', 0.079), ('infinispan', 0.079), ('hazelcast', 0.079), ('servers', 0.079), ('products', 0.078), ('ram', 0.075), ('research', 0.071), ('flight', 0.071), ('redhat', 0.071), ('known', 0.071), ('competitor', 0.07), ('grant', 0.07), ('worthy', 0.069), ('resiliency', 0.069), ('maintainable', 0.069), ('mitigates', 0.069), ('xap', 0.067)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000004 1160 high scalability-2011-12-21-In Memory Data Grid Technologies
Introduction: After winning a CSC Leading Edge Forum (LEF) research grant, I (Paul Colmer) wanted to publish some of the highlights of my research to share with the wider technology community. What is an In Memory Data Grid? It is not an in-memory relational database, a NOSQL database or a relational database. It is a different breed of software datastore. In summary an IMDG is an ‘off the shelf’ software product that exhibits the following characteristics: The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. All servers can be active in each site. All data is stored in the RAM of the servers. Servers can be added or removed non-disruptively, to increase the amount of RAM available. The data model is non-relational and is object-based. Distributed applications written on the .NET and Java application platforms are s
2 0.20126547 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
Introduction: We are on the edge of two potent technological changes: Clouds and Memory Based Architectures. This evolution will rip open a chasm where new players can enter and prosper. Google is the master of disk. You can't beat them at a game they perfected. Disk based databases like SimpleDB and BigTable are complicated beasts, typical last gasp products of any aging technology before a change. The next era is the age of Memory and Cloud which will allow for new players to succeed. The tipping point will be soon. Let's take a short trip down web architecture lane: It's 1993: Yahoo runs on FreeBSD, Apache, Perl scripts and a SQL database It's 1995: Scale-up the database. It's 1998: LAMP It's 1999: Stateless + Load Balanced + Database + SAN It's 2001: In-memory data-grid. It's 2003: Add a caching layer. It's 2004: Add scale-out and partitioning. It's 2005: Add asynchronous job scheduling and maybe a distributed file system. It's 2007: Move it all into the cloud. It's 2008: C
Introduction: Who's Hiring? Are you looking for people? Fun and Informative Events Sign up for this free 30-minute webinar exploring how new technology can determine which ads have been seen by users and will discuss the C3 Metrics Labs analysis of over 2 billion impressions. Cool Products and Services Reality Check Network offers powerful hosting solutions and managed servers for high traffic/bandwidth websites backed by unlimited network, server and application support. When you’re looking for the fastest, lightest, most complete toolset for rapidly building high performance Web 2.0 applications, you want NetAdvantage for ASP.NET . Create your most stunning, highly performant, and completely mobile HTML5 applications and dashboards on any browser, platform or device – only with NetAdvantage for jQuery . Take your application to the next level of performance & scalability with the GigaSpaces In-Memory Data Grid (IMDG) . aiCache creates a better user e
Introduction: Who's Hiring? Are you looking for people? Fun and Informative Events Sign up for this free 30-minute webinar exploring how new technology can determine which ads have been seen by users and will discuss the C3 Metrics Labs analysis of over 2 billion impressions. Cool Products and Services Reality Check Network offers powerful hosting solutions and managed servers for high traffic/bandwidth websites backed by unlimited network, server and application support. When you’re looking for the fastest, lightest, most complete toolset for rapidly building high performance Web 2.0 applications, you want NetAdvantage for ASP.NET . Create your most stunning, highly performant, and completely mobile HTML5 applications and dashboards on any browser, platform or device – only with NetAdvantage for jQuery . Take your application to the next level of performance & scalability with the GigaSpaces In-Memory Data Grid (IMDG) . aiCache creates a better user e
5 0.1720514 633 high scalability-2009-06-19-GemFire 6.0: New innovations in data management
Introduction: GemStone has unveiled GemFire 6.0 which is the culmination of several years of development and the continuous solving of the hardest data management problems in the world. With this release GemFire touts some of the latest innovative features in data management. In this release: - GemFire introduces a resource manager to continuously monitor and protect cache instances from running out of memory, triggering rebalancing to migrate data to less loaded nodes or allow dynamic increase/decrease in the number of nodes hosting data for linear scalability without impeding ongoing operations (no contention points). - GemFire provides explicit control over when rebalancing can be triggered, on what class of data and even allows the administrator to simulate a "rebalance" operation to quantify the benefits before actually doing it. - With built in instrumentation that captures throughput and latency metrics, GemFire now enables applications to sense changing performance patterns and proactiv
6 0.16289565 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
7 0.15947059 542 high scalability-2009-03-17-IBM WebSphere eXtreme Scale (IMDG)
8 0.15128551 373 high scalability-2008-08-29-Product: ScaleOut StateServer is Memcached on Steroids
17 0.13691238 1118 high scalability-2011-09-19-Big Iron Returns with BigMemory
20 0.13486651 696 high scalability-2009-09-07-Product: Infinispan - Open Source Data Grid
topicId topicWeight
[(0, 0.247), (1, 0.013), (2, 0.047), (3, -0.012), (4, -0.053), (5, 0.106), (6, 0.001), (7, -0.087), (8, -0.072), (9, 0.057), (10, -0.006), (11, 0.042), (12, 0.002), (13, 0.063), (14, 0.025), (15, 0.001), (16, 0.061), (17, -0.016), (18, 0.025), (19, -0.011), (20, -0.08), (21, 0.011), (22, 0.089), (23, 0.01), (24, -0.039), (25, -0.088), (26, -0.018), (27, -0.05), (28, -0.043), (29, 0.013), (30, 0.007), (31, 0.018), (32, -0.042), (33, -0.068), (34, -0.022), (35, -0.002), (36, -0.002), (37, -0.016), (38, -0.011), (39, 0.023), (40, 0.062), (41, -0.037), (42, 0.105), (43, 0.044), (44, 0.04), (45, -0.087), (46, 0.1), (47, 0.01), (48, 0.016), (49, -0.027)]
simIndex simValue blogId blogTitle
same-blog 1 0.95083517 1160 high scalability-2011-12-21-In Memory Data Grid Technologies
Introduction: After winning a CSC Leading Edge Forum (LEF) research grant, I (Paul Colmer) wanted to publish some of the highlights of my research to share with the wider technology community. What is an In Memory Data Grid? It is not an in-memory relational database, a NOSQL database or a relational database. It is a different breed of software datastore. In summary an IMDG is an ‘off the shelf’ software product that exhibits the following characteristics: The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. All servers can be active in each site. All data is stored in the RAM of the servers. Servers can be added or removed non-disruptively, to increase the amount of RAM available. The data model is non-relational and is object-based. Distributed applications written on the .NET and Java application platforms are s
2 0.78851742 393 high scalability-2008-09-25-GridGain: One Compute Grid, Many Data Grids
Introduction: GridGain was kind enough to present at the September 17th instance of the Silicon Valley Cloud Computing Group . I've been curious about GridGain so I was glad to see them there. In short GridGain is: an open source computational grid framework that enables Java developers to improve general performance of processing intensive applications by splitting and parallelizing the workload . GridGain can also be thought of as a set of middleware primitives for building applications. GridGain's peer group of competitors includes GigaSpaces, Terracotta, Coherence, and Hadoop. The speaker for GridGain was the President and Founder, Nikita Ivanov. He has a very pleasant down-to-earth way about him that contrasts nicely with a field given to religious discussions of complex taxomic definitions. Nikita first talked about cloud computing in general. He feels Java is the perfect gateway for cloud computing. Which is good because GridGain only works with Java . The Java centricity of G
3 0.77816153 696 high scalability-2009-09-07-Product: Infinispan - Open Source Data Grid
Introduction: Infinispan is a highly scalable, open source licensed data grid platform in the style of GigaSpaces and Oracle Coherence . From their website: The purpose of Infinispan is to expose a data structure that is highly concurrent, designed ground-up to make the most of modern multi-processor/multi-core architectures while at the same time providing distributed cache capabilities. At its core Infinispan exposes a JSR-107 (JCACHE) compatible Cache interface (which in turn extends java.util.Map). It is also optionally is backed by a peer-to-peer network architecture to distribute state efficiently around a data grid. Offering high availability via making replicas of state across a network as well as optionally persisting state to configurable cache stores, Infinispan offers enterprise features such as efficient eviction algorithms to control memory usage as well as JTA compatibility. In addition to the peer-to-peer architecture of Infinispan, on the roadmap is the ability to run f
4 0.75213104 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
Introduction: We are on the edge of two potent technological changes: Clouds and Memory Based Architectures. This evolution will rip open a chasm where new players can enter and prosper. Google is the master of disk. You can't beat them at a game they perfected. Disk based databases like SimpleDB and BigTable are complicated beasts, typical last gasp products of any aging technology before a change. The next era is the age of Memory and Cloud which will allow for new players to succeed. The tipping point will be soon. Let's take a short trip down web architecture lane: It's 1993: Yahoo runs on FreeBSD, Apache, Perl scripts and a SQL database It's 1995: Scale-up the database. It's 1998: LAMP It's 1999: Stateless + Load Balanced + Database + SAN It's 2001: In-memory data-grid. It's 2003: Add a caching layer. It's 2004: Add scale-out and partitioning. It's 2005: Add asynchronous job scheduling and maybe a distributed file system. It's 2007: Move it all into the cloud. It's 2008: C
5 0.74873453 633 high scalability-2009-06-19-GemFire 6.0: New innovations in data management
Introduction: GemStone has unveiled GemFire 6.0 which is the culmination of several years of development and the continuous solving of the hardest data management problems in the world. With this release GemFire touts some of the latest innovative features in data management. In this release: - GemFire introduces a resource manager to continuously monitor and protect cache instances from running out of memory, triggering rebalancing to migrate data to less loaded nodes or allow dynamic increase/decrease in the number of nodes hosting data for linear scalability without impeding ongoing operations (no contention points). - GemFire provides explicit control over when rebalancing can be triggered, on what class of data and even allows the administrator to simulate a "rebalance" operation to quantify the benefits before actually doing it. - With built in instrumentation that captures throughput and latency metrics, GemFire now enables applications to sense changing performance patterns and proactiv
6 0.74779856 373 high scalability-2008-08-29-Product: ScaleOut StateServer is Memcached on Steroids
7 0.74636227 542 high scalability-2009-03-17-IBM WebSphere eXtreme Scale (IMDG)
8 0.73274016 1118 high scalability-2011-09-19-Big Iron Returns with BigMemory
10 0.71932733 395 high scalability-2008-09-25-Is your cloud as scalable as you think it is?
11 0.7177493 396 high scalability-2008-09-26-Lucasfilm: The Real Magic is in the Data Center
12 0.71627069 597 high scalability-2009-05-12-GemStone Unveils GemFire Enterprise 6.0
13 0.71242505 364 high scalability-2008-08-14-Product: Terracotta - Open Source Network-Attached Memory
14 0.70879877 423 high scalability-2008-10-19-Alternatives to Google App Engine
15 0.69026309 1161 high scalability-2011-12-22-Architecting Massively-Scalable Near-Real-Time Risk Analysis Solutions
16 0.67912406 292 high scalability-2008-03-30-Scaling Out MySQL
17 0.67655528 268 high scalability-2008-03-06-Announce: First Meeting of Boston Scalability User Group
18 0.66802442 822 high scalability-2010-05-04-Business continuity with real-time data integration
19 0.6606245 126 high scalability-2007-10-20-Should you build your next website using 3tera's grid OS?
20 0.65993416 1016 high scalability-2011-04-04-Scaling Social Ecommerce Architecture Case study
topicId topicWeight
[(1, 0.247), (2, 0.129), (10, 0.037), (23, 0.014), (26, 0.016), (29, 0.016), (56, 0.033), (61, 0.066), (79, 0.073), (85, 0.012), (94, 0.285)]
simIndex simValue blogId blogTitle
1 0.96449423 241 high scalability-2008-02-05-SLA monitoring
Introduction: Hi, We're running a enterprise SaaS solution that currently holds about 700 customers with up to 50.000 users per customer (growing quickly). Our customers have SLA agreements with us that contains guaranteed uptimes, response times and other performance counters. With an increasing number of customers and traffic we find it difficult to provide our customer with actual SLA data. We could set up external probes that monitors certain parts of the application, but this is time consuming with 700 customers (we do it today for our biggest clients). We can also extract data from web logs but they are now approaching about 30-40 GB a day. What we really need is monitoring software that not only focuses on the internal performance counters but also lets us see the application from the customers viewpoint and allows us to aggregate data in different ways. Would the best approach be to develop a custom solution (for instance a distributed app that aggregates data from different logs e
2 0.95278281 1412 high scalability-2013-02-25-SongPop Scales to 1 Million Active Users on GAE, Showing PaaS is not Passé
Introduction: Should you use PaaS for your next project? Often the answer is no because you want control, but here's an example from SongPop showing why the promise of PaaS is not passé. SongPop was able to autoscale to 60 million users, 1 million daily active users, deliver 17 terabytes/day of songs and images worldwide, handle 10k+ queries/second, all with a 6 person engineering team, and only one engineer working full-time on the backend. Unfortunately there aren't a lot of details, but what there is can be found in Scaling SongPop to 60 million users with App Engine and Google Cloud Storage . The outline follows the script. You start small. Let PaaS do the heavy lifting. And when you need to scale you just buy more resources and tune a little (maybe a lot). The payoff is you get to focus on feature development and can get by with a small team. Here's a diagram of their architecture: Some lessons learned: Premier Support. This one sounds a bit like a sales pitch, but once they
3 0.93752146 115 high scalability-2007-10-07-Using ThreadLocal to pass context information around in web applications
Introduction: Hi, In java web servers, each http request is handled by a thread in thread pool. So for a Servlet handling the request, a thread is assigned. It is tempting (and very convinient) to keep context information in the threadlocal variable. I recently had a requirement where we need to assign logged in user id and timestamp to request sent to web services. Because we already had the code in place, it was extremely difficult to change the method signatures to pass user id everywhere. The solution I thought is class ReferenceIdGenerator { public static setReferenceId(String login) { threadLocal.set(login + System.currentMillis()); } public static String getReferenceId() { return threadLocal.get(); } private static ThreadLocal threadLocal = new ThreadLocal(); } class MySevlet { void service(.....) { HttpSession session = request.getSession(false); String userId = session.get("userId"); ReferenceIdGenerator.setRefernceId(userId
Introduction: The reference configurations described in this blueprint are starting points for building Sun Customer Ready HPC Clusters configured with Sun Fire X2100 M2 and X2200 M2 servers. The configurations define how Sun Systems Group products can be configured in a typical grid rack deployment. This document describes configurations in detail using Sun Fire X2100 M2 and X2200 M2 servers with a Gigabit Ethernet data fabric, as well as configurations using Sun Fire X2200 M2 servers with a high-speed InfiniBand fabric. These configurations focus on single rack solutions, with external connections through uplink ports of the switches. These reference configurations have been architected using Sun's expertise gained in actual, real-world installations. Within certain constraints, as described in the later sections, the system can be tailored to the customer needs. Certain system components described in this document are only available through Sun's factory integration. Although the information
5 0.92039633 827 high scalability-2010-05-14-Hot Scalability Links for May 14, 2010
Introduction: Lots of good ones this week... Scalability, Availability & Stability Patterns . Jonas Boner has 197 slides covering a very wide range of scalability topics. One stop scalability shopping. Horizontal Scalability via Transient, Shardable, and Share-Nothing Resources . Heroku's Adam Wiggins shares what they've learned about scaling based on their experiences building a cloud platform and the hundreds of apps running on it. He describes the next generation architecture he thinks all software should follow in the future. Scalability of the Hadoop Distributed File System . Konstantin V. Shvachko writes a great post analyzing if the limitations imposed on a distributed file system by the single-node namespace server architecture can support 100,000 clients and petabytes of files. Cassandra by Example . Eric Evans created a nice Cassandra tutorial using building a Twitter clone as an example. Many people want to see more data modeling examples. Here you are. UpSizeR: Synthet
same-blog 6 0.91999853 1160 high scalability-2011-12-21-In Memory Data Grid Technologies
7 0.91815078 78 high scalability-2007-09-01-2 tier switch selection for colocation
8 0.9172219 834 high scalability-2010-06-01-Web Speed Can Push You Off of Google Search Rankings! What Can You Do?
9 0.91426468 605 high scalability-2009-05-22-Distributed content system with bandwidth balancing
10 0.91278613 1222 high scalability-2012-04-05-Big Data Counting: How to count a billion distinct objects using only 1.5KB of Memory
11 0.91059601 486 high scalability-2009-01-07-Sun Acquires Q-layer in Cloud Computing Play
12 0.90570277 1025 high scalability-2011-04-16-The NewSQL Market Breakdown
13 0.9051314 39 high scalability-2007-07-30-Product: Akamai
14 0.89954275 1601 high scalability-2014-02-25-Peter Norvig's 9 Master Steps to Improving a Program
15 0.89115644 970 high scalability-2011-01-06-BankSimple Mini-Architecture - Using a Next Generation Toolchain
16 0.8907786 1305 high scalability-2012-08-16-Paper: A Provably Correct Scalable Concurrent Skip List
17 0.88984239 1084 high scalability-2011-07-22-Stuff The Internet Says On Scalability For July 22, 2011
18 0.87833166 976 high scalability-2011-01-20-75% Chance of Scale - Leveraging the New Scaleogenic Environment for Growth
19 0.87741542 266 high scalability-2008-03-04-Manage Downtime Risk by Connecting Multiple Data Centers into a Secure Virtual LAN
20 0.87312376 1174 high scalability-2012-01-13-Stuff The Internet Says On Scalability For January 13, 2012