high_scalability high_scalability-2009 high_scalability-2009-546 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: An alternate strategy for database sharding which avoids queries across different shards and merging results. A central repository of data is maintained for some tables along with other shards. Can be used in calculating top users, recent users, most read etc.
sentIndex sentText sentNum sentScore
1 An alternate strategy for database sharding which avoids queries across different shards and merging results. [sent-1, score-1.929]
2 A central repository of data is maintained for some tables along with other shards. [sent-2, score-1.224]
3 Can be used in calculating top users, recent users, most read etc. [sent-3, score-0.798]
wordName wordTfidf (topN-words)
[('alternate', 0.366), ('repository', 0.317), ('calculating', 0.314), ('merging', 0.308), ('avoids', 0.3), ('maintained', 0.282), ('shards', 0.247), ('central', 0.218), ('recent', 0.194), ('sharding', 0.193), ('tables', 0.191), ('along', 0.179), ('etc', 0.175), ('users', 0.169), ('strategy', 0.153), ('queries', 0.138), ('top', 0.128), ('read', 0.092), ('across', 0.09), ('different', 0.073), ('used', 0.07), ('database', 0.061), ('data', 0.037)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 546 high scalability-2009-03-20-Alternate strategy for database sharding
Introduction: An alternate strategy for database sharding which avoids queries across different shards and merging results. A central repository of data is maintained for some tables along with other shards. Can be used in calculating top users, recent users, most read etc.
2 0.16265185 672 high scalability-2009-08-06-An Unorthodox Approach to Database Design : The Coming of the Shard
Introduction: Update 4: Why you don’t want to shard. by Morgon on the MySQL Performance Blog. Optimize everything else first, and then if performance still isn’t good enough, it’s time to take a very bitter medicine. Update 3: Building Scalable Databases: Pros and Cons of Various Database Sharding Schemes by Dare Obasanjo. Excellent discussion of why and when you would choose a sharding architecture, how to shard, and problems with sharding. Update 2: Mr. Moore gets to punt on sharding by Alan Rimm-Kaufman of 37signals. Insightful article on design tradeoffs and the evils of premature optimization. With more memory, more CPU, and new tech like SSD, problems can be avoided before more exotic architectures like sharding are needed. Add features not infrastructure. Jeremy Zawodny says he's wrong wrong wrong. we're running multi-core CPUs at slower clock speeds. Moore won't save you. Update: Dan Pritchett shares some excellent Sharding Lessons : Size Your Shards, Use Math on Shard C
3 0.15222304 24 high scalability-2007-07-24-Product: Hibernate Shards
Introduction: If you want to adopt a shard architecture, but don't want to start from scratch, you may want to consider Hibernate's sharding system. Hibernate Shards is a framework that is designed to encapsulate and minimize this complexity by adding support for horizontal partitioning to Hibernate Core. Hibernate Shards key features: Standard Hibernate programming model - Hibernate Shards allows you to continue using the Hibernate APIs you know and love: SessionFactory, Session, Criteria, Query. If you already know how to use Hibernate, you already know how to use Hibernate Shards. Flexible sharding strategies - Distribute data across your shards any way you want. Use one of the default strategies we provide or plug in your own application-specific logic. Support for virtual shards - Think your sharding strategy is never going to change? Think again. Adding new shards and redistributing your data is one of the toughest operational challenges you will face once you've deployed your
4 0.14419644 358 high scalability-2008-07-26-Sharding the Hibernate Way
Introduction: Update : A very nice JavaWorld podcast interview with Google engineer Max Ross on Hibernate Shards . Max defines Hibernate Shards (horizontal partitioning), how it works (pretty well), virtual shards (don't ask), what they need to do in the future (query, replication, operational tools), and how it relates to Google AppEngine (not much). To scale you are supposed to partition your data. Sounds good, but how do you do it? When you actually sit down to work out all the details it’s not that easy. Hibernate Shards to the rescue! Hibernate shards is: an extension to the core Hibernate product that adds facilities for horizontal partitioning. If you know the core Hibernate API you know the shards API. No learning curve at all. Here is what a few members of the core group had to say about the Hibernate Shards open source project. Although there are some limitations, from the sound of it they are doing useful stuff in the right way and it’s very much worth looking at, especially if you us
5 0.11492443 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
Introduction: For everything given something seems to be taken. Caching is a great scalability solution, but caching also comes with problems . Sharding is a great scalability solution, but as Foursquare recently revealed in a post-mortem about their 17 hours of downtime, sharding also has problems. MongoDB, the database Foursquare uses, also contributed their post-mortem of what went wrong too. Now that everyone has shared and resharded, what can we learn to help us skip these mistakes and quickly move on to a different set of mistakes? First, like for Facebook , huge props to Foursquare and MongoDB for being upfront and honest about their problems. This helps everyone get better and is a sign we work in a pretty cool industry. Second, overall, the fault didn't flow from evil hearts or gross negligence. As usual the cause was more mundane: a key system, that could be a little more robust, combined with a very popular application built by a small group of people, under immense pressure
6 0.11337153 152 high scalability-2007-11-13-Flickr Architecture
7 0.095565163 847 high scalability-2010-06-23-Product: dbShards - Share Nothing. Shard Everything.
8 0.091936208 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
9 0.075990334 1114 high scalability-2011-09-13-Must see: 5 Steps to Scaling MongoDB (Or Any DB) in 8 Minutes
10 0.0710399 821 high scalability-2010-05-03-MocoSpace Architecture - 3 Billion Mobile Page Views a Month
11 0.070908472 1514 high scalability-2013-09-09-Need Help with Database Scalability? Understand I-O
12 0.068606481 912 high scalability-2010-10-01-Google Paper: Large-scale Incremental Processing Using Distributed Transactions and Notifications
13 0.06791351 1529 high scalability-2013-10-08-F1 and Spanner Holistically Compared
14 0.066348128 1191 high scalability-2012-02-13-Tumblr Architecture - 15 Billion Page Views a Month and Harder to Scale than Twitter
15 0.066348128 1360 high scalability-2012-11-19-Gone Fishin': Tumblr Architecture - 15 Billion Page Views A Month And Harder To Scale Than Twitter
16 0.065867364 1135 high scalability-2011-10-31-15 Ways to Make Your Application Feel More Responsive under Google App Engine
17 0.065606281 1461 high scalability-2013-05-20-The Tumblr Architecture Yahoo Bought for a Cool Billion Dollars
18 0.06459251 327 high scalability-2008-05-27-How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale
19 0.063314021 908 high scalability-2010-09-28-6 Strategies for Scaling BBC iPlayer
20 0.06305398 825 high scalability-2010-05-10-Sify.com Architecture - A Portal at 3900 Requests Per Second
topicId topicWeight
[(0, 0.072), (1, 0.059), (2, -0.028), (3, -0.066), (4, 0.04), (5, 0.051), (6, -0.007), (7, -0.042), (8, 0.037), (9, -0.02), (10, 0.012), (11, 0.065), (12, -0.047), (13, 0.023), (14, 0.019), (15, 0.038), (16, -0.045), (17, -0.034), (18, 0.001), (19, 0.029), (20, 0.016), (21, -0.023), (22, -0.015), (23, 0.005), (24, -0.039), (25, 0.016), (26, -0.033), (27, -0.032), (28, -0.022), (29, 0.073), (30, 0.028), (31, -0.0), (32, 0.02), (33, -0.023), (34, 0.066), (35, -0.002), (36, 0.023), (37, -0.022), (38, -0.036), (39, -0.004), (40, -0.043), (41, 0.069), (42, -0.009), (43, -0.012), (44, -0.004), (45, -0.006), (46, -0.022), (47, -0.037), (48, -0.051), (49, 0.052)]
simIndex simValue blogId blogTitle
same-blog 1 0.93924773 546 high scalability-2009-03-20-Alternate strategy for database sharding
Introduction: An alternate strategy for database sharding which avoids queries across different shards and merging results. A central repository of data is maintained for some tables along with other shards. Can be used in calculating top users, recent users, most read etc.
2 0.83753759 358 high scalability-2008-07-26-Sharding the Hibernate Way
Introduction: Update : A very nice JavaWorld podcast interview with Google engineer Max Ross on Hibernate Shards . Max defines Hibernate Shards (horizontal partitioning), how it works (pretty well), virtual shards (don't ask), what they need to do in the future (query, replication, operational tools), and how it relates to Google AppEngine (not much). To scale you are supposed to partition your data. Sounds good, but how do you do it? When you actually sit down to work out all the details it’s not that easy. Hibernate Shards to the rescue! Hibernate shards is: an extension to the core Hibernate product that adds facilities for horizontal partitioning. If you know the core Hibernate API you know the shards API. No learning curve at all. Here is what a few members of the core group had to say about the Hibernate Shards open source project. Although there are some limitations, from the sound of it they are doing useful stuff in the right way and it’s very much worth looking at, especially if you us
3 0.82713848 476 high scalability-2008-12-28-How to Organize a Database Table’s Keys for Scalability
Introduction: The key (no pun intended) to understanding how to organize your dataset’s data is to think of each shard not as an individual database, but as one large singular database. Just as in a normal single server database setup where you have a unique key for each row within a table, each row key within each individual shard must be unique to the whole dataset partitioned across all shards. There are a few different ways we can accomplish uniqueness of row keys across a shard cluster. Each has its pro’s and con’s and the one chosen should be specific to the problems you’re trying to solve.
4 0.80122334 24 high scalability-2007-07-24-Product: Hibernate Shards
Introduction: If you want to adopt a shard architecture, but don't want to start from scratch, you may want to consider Hibernate's sharding system. Hibernate Shards is a framework that is designed to encapsulate and minimize this complexity by adding support for horizontal partitioning to Hibernate Core. Hibernate Shards key features: Standard Hibernate programming model - Hibernate Shards allows you to continue using the Hibernate APIs you know and love: SessionFactory, Session, Criteria, Query. If you already know how to use Hibernate, you already know how to use Hibernate Shards. Flexible sharding strategies - Distribute data across your shards any way you want. Use one of the default strategies we provide or plug in your own application-specific logic. Support for virtual shards - Think your sharding strategy is never going to change? Think again. Adding new shards and redistributing your data is one of the toughest operational challenges you will face once you've deployed your
5 0.73732632 672 high scalability-2009-08-06-An Unorthodox Approach to Database Design : The Coming of the Shard
Introduction: Update 4: Why you don’t want to shard. by Morgon on the MySQL Performance Blog. Optimize everything else first, and then if performance still isn’t good enough, it’s time to take a very bitter medicine. Update 3: Building Scalable Databases: Pros and Cons of Various Database Sharding Schemes by Dare Obasanjo. Excellent discussion of why and when you would choose a sharding architecture, how to shard, and problems with sharding. Update 2: Mr. Moore gets to punt on sharding by Alan Rimm-Kaufman of 37signals. Insightful article on design tradeoffs and the evils of premature optimization. With more memory, more CPU, and new tech like SSD, problems can be avoided before more exotic architectures like sharding are needed. Add features not infrastructure. Jeremy Zawodny says he's wrong wrong wrong. we're running multi-core CPUs at slower clock speeds. Moore won't save you. Update: Dan Pritchett shares some excellent Sharding Lessons : Size Your Shards, Use Math on Shard C
6 0.71487296 345 high scalability-2008-06-11-Pyshards aspires to build sharding toolkit for Python
7 0.69884682 152 high scalability-2007-11-13-Flickr Architecture
8 0.67133981 847 high scalability-2010-06-23-Product: dbShards - Share Nothing. Shard Everything.
9 0.67131311 857 high scalability-2010-07-13-DbShards Part Deux - The Internals
10 0.66961557 207 high scalability-2008-01-10-Sharding with Cookie-Based Session Storage
11 0.6526112 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
12 0.64503109 1514 high scalability-2013-09-09-Need Help with Database Scalability? Understand I-O
13 0.63014334 561 high scalability-2009-04-08-N+1+caching is ok?
14 0.60312313 281 high scalability-2008-03-18-Database Design 101
15 0.59959245 578 high scalability-2009-04-23-Which Key value pair database to be used
16 0.58517152 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
17 0.57985079 775 high scalability-2010-02-10-ElasticSearch - Open Source, Distributed, RESTful Search Engine
18 0.57906705 348 high scalability-2008-07-09-Federation at Flickr: Doing Billions of Queries Per Day
19 0.57172132 1046 high scalability-2011-05-23-Evernote Architecture - 9 Million Users and 150 Million Requests a Day
20 0.5679037 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
topicId topicWeight
[(1, 0.149), (10, 0.171), (25, 0.382), (61, 0.118)]
simIndex simValue blogId blogTitle
1 0.90172005 246 high scalability-2008-02-12-Search the tags across all post
Introduction: Let suppose i have table which stored tags .Now user can enter keywords and i have to search through all the records in table and find post which contain tags entered by user .user can enter more than 1 keywords. What strategy ,technique i use to search fast .There maybe more than millions records and many users are firing same query. Thanks
same-blog 2 0.78122908 546 high scalability-2009-03-20-Alternate strategy for database sharding
Introduction: An alternate strategy for database sharding which avoids queries across different shards and merging results. A central repository of data is maintained for some tables along with other shards. Can be used in calculating top users, recent users, most read etc.
Introduction: One of the world's leading technical institutes, the Tokyo Institute of Technology (Tokyo Tech) created the fastest supercomputer in Asia, and one of the largest outside of the United States. Using Sun x64 servers and data servers deployed in a grid architecture, Tokyo Tech built a cost-effective, flexible supercomputer that meets the demands of compute and data-intensive applications. Built in just 35 days, the TSUBAME grid includes hundreds of systems incorporating thousands of processor cores and terabytes of memory, and delivers 47.38 trillion floating-point operations per second (TeraFLOPS) of sustained LINPACK benchmark performance and 1.1 petabyte of storage to users running common off-the-shelf applications. Based on the deployment architecture, the grid is expected to reach 100 TeraFLOPS in the future. This article provides an overview of the Tokyo Tech grid, named TSUBAME. The first in a series of Sun BluePrints articles on the TSUBAME grid, this document discusses the re
4 0.60698777 412 high scalability-2008-10-14-Sun N1 Grid Engine Software and the Tokyo Institute of Technology Super Computer Grid
Introduction: One of the world's leading technical institutes, the Tokyo Institute of Technology (Tokyo Tech) created the fastest supercomputer in Asia, and one of the largest outside of the United States. Using Sun x64 servers and data servers deployed in a grid architecture, Tokyo Tech built a cost-effective, flexible supercomputer that meets the demands of compute- and data-intensive applications. Built in just 35 days, the TSUBAME grid includes hundreds of systems incorporating thousands of processor cores and terabytes of memory, and delivers 47.38 trillion1 floating-point operations per second (TeraFLOPS) of sustained LINPACK benchmark performance and 1.1 petabyte of storage to users running common off-the-shelf applications. Based on the deployment architecture, the grid is expected to reach 100 TeraFLOPS in the future. This Sun BluePrints article provides an overview of the Tokyo Tech grid, named TSUBAME. The third in a series of Sun BluePrints articles on the TSUBAME grid, this document pro
5 0.58110023 408 high scalability-2008-10-10-Useful Corporate Blogs that Talk About Scalability
Introduction: Some intrepid company blogs are posting their technical challenges and how they solve them. I wish more would open up and talk about what they are doing as it helps everyone move forward. Here are a few blogs documenting their encounters with the bleeding edge: Flickr Digg LinkedIn Facebook Amazon Web Services blog Twitter blog Reddit blog Photobucket blog Second Life blog PlentyofFish blog Joyent's Blog Any others that should be added?
6 0.57290936 203 high scalability-2008-01-07-How Ruby on Rails Survived a 550k Pageview Digging
7 0.50883543 570 high scalability-2009-04-15-Implementing large scale web analytics
8 0.48991594 874 high scalability-2010-08-07-ArchCamp: Scalable Databases (NoSQL)
9 0.46791497 584 high scalability-2009-04-27-Some Questions from a newbie
10 0.4469614 170 high scalability-2007-12-02-Database-Clustering: a8cjdbc - update: version 1.3
11 0.44694963 171 high scalability-2007-12-02-a8cjdbc - update verision 1.3
12 0.44385913 652 high scalability-2009-07-08-Art of Parallelism presentation
13 0.43667445 239 high scalability-2008-02-04-Streaming Video on Amazon EC2?
14 0.43390769 992 high scalability-2011-02-18-Stuff The Internet Says On Scalability For February 18, 2011
16 0.4302282 1635 high scalability-2014-04-21-This is why Microsoft won. And why they lost.
17 0.4259606 641 high scalability-2009-06-29-Google App Engine plus Amazon AWS: Best of both worlds
18 0.42312568 430 high scalability-2008-10-26-Should you use a SAN to scale your architecture?
19 0.41599336 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day
20 0.41240159 476 high scalability-2008-12-28-How to Organize a Database Table’s Keys for Scalability