high_scalability high_scalability-2009 high_scalability-2009-732 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Digg has been researching ways to scale our database infrastructure for some time now. We’ve adopted a traditional vertically partitioned master-slave configuration with MySQL, and also investigated sharding MySQL with IDDB . Ultimately, these solutions left us wanting. In the case of the traditional architecture, the lack of redundancy on the write masters is painful, and both approaches have significant management overhead to keep running. Since it was already necessary to abandon data normalization and consistency to make these approaches work, we felt comfortable looking at more exotic, non-relational data stores. After considering HBase, Hypertable, Cassandra, Tokyo Cabinet/Tyrant, Voldemort, and Dynomite, we settled on Cassandra . Each system has its own strengths and weaknesses, but Cassandra has a good blend of everything. It offers column-oriented data storage, so you have a bit more structure than plain key/value stores. It operates in a distributed, highly available,
sentIndex sentText sentNum sentScore
1 Digg has been researching ways to scale our database infrastructure for some time now. [sent-1, score-0.274]
2 We’ve adopted a traditional vertically partitioned master-slave configuration with MySQL, and also investigated sharding MySQL with IDDB . [sent-2, score-0.876]
3 In the case of the traditional architecture, the lack of redundancy on the write masters is painful, and both approaches have significant management overhead to keep running. [sent-4, score-0.898]
4 Since it was already necessary to abandon data normalization and consistency to make these approaches work, we felt comfortable looking at more exotic, non-relational data stores. [sent-5, score-1.004]
5 After considering HBase, Hypertable, Cassandra, Tokyo Cabinet/Tyrant, Voldemort, and Dynomite, we settled on Cassandra . [sent-6, score-0.266]
6 Each system has its own strengths and weaknesses, but Cassandra has a good blend of everything. [sent-7, score-0.37]
7 It offers column-oriented data storage, so you have a bit more structure than plain key/value stores. [sent-8, score-0.388]
8 It operates in a distributed, highly available, peer-to-peer cluster. [sent-9, score-0.132]
9 While it’s currently lacking some core features, it gets us closer to where we want to be than the other solutions. [sent-10, score-0.615]
wordName wordTfidf (topN-words)
[('exotic', 0.207), ('dynomite', 0.207), ('investigated', 0.207), ('researching', 0.199), ('approaches', 0.198), ('weaknesses', 0.193), ('blend', 0.187), ('lacking', 0.187), ('strengths', 0.183), ('abandon', 0.183), ('normalization', 0.175), ('hypertable', 0.166), ('vertically', 0.159), ('traditional', 0.159), ('settled', 0.157), ('cassandra', 0.156), ('adopted', 0.155), ('painful', 0.153), ('voldemort', 0.153), ('tokyo', 0.15), ('plain', 0.147), ('felt', 0.147), ('masters', 0.141), ('ultimately', 0.139), ('comfortable', 0.135), ('operates', 0.132), ('closer', 0.125), ('digg', 0.123), ('left', 0.113), ('considering', 0.109), ('hbase', 0.108), ('redundancy', 0.108), ('partitioned', 0.107), ('lack', 0.102), ('mysql', 0.101), ('significant', 0.096), ('us', 0.094), ('overhead', 0.094), ('necessary', 0.09), ('sharding', 0.089), ('offers', 0.084), ('structure', 0.081), ('bit', 0.076), ('consistency', 0.076), ('ways', 0.075), ('gets', 0.074), ('currently', 0.073), ('solutions', 0.067), ('configuration', 0.063), ('core', 0.062)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 732 high scalability-2009-10-29-Digg - Looking to the Future with Cassandra
Introduction: Digg has been researching ways to scale our database infrastructure for some time now. We’ve adopted a traditional vertically partitioned master-slave configuration with MySQL, and also investigated sharding MySQL with IDDB . Ultimately, these solutions left us wanting. In the case of the traditional architecture, the lack of redundancy on the write masters is painful, and both approaches have significant management overhead to keep running. Since it was already necessary to abandon data normalization and consistency to make these approaches work, we felt comfortable looking at more exotic, non-relational data stores. After considering HBase, Hypertable, Cassandra, Tokyo Cabinet/Tyrant, Voldemort, and Dynomite, we settled on Cassandra . Each system has its own strengths and weaknesses, but Cassandra has a good blend of everything. It offers column-oriented data storage, so you have a bit more structure than plain key/value stores. It operates in a distributed, highly available,
Introduction: This is a guest post by Doug Judd , original creator of Hypertable and the CEO of Hypertable, Inc. Hypertable delivers 2X better throughput in most tests -- HBase fails 41 and 167 billion record insert tests, overwhelmed by garbage collection -- Both systems deliver similar results for random read uniform test We recently conducted a test comparing the performance of Hypertable ( @hypertable ) version 0.9.5.5 to that of HBase ( @HBase ) version 0.90.4 (CDH3u2) running Zookeeper 3.3.4. In this post, we summarize the results and offer explanations for the discrepancies. For the full test report, see Hypertable vs. HBase II . Introduction Hypertable and HBase are both open source, scalable databases modeled after Google's proprietary Bigtable database. The primary difference between the two systems is that Hypertable is written in C++, while HBase is written in Java. We modeled this test after the one described in section 7 of the Bigtable paper and tuned both systems fo
3 0.12861867 670 high scalability-2009-08-05-Anti-RDBMS: A list of distributed key-value stores
Introduction: Update 8: Introducing MongoDB by Eliot Horowit . Update 7: The Future of Scalable Databases by Robin Mathew. Update 6: NoSQL : If Only it Was that Easy . BJ Clark lays down the law on which databases are scalable: Tokyo - NO, Redis - NO, Voldemort - YES, MongoDB - Not Yet, Cassandra - Probably, Amazon S3 - YES * 2, MySQL - NO. The real thing to point out is that if you are being held back from making something super awesome because you can’t choose a database, you are doing it wrong. Update 5: Exciting stuff happening in Japan at this Key-Value Storage meeting in Tokyo . Presentations on Groonga, Senna, Lux IO, Tokyo-Cabinet, Tx, repcached, Kai, Cagra, kumofs, ROMA, and Flare. Update 4: NoSQL and the Relational Model: don’t throw the baby out with the bathwater by Matthew Willson. So my key point is, this kind of modelling is WORTH DOING, regardless of which database tool you end up using for physical storage. Update 3: Choosing a non-relational database
4 0.12655692 634 high scalability-2009-06-20-Building a data cycle at LinkedIn with Hadoop and Project Voldemort
Introduction: Update : Building Voldemort read-only stores with Hadoop . A write up on what LinkedIn is doing to integrate large offline Hadoop data processing jobs with a fast, distributed online key-value storage system, Project Voldemort .
5 0.1194315 647 high scalability-2009-07-02-Hypertable is a New BigTable Clone that Runs on HDFS or KFS
Introduction: Update 3 : Presentation from the NoSQL conference : slides , video 1 , video 2 . Update 2 : The folks at Hypertable would like you to know that Hypertable is now officially sponsored by Baidu , China’s Leading Search Engine. As a sponsor of Hypertable, Baidu has committed an industrious team of engineers, numerous servers, and support resources to improve the quality and development of the open source technology. Update : InfoQ interview on Hypertable Lead Discusses Hadoop and Distributed Databases . Hypertable differs from HBase in that it is a higher performance implementation of Bigtable. Skrentablog gives the heads up on Hypertable , Zvents' open-source BigTable clone. It's written in C++ and can run on top of either HDFS or KFS. Performance looks encouraging at 28M rows of data inserted at a per-node write rate of 7mb/sec .
6 0.1061082 855 high scalability-2010-07-11-So, Why is Twitter Really Not Using Cassandra to Store Tweets?
7 0.10444978 554 high scalability-2009-04-04-Digg Architecture
8 0.10228053 736 high scalability-2009-11-04-Damn, Which Database do I Use Now?
9 0.098375373 943 high scalability-2010-11-16-Facebook's New Real-time Messaging System: HBase to Store 135+ Billion Messages a Month
10 0.098031424 672 high scalability-2009-08-06-An Unorthodox Approach to Database Design : The Coming of the Shard
11 0.093727037 1180 high scalability-2012-01-24-The State of NoSQL in 2012
12 0.089788862 875 high scalability-2010-08-09-NoSQL on the Microsoft Platform
13 0.088099211 833 high scalability-2010-06-01-Sponsored Post: Get Your High Scalability Fix at Digg
14 0.085242316 1382 high scalability-2013-01-07-Analyzing billions of credit card transactions and serving low-latency insights in the cloud
15 0.083825216 793 high scalability-2010-03-10-Saying Yes to NoSQL; Going Steady with Cassandra at Digg
16 0.081686541 795 high scalability-2010-03-16-1 Billion Reasons Why Adobe Chose HBase
17 0.08088693 848 high scalability-2010-06-25-Hot Scalability Links for June 25, 2010
18 0.079584524 1109 high scalability-2011-09-02-Stuff The Internet Says On Scalability For September 2, 2011
19 0.079388328 545 high scalability-2009-03-19-Product: Redis - Not Just Another Key-Value Store
20 0.078878209 1022 high scalability-2011-04-13-Paper: NoSQL Databases - NoSQL Introduction and Overview
topicId topicWeight
[(0, 0.112), (1, 0.039), (2, -0.01), (3, 0.031), (4, 0.048), (5, 0.092), (6, -0.054), (7, -0.091), (8, 0.018), (9, -0.001), (10, 0.013), (11, 0.014), (12, 0.02), (13, 0.003), (14, 0.01), (15, 0.015), (16, 0.01), (17, -0.006), (18, -0.093), (19, -0.056), (20, 0.024), (21, 0.031), (22, 0.005), (23, -0.002), (24, 0.029), (25, -0.029), (26, 0.004), (27, 0.0), (28, -0.015), (29, -0.077), (30, 0.032), (31, 0.03), (32, 0.02), (33, 0.001), (34, 0.043), (35, -0.017), (36, 0.01), (37, -0.023), (38, -0.036), (39, 0.015), (40, 0.028), (41, 0.042), (42, 0.055), (43, -0.004), (44, 0.023), (45, 0.006), (46, -0.059), (47, -0.01), (48, 0.011), (49, -0.022)]
simIndex simValue blogId blogTitle
same-blog 1 0.91192609 732 high scalability-2009-10-29-Digg - Looking to the Future with Cassandra
Introduction: Digg has been researching ways to scale our database infrastructure for some time now. We’ve adopted a traditional vertically partitioned master-slave configuration with MySQL, and also investigated sharding MySQL with IDDB . Ultimately, these solutions left us wanting. In the case of the traditional architecture, the lack of redundancy on the write masters is painful, and both approaches have significant management overhead to keep running. Since it was already necessary to abandon data normalization and consistency to make these approaches work, we felt comfortable looking at more exotic, non-relational data stores. After considering HBase, Hypertable, Cassandra, Tokyo Cabinet/Tyrant, Voldemort, and Dynomite, we settled on Cassandra . Each system has its own strengths and weaknesses, but Cassandra has a good blend of everything. It offers column-oriented data storage, so you have a bit more structure than plain key/value stores. It operates in a distributed, highly available,
2 0.80533719 670 high scalability-2009-08-05-Anti-RDBMS: A list of distributed key-value stores
Introduction: Update 8: Introducing MongoDB by Eliot Horowit . Update 7: The Future of Scalable Databases by Robin Mathew. Update 6: NoSQL : If Only it Was that Easy . BJ Clark lays down the law on which databases are scalable: Tokyo - NO, Redis - NO, Voldemort - YES, MongoDB - Not Yet, Cassandra - Probably, Amazon S3 - YES * 2, MySQL - NO. The real thing to point out is that if you are being held back from making something super awesome because you can’t choose a database, you are doing it wrong. Update 5: Exciting stuff happening in Japan at this Key-Value Storage meeting in Tokyo . Presentations on Groonga, Senna, Lux IO, Tokyo-Cabinet, Tx, repcached, Kai, Cagra, kumofs, ROMA, and Flare. Update 4: NoSQL and the Relational Model: don’t throw the baby out with the bathwater by Matthew Willson. So my key point is, this kind of modelling is WORTH DOING, regardless of which database tool you end up using for physical storage. Update 3: Choosing a non-relational database
3 0.73792374 1022 high scalability-2011-04-13-Paper: NoSQL Databases - NoSQL Introduction and Overview
Introduction: Christof Strauch, from Stuttgart Media University, has written an incredible 120+ page paper titled NoSQL Databases as an introduction and overview to NoSQL databases . The paper was written between 2010-06 and 2011-02, so it may be a bit out of date, but if you are looking to take in the NoSQL world in one big gulp, this is your chance. I asked Christof to give us a short taste of what he was trying to accomplish in his paper: The paper aims at giving a systematic and thorough introduction and overview of the NoSQL field by assembling information dispersed among blogs, wikis and scientific papers. It firstly discusses reasons, rationales and motives for the development and usage of nonrelational database systems. These can be summarized by the need for high scalability, the processing of large amounts of data, the ability to distribute data among many (often commodity) servers, consequently a distribution-aware design of DBMSs. The paper then introduces fundamental concepts,
4 0.69804132 875 high scalability-2010-08-09-NoSQL on the Microsoft Platform
Introduction: NoSQL is a trend that is gaining steam primarily in the world of Open Source. There are numerous NoSQL solutions available for all levels of complexity: from queryable distributed solutions like MongoDB to simpler distributed key-value storage solutions like Cassandra. Then there’s Riak, Tokyo Cabinet, Voldemort, CouchDB, and Redis. However, very few of these packaged NoSQL products are available for the other end of the platform market: Microsoft Windows. I’m going to outline what’s available now and briefly touch on some opportunities that are still available to the daring Microsoft engineer. You can read the full story here .
5 0.67378736 1180 high scalability-2012-01-24-The State of NoSQL in 2012
Introduction: This is a guest post by Siddharth Anand , a senior member of LinkedIn's Distributed Data Systems team. Preamble Ramble If you’ve been working in the online (e.g. internet) space over the past 3 years, you are no stranger to terms like “the cloud” and “NoSQL”. In 2007, Amazon published a paper on Dynamo . The paper detailed how Dynamo, employing a collection of techniques to solve several problems in fault-tolerance, provided a resilient solution to the on-line shopping cart problem. A few years go by while engineers at AWS toil in relative obscurity at standing up their public cloud. It’s December 2008 and I am a member of Netflix’s Software Infrastructure team. We’ve just been told that there is something called the “CAP theorem” and because of it, we are to abandon our datacenter in hopes of leveraging Cloud Computing. Huh? A month into the investigation, we start wondering about our Oracle database. How are we are going to move it into the cloud? That’s when we are
6 0.67195809 872 high scalability-2010-08-05-Pairing NoSQL and Relational Data Storage: MySQL with MongoDB
7 0.6700868 784 high scalability-2010-02-25-Paper: High Performance Scalable Data Stores
8 0.66899639 737 high scalability-2009-11-05-A Yes for a NoSQL Taxonomy
9 0.65840906 770 high scalability-2010-02-03-NoSQL Means Never Having to Store Blobs Again
10 0.6461404 739 high scalability-2009-11-09-10 NoSQL Systems Reviewed
12 0.62963939 1025 high scalability-2011-04-16-The NewSQL Market Breakdown
13 0.62653685 935 high scalability-2010-11-05-Hot Scalability Links For November 5th, 2010
14 0.62442678 787 high scalability-2010-03-03-Hot Scalability Links for March 3, 2010
15 0.61127961 961 high scalability-2010-12-21-SQL + NoSQL = Yes !
16 0.60463715 1054 high scalability-2011-06-06-NoSQL Pain? Learn How to Read-write Scale Without a Complete Re-write
17 0.59499258 793 high scalability-2010-03-10-Saying Yes to NoSQL; Going Steady with Cassandra at Digg
18 0.58784354 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
19 0.58770049 931 high scalability-2010-10-28-Notes from A NOSQL Evening in Palo Alto
20 0.58467513 736 high scalability-2009-11-04-Damn, Which Database do I Use Now?
topicId topicWeight
[(1, 0.173), (2, 0.146), (56, 0.458), (61, 0.086), (79, 0.025)]
simIndex simValue blogId blogTitle
1 0.95665264 1394 high scalability-2013-01-25-Stuff The Internet Says On Scalability For January 25, 2013
Introduction: Sorry, Stuff the Internet Says has been called on the account of a power outage. Gods of rain and tree have interfered with thee. Instead, how about watching a little Python? (that's Monty, not the language)
2 0.88048428 45 high scalability-2007-07-30-Product: SmarterStats
Introduction: SmarterStats provides a solid architecture businesses and individual end users can use to track growth and forecast internet trends. * Track your website's growth and forecast internet trends * Features over 130 report items, plus Geographic Reporting * Log comparison saving 90% of your disk space * Email Reports available in Enterprise Edition * Enhanced data mining available in both editions
same-blog 3 0.82520002 732 high scalability-2009-10-29-Digg - Looking to the Future with Cassandra
Introduction: Digg has been researching ways to scale our database infrastructure for some time now. We’ve adopted a traditional vertically partitioned master-slave configuration with MySQL, and also investigated sharding MySQL with IDDB . Ultimately, these solutions left us wanting. In the case of the traditional architecture, the lack of redundancy on the write masters is painful, and both approaches have significant management overhead to keep running. Since it was already necessary to abandon data normalization and consistency to make these approaches work, we felt comfortable looking at more exotic, non-relational data stores. After considering HBase, Hypertable, Cassandra, Tokyo Cabinet/Tyrant, Voldemort, and Dynomite, we settled on Cassandra . Each system has its own strengths and weaknesses, but Cassandra has a good blend of everything. It offers column-oriented data storage, so you have a bit more structure than plain key/value stores. It operates in a distributed, highly available,
4 0.802109 779 high scalability-2010-02-16-Seven Signs You May Need a NoSQL Database
Introduction: While exploring deep into some dusty old library stacks, I dug up Nostradamus' long lost NoSQL codex. What are the chances? Strangely, it also gave the plot to the next Dan Brown novel, but I left that out for reasons of sanity. About NoSQL, here is what Nosty (his friends call him Nosty) predicted are the signs you may need a NoSQL database... You noticed a lot of your database fields are really serialized complex objects in disguise . Why bother with a RDBMS at all then? Storing serialized objects in a relational database is like being on the pill while trying to get pregnant, a bit counter productive. Just use a schemaless database from the start. Using a standard query language has become too confining . You just want to be free. SQL is so easy, so convenient, and so standard, it's really not a challenge anymore. You need to be different. Then NoSQL is for you. Each has their own completely different query mechanism . Your toolbox only contains a hammer . Hammers wh
5 0.74238956 67 high scalability-2007-08-17-What is the best hosting option?
Introduction: The questions was extracted from: http://highscalability.com/plentyoffish-architecture#comment-126 For startup like Markus, what is the best hosting option (and grow more later)? host your own server or use ISP co-location option? He still has to pay huge money on the bandwidth with that payload, right?
6 0.73495579 941 high scalability-2010-11-15-How Google's Instant Previews Reduces HTTP Requests
7 0.72342789 479 high scalability-2008-12-29-Platform virtualization - top 25 providers (software, hardware, combined)
8 0.72139454 1022 high scalability-2011-04-13-Paper: NoSQL Databases - NoSQL Introduction and Overview
9 0.70218372 854 high scalability-2010-07-09-Hot Scalability Links for July 9, 2010
10 0.70042276 446 high scalability-2008-11-18-Scalability Perspectives #2: Van Jacobson – Content-Centric Networking
11 0.64588571 659 high scalability-2009-07-20-A Scalability Lament
12 0.64374465 759 high scalability-2010-01-11-Strategy: Don't Use Polling for Real-time Feeds
13 0.64101028 1322 high scalability-2012-09-14-Stuff The Internet Says On Scalability For September 14, 2012
14 0.60420609 815 high scalability-2010-04-27-Paper: Dapper, Google's Large-Scale Distributed Systems Tracing Infrastructure
15 0.57220566 245 high scalability-2008-02-12-Product: rPath - Creating and Managing Virtual Appliances
16 0.55099857 1565 high scalability-2013-12-16-22 Recommendations for Building Effective High Traffic Web Software
17 0.54844707 947 high scalability-2010-11-23-Sponsored Post: Imo, Undertone, Joyent, Appirio, Tuenti, CloudSigma, ManageEngine, Site24x7
19 0.51617992 1408 high scalability-2013-02-19-Puppet monitoring: how to monitor the success or failure of Puppet runs
20 0.51151878 1236 high scalability-2012-04-30-Masstree - Much Faster than MongoDB, VoltDB, Redis, and Competitive with Memcached